The sensitivity, specificity and likelihood ratios of binary diagnostic tests are often thought of as being independent of disease prevalence. Empirical studies, however, have frequently revealed substantial variation of these measures for the same diagnostic test in different populations. One reason for this discrepancy is related to the fact that only few diagnostic tests are inherently dichotomous. The majority of tests are based on categorization of individuals according to one or several underlying continuous traits. For these tests, the magnitude of diagnostic misclassification depends not only on the magnitude of the measurement or perception error of the underlying trait(s), but also on the distribution of the underlying trait(s) in the population relative to the diagnostic cutpoint. Since this distribution also determines prevalence of the disease in the population, diagnostic misclassification and disease prevalence are related for this type of test. We assess the variation of various measures of validity of diagnostic tests with disease prevalence for simple models of the distribution of the underlying trait(s) and the measurement or perception error. We illustrate that variation with disease prevalence is typically strong for sensitivity and specificity, and even more so for the likelihood ratios. Although positive and negative predictive values also strongly vary with disease prevalence, this variation is usually less pronounced than one would expect if sensitivity and specificity were independent of disease prevalence.