Abstract
Validity refers to the degree to which evidence and theory support interpretations from a given measurement device. Validity is the most important aspect of measurement in all areas of science. In substance abuse research, validity is sometimes misunderstood and inconsistent with modern validity theory. This chapter provides an overview of modern validity theory. It also highlights conceptual and methodological issues that are important to improving the quality of measures in substance abuse research.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
American Psychiatric Association. (2000). Diagnostic and statistical manual of mental disorders (4th ed.). Washington, DC: Author.
American Psychiatric Association. (2013). Diagnostic and statistical manual of mental disorders (5th ed.). Washington, DC: Author.
American Educational Research Association, American Psychological Association, & National Council on Measurement in Education [AERA/APA/NCME]. (1999). Standards for educational and psychological testing. Washington, DC: American Psychological Association.
Brennan, R. L. (2006). Perspectives on the evolution and future of educational measurement. In R. L. Brennan (Ed.), Educational measurement (4th ed.). Westport, CT: Praeger.
Cizek, G., Bowen, D., & Church, K. (2010). Sources of validity evidence for educational and psychological tests: A follow-up study. Educational and Psychological Measurement, 70(5), 732–743.
Cronbach, L. J. (1951). Coefficient alpha and the internal structure of test. Psychometrika, 16(3), 297–334.
Davis, L. L. (1992). Instrument review: Getting the most from a panel of experts. Applied Nursing Research, 5(4), 194–197.
Derringer, J., Krueger, R. F., Dick, D. M., Agrawal, A., Bucholz, K. K., Foroud, T., et al. (2013). Measurement invariance of DSM-IV alcohol, marijuana and cocaine dependence between community-sampled and clinically overselected studies. Addiction, 108(10), 1767–1776.
Fabrigar, L. R., Wegener, D. T., MacCallum, R. C., & Strahan, E. J. (1999). Evaluating the use of exploratory factor analysis in psychological research. Psychological Methods, 4(3), 272.
Gardner, B., & Tang, V. (2013). Reflecting on non-reflective action: An exploratory think-aloud study of self-report habit measures. British Journal of Health Psychology, 19(2), 258–273.
Green, S. B., Lissitz, R. W., & Mulaik, S. A. (1977). Limitations of coefficient alpha as an index of test unidimensionality1. Educational and Psychological Measurement, 37(4), 827–838.
Hattie, J. (1985). Methodology review: Assessing unidimensionality of tests and items. Applied Psychological Measurement, 9(2), 139–164.
Haynes, S. N., Richard, D., & Kubany, E. S. (1995). Content validity in psychological assessment: A functional approach to concepts and methods. Psychological Assessment, 7(3), 238.
Hevey, D. (2010). Think-aloud methods. In N. Salkind (Ed.), Encyclopedia of research design. Thousand Oaks, CA: SAGE Publications Inc.
Humphreys, K., Kaskutas, L. A., & Weisner, C. (1998). The Alcoholics Anonymous Affiliation Scale: development, reliability, and norms for diverse treated and untreated populations. Alcoholism, Clinical and Experimental Research, 22(5), 974–978.
Jellinek, E. M. (1960). The disease concept of alcoholism. New Haven, CT: Hillhouse.
Kline, P. (1979). Psychometrics and psychology. London, UK: Academic Press.
Kline, R. B. (1998). Principles and practice of structural equation modeling. New York, NY: The Guilford Press.
Marsh, H. W., & Hocevar, D. (1985). Application of confirmatory factor analysis to the study of self-concept: First-and higher order factor models and their invariance across groups. Psychological Bulletin, 97(3), 562.
McGrath, R. E. (2005). Conceptual complexity and construct validity. Journal of Personality Assessment, 85(2), 112–124.
Midanik, L. T., & Hines, A. M. (1991). ‘Unstandard’ ways of answering standard questions: protocol analysis in alcohol survey research. Drug and Alcohol Dependence, 27(3), 245–252.
Messick, S. (1989). Validity. In R. Linn (Ed.), Educational measurement (3rd ed.). Washington, DC: American Council on Education/Macmillan.
Messick, S. (1995). Validity of psychological assessment: Validation of inferences from persons’ responses and performances as scientific inquiry into score meaning. American Psychologist, 50(9), 741–749.
Rindskopf, D., & Rose, T. (1988). Some theory and applications of confirmatory second-order factor analysis. Multivariate Behavioral Research, 23(1), 51–67.
Room, R., Janca, A., Bennett, L. A., Schmidt, L., & Sartorius, N. (1996). WHO cross-cultural applicability research on diagnosis and assessment of substance use disorders: An overview of methods and selected results. Addiction, 91(2), 199–220.
Rubio, D. M., Berg-Weger, M., Tebb, S. S., Lee, E. S., & Rauch, S. (2003). Objectifying content validity: Conducting a content validity study in social work research. Social Work Research, 27(2), 94–104.
Tonigan, J. S., Connors, G. J., & Miller, W. R. (1996). Alcoholics Anonymous Involvement (AAI) scale: Reliability and norms. Psychology of Addictive Behaviors, 10(2), 75–80.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Appendix
Appendix
Suggested readings related to the theoretical and methodological issues for each major source of validity evidence:
-
Content
-
Grant, J. S., & Davis, L. L. (1997). Selection and use of content experts for instrument development. Research in Nursing & Health, 20(3), 269–274.
-
Haynes, S. N., Richard, D., & Kubany, E. S. (1995). Content validity in psychological assessment: A functional approach to concepts and methods. Psychological Assessment, 7(3), 238.
-
Lynn, M. R. (1986). Determination and quantification of content validity. Nursing Research, 35(6), 382–386.
-
Polit, D. F., & Beck, C. T. (2006). The content validity index: Are you sure you know what’s being reported? Critique and recommendations. Research in Nursing & Health, 29(5), 489–497.
-
Rubio, D. M., Berg-Weger, M., Tebb, S. S., Lee, E. S., & Rauch, S. (2003). Objectifying content validity: Conducting a content validity study in social work research. Social Work Research, 27(2), 94–104.
-
-
Response processes
-
Chung, T., & Martin, C. S. (2005). What were they thinking?: Adolescents’ interpretations of DSM-IV alcohol dependence symptom queries and implications for diagnostic validity. Drug and Alcohol Dependence, 80(2), 191–200.
-
Greenfield, T. K. (2000). Ways of measuring drinking patterns and the difference they make: Experience with graduated frequencies. Journal of Substance Abuse, 12(1), 33–49.
-
Hines, A. M. (1993). Linking qualitative and quantitative methods in cross-cultural survey research: Techniques from cognitive science. American Journal of Community Psychology, 21(6), 729–746.
-
Midanik, L. T., Hines, A. M., Greenfield, T. K., & Rogers, J. D. (1999). Face-to-face versus telephone interviews: Using cognitive methods to assess alcohol survey questions. Contemporary Drug Problems, 26, 673.
-
Cheung, M. W. L., & Chan, W. (2002). Reducing uniform response bias with ipsative measurement in multiple-group confirmatory factor analysis. Structural Equation Modeling, 9(1), 55–77.
-
Chen, F. F., Sousa, K. H., & West, S. G. (2005). Teacher’s corner: Testing measurement invariance of second-order factor models. Structural Equation Modeling, 12(3), 471–492.
-
Abbey, A., McAuslan, P., Ross, L. T., & Zawacki, T. (1999). Alcohol expectancies regarding sex, aggression, and sexual vulnerability: Reliability and validity assessment. Psychology of Addictive Behaviors, 13(3), 174.
-
Wiers, R. W., Hoogeveen, K. J., Sergeant, J. A., & Gunning, W. B. (1997). High‐and low‐dose alcohol‐related expectancies and the differential associations with drinking in male and female adolescents and young adults. Addiction, 92(7), 871–888.
-
Widaman, K. F., & Reise, S. P. (1997). Exploring the measurement invariance of psychological instruments: Applications in the substance use domain. In K. J. Bryant, M. Windle, & S. G. West (Eds.), The science of prevention: Methodological advances from alcohol and substance abuse research. Washington, DC: American Psychological Association.
-
Agrawal, A., & Lynskey, M. T. (2007). Does gender contribute to heterogeneity in criteria for cannabis abuse and dependence? Results from the national epidemiological survey on alcohol and related conditions. Drug and Alcohol Dependence, 88(2), 300–307.
-
Bowling, A. (2005). Mode of questionnaire administration can have serious effects on data quality. Journal of Public Health, 27(3), 281–291.
-
Robinson, M. E., Myers, C. D., Sadler, I. J., Riley III, J. L., Kvaal, S. A., & Geisser, M. E. (1997). Bias effects in three common self-report pain assessment measures. The Clinical Journal of Pain, 13(1), 74–81.
-
Rogler, L. H., Mroczek, D. K., Fellows, M., & Loftus, S. T. (2001). The neglect of response bias in mental health research. The Journal of Nervous and Mental Disease, 189(3), 182–187.
-
-
Internal Structure
-
Cortina, J. M. (1993). What is coefficient alpha? An examination of theory and applications. Journal of Applied Psychology, 78(1), 98.
-
Curran, P. J., West, S. G., & Finch, J. F. (1996). The robustness of test statistics to nonnormality and specification error in confirmatory factor analysis. Psychological Methods, 1(1), 16.
-
Fabrigar, L. R., Wegener, D. T., MacCallum, R. C., & Strahan, E. J. (1999). Evaluating the use of exploratory factor analysis in psychological research. Psychological Methods, 4(3), 272.
-
Gerbing, D. W., & Anderson, J. C. (1988). An updated paradigm for scale development incorporating unidimensionality and its assessment. Journal of Marketing Research (JMR), 25(2).
-
Hattie, J. (1985). Methodology review: Assessing unidimensionality of tests and items. Applied Psychological Measurement, 9(2), 139–164.
-
Muthén, B. O., Grant, B., & Hasin, D. (1993). The dimensionality of alcohol abuse and dependence: Factor analysis of DSM‐III‐R and proposed DSM‐IV criteria in the 1988 National Health Interview Survey. Addiction, 88(8), 1079–1090.
-
Reise, S. P., Waller, N. G., & Comrey, A. L. (2000). Factor analysis and scale revision. Psychological Assessment, 12(3), 287.
-
Sijtsma, K. (2009). On the use, the misuse, and the very limited usefulness of Cronbach’s alpha. Psychometrika, 74(1), 107–120.
-
-
Associations with Other Variables
-
Allen, J. P., Litten, R. Z., Fertig, J. B., & Babor, T. (1997). A review of research on the Alcohol Use Disorders Identification Test (AUDIT). Alcoholism: Clinical and Experimental Research, 21(4), 613–619.
-
Ashton, M. C., Jackson, D. N., Paunonen, S. V., Helmes, E., & Rothstein, M. G. (1995). The criterion validity of broad factor scales versus specific facet scales. Journal of Research in Personality, 29(4), 432–442.
-
Campbell, D. T., & Fiske, D. W. (1959). Convergent and discriminant validation by the multitrait-multimethod matrix. Psychological Bulletin, 56(2), 81.
-
Hasin, D., Rossem, R., McCloud, S., & Endicott, J. (1997). Alcohol dependence and abuse diagnoses: Validity in community sample heavy drinkers. Alcoholism: Clinical and Experimental Research, 21(2), 213–219.
-
Hasin, D. S., Schuckit, M. A., Martin, C. S., Grant, B. F., Bucholz, K. K., & Helzer, J. E. (2003). The validity of DSM‐IV alcohol dependence: What Do We Know and What Do We Need to Know? Alcoholism: Clinical and Experimental Research, 27(2), 244–252.
-
Hesselbrock, M., Babor, T. F., Hesselbrock, V., Meyer, R. E., & Workman, K. (1983). “Never believe an alcoholic”? On the validity of self-report measures of alcohol dependence and related constructs. Substance Use & Misuse, 18(5), 593–609.
-
Midanik, L. T. (1988). Validity of self‐reported alcohol use: A literature review and assessment. British Journal of Addiction, 83(9), 1019–1029.
-
Stacy, A. W., Widaman, K. F., Hays, R., & DiMatteo, M. R. (1985). Validity of self-reports of alcohol and other drug use: A multitrait-multimethod assessment. Journal of Personality and Social Psychology, 49(1), 219.
-
Rights and permissions
Copyright information
© 2017 Springer International Publishing AG
About this chapter
Cite this chapter
Perron, B.E., Cordova, D., Salas-Wright, C., Vaughn, M.G. (2017). Validity: Conceptual and Methodological Issues in Substance Abuse Research. In: VanGeest, J., Johnson, T., Alemagno, S. (eds) Research Methods in the Study of Substance Abuse. Springer, Cham. https://doi.org/10.1007/978-3-319-55980-3_12
Download citation
DOI: https://doi.org/10.1007/978-3-319-55980-3_12
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-55978-0
Online ISBN: 978-3-319-55980-3
eBook Packages: Social SciencesSocial Sciences (R0)