Skip to main content
SpringerLink
Log in

Validity: Conceptual and Methodological Issues in Substance Abuse Research

  • Chapter
  • First Online:
Research Methods in the Study of Substance Abuse

Abstract

Validity refers to the degree to which evidence and theory support interpretations from a given measurement device. Validity is the most important aspect of measurement in all areas of science. In substance abuse research, validity is sometimes misunderstood and inconsistent with modern validity theory. This chapter provides an overview of modern validity theory. It also highlights conceptual and methodological issues that are important to improving the quality of measures in substance abuse research.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 129.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 169.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 169.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

References

  • American Psychiatric Association. (2000). Diagnostic and statistical manual of mental disorders (4th ed.). Washington, DC: Author.

    Google Scholar 

  • American Psychiatric Association. (2013). Diagnostic and statistical manual of mental disorders (5th ed.). Washington, DC: Author.

    Book  Google Scholar 

  • American Educational Research Association, American Psychological Association, & National Council on Measurement in Education [AERA/APA/NCME]. (1999). Standards for educational and psychological testing. Washington, DC: American Psychological Association.

    Google Scholar 

  • Brennan, R. L. (2006). Perspectives on the evolution and future of educational measurement. In R. L. Brennan (Ed.), Educational measurement (4th ed.). Westport, CT: Praeger.

    Google Scholar 

  • Cizek, G., Bowen, D., & Church, K. (2010). Sources of validity evidence for educational and psychological tests: A follow-up study. Educational and Psychological Measurement, 70(5), 732–743.

    Google Scholar 

  • Cronbach, L. J. (1951). Coefficient alpha and the internal structure of test. Psychometrika, 16(3), 297–334.

    Google Scholar 

  • Davis, L. L. (1992). Instrument review: Getting the most from a panel of experts. Applied Nursing Research, 5(4), 194–197.

    Article  Google Scholar 

  • Derringer, J., Krueger, R. F., Dick, D. M., Agrawal, A., Bucholz, K. K., Foroud, T., et al. (2013). Measurement invariance of DSM-IV alcohol, marijuana and cocaine dependence between community-sampled and clinically overselected studies. Addiction, 108(10), 1767–1776.

    Article  Google Scholar 

  • Fabrigar, L. R., Wegener, D. T., MacCallum, R. C., & Strahan, E. J. (1999). Evaluating the use of exploratory factor analysis in psychological research. Psychological Methods, 4(3), 272.

    Article  Google Scholar 

  • Gardner, B., & Tang, V. (2013). Reflecting on non-reflective action: An exploratory think-aloud study of self-report habit measures. British Journal of Health Psychology, 19(2), 258–273.

    Article  Google Scholar 

  • Green, S. B., Lissitz, R. W., & Mulaik, S. A. (1977). Limitations of coefficient alpha as an index of test unidimensionality1. Educational and Psychological Measurement, 37(4), 827–838.

    Article  Google Scholar 

  • Hattie, J. (1985). Methodology review: Assessing unidimensionality of tests and items. Applied Psychological Measurement, 9(2), 139–164.

    Article  Google Scholar 

  • Haynes, S. N., Richard, D., & Kubany, E. S. (1995). Content validity in psychological assessment: A functional approach to concepts and methods. Psychological Assessment, 7(3), 238.

    Article  Google Scholar 

  • Hevey, D. (2010). Think-aloud methods. In N. Salkind (Ed.), Encyclopedia of research design. Thousand Oaks, CA: SAGE Publications Inc.

    Google Scholar 

  • Humphreys, K., Kaskutas, L. A., & Weisner, C. (1998). The Alcoholics Anonymous Affiliation Scale: development, reliability, and norms for diverse treated and untreated populations. Alcoholism, Clinical and Experimental Research, 22(5), 974–978.

    Article  Google Scholar 

  • Jellinek, E. M. (1960). The disease concept of alcoholism. New Haven, CT: Hillhouse.

    Book  Google Scholar 

  • Kline, P. (1979). Psychometrics and psychology. London, UK: Academic Press.

    Google Scholar 

  • Kline, R. B. (1998). Principles and practice of structural equation modeling. New York, NY: The Guilford Press.

    Google Scholar 

  • Marsh, H. W., & Hocevar, D. (1985). Application of confirmatory factor analysis to the study of self-concept: First-and higher order factor models and their invariance across groups. Psychological Bulletin, 97(3), 562.

    Article  Google Scholar 

  • McGrath, R. E. (2005). Conceptual complexity and construct validity. Journal of Personality Assessment, 85(2), 112–124.

    Article  Google Scholar 

  • Midanik, L. T., & Hines, A. M. (1991). ‘Unstandard’ ways of answering standard questions: protocol analysis in alcohol survey research. Drug and Alcohol Dependence, 27(3), 245–252.

    Article  Google Scholar 

  • Messick, S. (1989). Validity. In R. Linn (Ed.), Educational measurement (3rd ed.). Washington, DC: American Council on Education/Macmillan.

    Google Scholar 

  • Messick, S. (1995). Validity of psychological assessment: Validation of inferences from persons’ responses and performances as scientific inquiry into score meaning. American Psychologist, 50(9), 741–749.

    Article  Google Scholar 

  • Rindskopf, D., & Rose, T. (1988). Some theory and applications of confirmatory second-order factor analysis. Multivariate Behavioral Research, 23(1), 51–67.

    Article  Google Scholar 

  • Room, R., Janca, A., Bennett, L. A., Schmidt, L., & Sartorius, N. (1996). WHO cross-cultural applicability research on diagnosis and assessment of substance use disorders: An overview of methods and selected results. Addiction, 91(2), 199–220.

    Article  Google Scholar 

  • Rubio, D. M., Berg-Weger, M., Tebb, S. S., Lee, E. S., & Rauch, S. (2003). Objectifying content validity: Conducting a content validity study in social work research. Social Work Research, 27(2), 94–104.

    Article  Google Scholar 

  • Tonigan, J. S., Connors, G. J., & Miller, W. R. (1996). Alcoholics Anonymous Involvement (AAI) scale: Reliability and norms. Psychology of Addictive Behaviors, 10(2), 75–80.

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. School of Social Work, University of Michigan, Ann Arbor, MI, USA

    Brian E. Perron & David Cordova

  2. School of Social Work, The University of Texas at Austin, Austin, TX, USA

    Christopher Salas-Wright

  3. School of Social Work, Saint Louis University, Saint Louis, MO, USA

    Michael G. Vaughn

Authors
  1. Brian E. Perron
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. David Cordova
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Christopher Salas-Wright
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Michael G. Vaughn
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Brian E. Perron .

Editor information

Editors and Affiliations

  1. Department of Health Policy and Management, College of Public Health, Kent State University, Kent, Ohio, USA

    Jonathan B. VanGeest

  2. Survey Research Laboratory, Department of Public Administration, College of Urban Planning and Public Affairs, University of Illinois at Chicago, Chicago, Illinois, USA

    Timothy P. Johnson

  3. Department of Health Policy and Management, College of Public Health, Kent State University, Kent, Ohio, USA

    Sonia A. Alemagno

Appendix

Appendix

Suggested readings related to the theoretical and methodological issues for each major source of validity evidence:

  • Content

    • Grant, J. S., & Davis, L. L. (1997). Selection and use of content experts for instrument development. Research in Nursing & Health, 20(3), 269–274.

    • Haynes, S. N., Richard, D., & Kubany, E. S. (1995). Content validity in psychological assessment: A functional approach to concepts and methods. Psychological Assessment, 7(3), 238.

    • Lynn, M. R. (1986). Determination and quantification of content validity. Nursing Research, 35(6), 382–386.

    • Polit, D. F., & Beck, C. T. (2006). The content validity index: Are you sure you know what’s being reported? Critique and recommendations. Research in Nursing & Health, 29(5), 489–497.

    • Rubio, D. M., Berg-Weger, M., Tebb, S. S., Lee, E. S., & Rauch, S. (2003). Objectifying content validity: Conducting a content validity study in social work research. Social Work Research, 27(2), 94–104.

  • Response processes

    • Chung, T., & Martin, C. S. (2005). What were they thinking?: Adolescents’ interpretations of DSM-IV alcohol dependence symptom queries and implications for diagnostic validity. Drug and Alcohol Dependence, 80(2), 191–200.

    • Greenfield, T. K. (2000). Ways of measuring drinking patterns and the difference they make: Experience with graduated frequencies. Journal of Substance Abuse, 12(1), 33–49.

    • Hines, A. M. (1993). Linking qualitative and quantitative methods in cross-cultural survey research: Techniques from cognitive science. American Journal of Community Psychology, 21(6), 729–746.

    • Midanik, L. T., Hines, A. M., Greenfield, T. K., & Rogers, J. D. (1999). Face-to-face versus telephone interviews: Using cognitive methods to assess alcohol survey questions. Contemporary Drug Problems, 26, 673.

    • Cheung, M. W. L., & Chan, W. (2002). Reducing uniform response bias with ipsative measurement in multiple-group confirmatory factor analysis. Structural Equation Modeling, 9(1), 55–77.

    • Chen, F. F., Sousa, K. H., & West, S. G. (2005). Teacher’s corner: Testing measurement invariance of second-order factor models. Structural Equation Modeling, 12(3), 471–492.

    • Abbey, A., McAuslan, P., Ross, L. T., & Zawacki, T. (1999). Alcohol expectancies regarding sex, aggression, and sexual vulnerability: Reliability and validity assessment. Psychology of Addictive Behaviors, 13(3), 174.

    • Wiers, R. W., Hoogeveen, K. J., Sergeant, J. A., & Gunning, W. B. (1997). High‐and low‐dose alcohol‐related expectancies and the differential associations with drinking in male and female adolescents and young adults. Addiction, 92(7), 871–888.

    • Widaman, K. F., & Reise, S. P. (1997). Exploring the measurement invariance of psychological instruments: Applications in the substance use domain. In K. J. Bryant, M. Windle, & S. G. West (Eds.), The science of prevention: Methodological advances from alcohol and substance abuse research. Washington, DC: American Psychological Association.

    • Agrawal, A., & Lynskey, M. T. (2007). Does gender contribute to heterogeneity in criteria for cannabis abuse and dependence? Results from the national epidemiological survey on alcohol and related conditions. Drug and Alcohol Dependence, 88(2), 300–307.

    • Bowling, A. (2005). Mode of questionnaire administration can have serious effects on data quality. Journal of Public Health, 27(3), 281–291.

    • Robinson, M. E., Myers, C. D., Sadler, I. J., Riley III, J. L., Kvaal, S. A., & Geisser, M. E. (1997). Bias effects in three common self-report pain assessment measures. The Clinical Journal of Pain, 13(1), 74–81.

    • Rogler, L. H., Mroczek, D. K., Fellows, M., & Loftus, S. T. (2001). The neglect of response bias in mental health research. The Journal of Nervous and Mental Disease, 189(3), 182–187.

  • Internal Structure

    • Cortina, J. M. (1993). What is coefficient alpha? An examination of theory and applications. Journal of Applied Psychology, 78(1), 98.

    • Curran, P. J., West, S. G., & Finch, J. F. (1996). The robustness of test statistics to nonnormality and specification error in confirmatory factor analysis. Psychological Methods, 1(1), 16.

    • Fabrigar, L. R., Wegener, D. T., MacCallum, R. C., & Strahan, E. J. (1999). Evaluating the use of exploratory factor analysis in psychological research. Psychological Methods, 4(3), 272.

    • Gerbing, D. W., & Anderson, J. C. (1988). An updated paradigm for scale development incorporating unidimensionality and its assessment. Journal of Marketing Research (JMR), 25(2).

    • Hattie, J. (1985). Methodology review: Assessing unidimensionality of tests and items. Applied Psychological Measurement, 9(2), 139–164.

    • Muthén, B. O., Grant, B., & Hasin, D. (1993). The dimensionality of alcohol abuse and dependence: Factor analysis of DSM‐III‐R and proposed DSM‐IV criteria in the 1988 National Health Interview Survey. Addiction, 88(8), 1079–1090.

    • Reise, S. P., Waller, N. G., & Comrey, A. L. (2000). Factor analysis and scale revision. Psychological Assessment, 12(3), 287.

    • Sijtsma, K. (2009). On the use, the misuse, and the very limited usefulness of Cronbach’s alpha. Psychometrika, 74(1), 107–120.

  • Associations with Other Variables

    • Allen, J. P., Litten, R. Z., Fertig, J. B., & Babor, T. (1997). A review of research on the Alcohol Use Disorders Identification Test (AUDIT). Alcoholism: Clinical and Experimental Research, 21(4), 613–619.

    • Ashton, M. C., Jackson, D. N., Paunonen, S. V., Helmes, E., & Rothstein, M. G. (1995). The criterion validity of broad factor scales versus specific facet scales. Journal of Research in Personality, 29(4), 432–442.

    • Campbell, D. T., & Fiske, D. W. (1959). Convergent and discriminant validation by the multitrait-multimethod matrix. Psychological Bulletin, 56(2), 81.

    • Hasin, D., Rossem, R., McCloud, S., & Endicott, J. (1997). Alcohol dependence and abuse diagnoses: Validity in community sample heavy drinkers. Alcoholism: Clinical and Experimental Research, 21(2), 213–219.

    • Hasin, D. S., Schuckit, M. A., Martin, C. S., Grant, B. F., Bucholz, K. K., & Helzer, J. E. (2003). The validity of DSM‐IV alcohol dependence: What Do We Know and What Do We Need to Know? Alcoholism: Clinical and Experimental Research, 27(2), 244–252.

    • Hesselbrock, M., Babor, T. F., Hesselbrock, V., Meyer, R. E., & Workman, K. (1983). “Never believe an alcoholic”? On the validity of self-report measures of alcohol dependence and related constructs. Substance Use & Misuse, 18(5), 593–609.

    • Midanik, L. T. (1988). Validity of self‐reported alcohol use: A literature review and assessment. British Journal of Addiction, 83(9), 1019–1029.

    • Stacy, A. W., Widaman, K. F., Hays, R., & DiMatteo, M. R. (1985). Validity of self-reports of alcohol and other drug use: A multitrait-multimethod assessment. Journal of Personality and Social Psychology, 49(1), 219.

Rights and permissions

Reprints and permissions

Copyright information

© 2017 Springer International Publishing AG

About this chapter

Cite this chapter

Perron, B.E., Cordova, D., Salas-Wright, C., Vaughn, M.G. (2017). Validity: Conceptual and Methodological Issues in Substance Abuse Research. In: VanGeest, J., Johnson, T., Alemagno, S. (eds) Research Methods in the Study of Substance Abuse. Springer, Cham. https://doi.org/10.1007/978-3-319-55980-3_12

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-55980-3_12

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-55978-0

  • Online ISBN: 978-3-319-55980-3

  • eBook Packages: Social SciencesSocial Sciences (R0)

Publish with us

Policies and ethics

Search

Navigation