Abstract
Background
Electronic administration of clinician-reported outcomes (eClinROs) has advantages over paper-based methods, but the mode of administration change has the potential to affect the validity of the scale. The literature on migration of patient-reported outcomes (PROs) suggests that there are different levels of modification, which necessitate different approaches to demonstrating mode equivalence. However, little has been written on the migration of ClinROs to electronic administration.
Methods
We propose a method of comparing paper and electronic versions of scales that includes a comparison based on content and a comparison based on format. The determination of whether the eClinRO has undergone minor, moderate, or substantial modification will drive the necessary studies required for validation.
Results
The unique characteristics of ClinROs suggest 2 additional types of modifications, including functionality adaptation and adaptation of instructions.
Conclusions
In many respects, the migration of a ClinRO to electronic administration is similar to that of a PRO. This article has explored the ways in which there might be special considerations for ClinROs that have not been elaborated for PROs.
Article PDF
Similar content being viewed by others
Avoid common mistakes on your manuscript.
References
US Food and Drug Administration. Definitions. http://www.fda.gov/drugs/developmentapprovalprocess/drugdevelopmenttoolsqualificationprogram/ucm284395.htm.
US Food and Drug Administration. Office of the Commissioner. Guidance for Industry Computerized Systems Used in Clinical Investigations. Rockland, MD: US Department of Health and Human Services; 2007.
Ene-Iordache B, Carminati S, Antiga L, et al. Developing regulatory-compliant electronic case report forms for clinical trials: experience with the Demand Trial. J Am Med Inform Assoc. 2009;16:404–408. doi:https://doi.org/10.1197/jamia.M2787.
Center for Drug Evaluation and Research (US); Center for Biologics Evaluation and Research (US); Center for Devices and Radiological Health (US). Guidance for Industry Patient Reported Outcome Measures: Use in Medical Product Development to Support Labeling Claims. Rockland, MD: US Department of Health and Human Services; 2009.
Bauer RM, Iverson GL, Cernich AN, Binder LM, Ruff RM, Naugle RI. Computerized neuropsychological assessment devices: joint position paper of the American Academy of Clinical Neuropsychology and the National Academy of Neuropsychology. Arch Clin Neuropsychol. 2012;27:362–373. doi:https://doi.org/10.1093/arclin/acs027.
O’Halloran JP, Kemp AS, Salmon DP, Tariot PN, Schneider LS. Psychometric comparison of standard and computerized administration of the Alzheimer’s Disease Assessment Scale: Cognitive Subscale (ADASCog). Curr Alzheimer Res. 2011;8:323–328.
Moore RC, Harmell AL, Ho J, et al. NIH public access. Schizophr Res. 2013;144:87–92. doi:https://doi.org/10.1016/j.schres.2012.12.028.Initial.
US Food and Drug Administration. Guidance for Industry Electronic Source Data in Clinical Investigations; Availability. Rockland, MD: US Department of Health and Human Services; 2013.
Le Jeannic A, Quelen C, Alberti C, Durand-Zaleski I. Comparison of two data collection processes in clinical studies: electronic and paper case report forms. BMC Med Res Methodol. 2014;14:7. http://www.biomedcentral.com/1471-2288/14/7.
US Food and Drug Administration 21 CFR Part 11: Electronic Records; Electronic Signatures. http://www.accessdata.fda.gov/scripts/cdrh/cfdocs/cfcfr/CFRSearch.cfm?CFRPart=11&showFR=1. Published 1997.
GCP Inspectors Working Group. Reflection Paper on Expectations for Electronic Source Data and Data Transcribed to Electronic Data Collection Tools in Clinical Trials. 2010.
Gwaltney CJ, Shields AL, Shiffman S. Equivalence of electronic and paper-and-pencil administration of patient-reported outcome measures: a meta-analytic review. Value Health. 2008;11:322–333. doi:https://doi.org/10.1111/j.1524-4733.2007.00231.x.
Abernethy AP, Herndon JE, Wheeler JL, et al. Improving health care efficiency and quality using tablet personal computers to collect research-quality, patient-reported data. Health Serv Res. 2008;43:1975–1991. doi:https://doi.org/10.1111/j.1475-6773.2008.00887.x.
Coons SJ, Gwaltney CJ, Hays RD, et al. Recommendations on evidence needed to support measurement equivalence between electronic and paper-based patient-reported outcome (PRO) measures: ISPOR ePRO good research practices task force report. Value Heal. 2009;12(4):419–429.
Willis GB. Cognitive Interviewing: A Tool for Improving Questionnaire Design. Thousand Oaks, CA: Sage; 2005.
Shrout PE, Fleiss JL. Intraclass correlations: uses in assessing rater reliability. Psychol Bull. 1979;86:420–428.
Hallgren KA. Computing inter-rater reliability for observational data: an overview and tutorial. Tutor Quant Methods Psychol. 2012;8:23–34. doi:https://doi.org/10.1016/j.biotechadv.2011.08.021.Secreted.
Cappelleri JC, Zou KH, Bushmakin AG, Alvir JMJ, Alemayehu D, Symonds T. Patient-Reported Outcomes: Measurement, Implementation and Interpretation. Boca Raton, FL: Chapman & Hall/CRC Press; 2013. http://books.google.com/books?id=vzUTAgAAQBAJ&pgis=1.
Cohen J. A coefficient of agreement for nominal scales. Educ Psychol Meas. 1960;20:37–46.
Cohen J. Weighted kappa: nominal scale agreement provision for scaled disagreement or partial credit. Psychol Bull. 1968;70:213–220.
McGraw KO, Wong SP. Forming inferences about some intraclass correlation coefficients. Psychol Methods. 1996;1:30–46.
McDowell I. Measuring Health: A Guide to Rating Scales and Questionnaires. 3rd ed. Oxford: Oxford University Press; 2006.
Cohen J. Quantitative methods in psychology: a power primer. Psychol Bull. 1992;112:155–159. doi:https://doi.org/10.1038/141613a0.
Cohen J. Statistical Power analysis. Curr Dir Psychol Sci. 1992;1:98–101.
Author information
Authors and Affiliations
Corresponding author
Electronic supplementary material
Rights and permissions
About this article
Cite this article
Fuller, R.L.M., McNamara, C.W., Lenderking, W.R. et al. Establishing Equivalence of Electronic Clinician-Reported Outcome Measures. Ther Innov Regul Sci 50, 30–36 (2016). https://doi.org/10.1177/2168479015618693
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1177/2168479015618693