Keywords

At Vanderbilt University Medical Center, an important refrain from our information technology (IT) colleagues is that IT solutions cannot fix underlying flaws in process or function. For IT to be innovative, useful, and easily adopted—the key is to have a well-functioning and reliable process. To that end, in order to have tools supportive of DSM 5, we first need to start with an understanding of the validity and reliability of psychiatric diagnoses. Inherent to the success of what can be built is the ability to maintain both the face validity and test validity of the diagnostic schema. We begin this chapter with a discussion regarding the development of DSM 5. We then consider how technology can support accurate diagnosis and treatment planning.

10.1 Background/History

Throughout the history of psychiatry, the foundation of diagnosis has remained consistent: observation, clinical interview, and judgment [1]. The earliest tools available to begin the categorization and classification for psychiatric illness consisted of the observation of a disturbance in behavior [2]. The identified psychiatric patient was then interviewed, focusing on the nature and etiology of the apparent behavior. Finally, it was incumbent upon the expert diagnostician to utilize knowledge of existing psychiatric diagnostic systems to apply them in a common sense fashion to the patient whom had been observed and interviewed. The obvious limitations of this approach consisted of how to demonstrate the reliability and validity of a psychiatric diagnosis as a construct. More specifically, how would we know that the constellation of symptoms that this expert called bipolar disorder, for example, would indeed be diagnosed as bipolar disorder by the next astute clinician? In addition, how would we know that what they are describing really was bipolar disorder, and not simply two experts concurrently misdiagnosing borderline personality disorder?

Psychiatric classification, or the creation of diagnostic systems, arose from this problem. Emil Kraeplin developed the earliest major diagnostic system, Compendium der Psychiatrie, in 1883 [3]. His diagnostic nosology pioneered several concepts fundamental to future psychiatric diagnostic systems. First was that psychiatric illness was to be held as a disease of the brain and nervous system. In addition to its pathology beginning in the brain and nervous system, he posited that psychiatric illness was naturally occurring and degenerative. In regards to specific disease conditions and differential diagnosis, he defined manic-depression and dementia praecox as distinct illnesses. Kraeplin developed successive versions of his textbook and was working on the ninth edition when he died in 1926 [3].

10.2 Development of the Diagnostic and Statistical Manual (DSM)

In the United States, the major diagnostic classification system emerging in the twentieth century was the Diagnostic and Statistical Manual (DSM) [4]. The DSM represented an effort to form consensus in the US around diagnostic validity and reliability. It developed out of an 80-year history in the US focused on the gathering of statistics on mental health diagnosis, stemming from a recording in the 1840 census of the frequency of “idiocy” and “insanity” and also establishing how many were “at public or private charge” [5]. By 1952, the American Psychiatric Association published the first edition of the Diagnostic and Statistical Manual as an offshoot of the International Classification of Diseases sixth edition (ICD-6) [4]. In 1994, the DSM-IV was published, with a major revision in 2000—the DSM-IV TR. As a result of the concerns about a new edition of the DSM in the wake of the publication of the DSM-III in 1980, the DSM-IV was designed with a number of “procedural safeguards … instituted to minimize arbitrary and idiosyncratic revisions” [6].

These safeguards consisted of process oriented changes: (1) expert advisers were appointed to each of the DSM diagnostic workgroups on illness categories, (2) methods conferences were utilized to review methodological issues facing the development of the DSM, (3) specific change criteria were developed for the diagnoses under review, and (4) a balanced review of literature with inclusion of the body of evidence both supporting and opposing the diagnostic constructs that had been developed in the DSM III [6]. These reviews were intended to be “descriptive, comprehensive, explicit, and systematic. The goal [was] not to generate the data to argue for a certain position, but rather to provide a fair, balanced, and descriptive summary of the literature” [6]. This review then forms the foundation of the three-step process of the empirical review that leads directly to the development of diagnostic criteria, as well as the inclusion or exclusion of diagnostic constructs from the manual.

To complete the process, the work groups developed a standard that reanalysis of existing but unanalyzed data sets and ultimately field trials are required [7]. The DSM-IV field trials were conducted for 12 diagnostic constructs: Antisocial Personality Disorder, Autism and Pervasive Developmental Disorders, Disruptive Behavior Disorders (including Conduct Disorder, Attention Deficit Hyperactivity Disorder [8], and Oppositional Defiant Disorder), Insomnia, Major Depression and Dysthymia, Mixed Anxiety-Depression, Panic Disorder, Obsessive-Compulsive Disorder, Posttraumatic Stress Disorder, Schizophrenia, Somatization Disorder, and Substance Use Disorders [9]. Practically speaking, the field trials represented an effort to capture how psychopathology appeared to the clinician. An effort was made to ensure a variety of clinical sites, generally between 5 and 10 in number, as well as a mix of new and follow-up patients. Definitions also included the use of standardized assessment tools such as the Yale-Brown Obsessive Compulsive Scale, the Fixity of Beliefs Scale and the Structured Clinical Interview for DSM Disorders (SCID) for patients being assessed for obsessive compulsive disorder [10].

Despite these advances in the development of DSM-IV, concerns regarding the validity and reliability of the diagnostic system arose. Specific concerns were raised regarding the validity of the distinction between mania and schizoaffective disorder [11] and between depression and anxiety [12]. The validity of the construct of specific diagnoses were also questioned, such as for bipolar disorder [13] and seasonal affective disorder [14]. The news was not all bad, as evidence and research emerged supporting diagnostic constructs from the DSM-IV, such as for Psychotic Depression [15]. More fundamental research was also conducted which investigated the diagnostic structure and hierarchy of the DSM-IV as well as the diagnostic reliability and validity of Axis V [16].

10.2.1 Development of the DSM-5

In the planning of the DSM-5 revision, specific attention was given to address concerns regarding previous editions. Research into the validity and reliability of the DSM-IV diagnostic constructs revealed problems regarding test-retest reliability [17]. There was also the logistical challenge of accurate data collection across thousand of patients and multiple centers, compilation and analysis of that data in an expedient fashion, and the application of the most current advances in statistical measures of reliability and validity. The process leading up to the field trials of the DSM 5 was similar to that of the DSM-IV.

In 1999, an initial planning conference involving thought leaders and experts in research was convened to determine the research direction heading into DSM 5 development [18]. Interestingly, leaders of the DSM-IV process typically were excluded so as to foster creativity and an objective look at the challenges which had arisen in the previous edition. Work groups were convened, research agendas were developed and set, and white papers were ultimately published in “A Research Agenda for DSM 5” [19] and “Age and Gender Considerations in Psychiatric Diagnosis” [20]. Thirteen conferences were subsequently held in which specific research questions were addressed and presented by content experts; these findings were also subsequently published [21].

A task force was convened to review the DSM-IV, compare the findings with the research generated by the research conferences, and this group developed the drafts of the DSM 5 [18]. Concurrently, from 2010 through 2012, field trials were undertaken [22]. There were two phases of field trials; the first consisted of Field Trial Testing in Large, Academic Medical Centers in which 11 Academic Centers participated. Data collection took approximately 10 months to complete. The second phase consisted of Field Trial Testing in Routine Clinical Practices, which were composed of solo and small group practices, randomly selected from an AMA Database of physicians. Data collection for this group took approximately 8 months.

The challenges for this version of the DSM included the greater number of patients, large academic medical centers participating, and the exponentially greater number of routine clinical practices taking part in the field trials. The design group also planned to take advantage of improved statistical technology developed subsequent to the publication of the DSM-IV. The complexity of this project greatly eclipsed that of the previous edition. The technology tools supportive of this updated process (data collection, sampling strategy, and data analysis) are described in several publications by the implementation workgroup [23–25].

There were 11 identified sites, 7 adult and 4 pediatric [23]. There were 2,246 patients enrolled in the field trials throughout the study. In contrast with the typical prospective double blind randomized studies characteristic of pharmaceutical trials, the field trials were designed to test the reliability of diagnosis in the “real world”—the degree to which two examiners could agree on a diagnosis across a variety of settings ranging from the solo practitioner to the large academic medical center, and in a patient population in which comorbidity was common.

The methodology used to simulate the use of the manual by the clinician included: clinical interview approaches versus structured research interviews; separate interviewers interviewing the same patients at short intervals of separation; inclusion and stratification of patients into multiple diagnostic groups for study to account for comorbidity; assessment of “cross-cutting symptoms” which could indicate another diagnosis, or dimension to the present diagnosis [24], and assess if diagnoses held up under these conditions. 86 % of all enrolled patients were interviewed twice, and a total of 279 clinicians of varied disciplines were involved in the total study. 33 total diagnoses were tested, on average about two trials per diagnosis, and each site studied approximately 5 diagnoses [23].

Furthermore, the DSM 5 Field Trial utilized centrally designed protocols that required uniformity of data collection and data analysis. The project’s main challenge in terms of data collection placed a significant strain on available technology. The main areas of tension included central protocol development and implementation, data collection, and ongoing monitoring of the field trials, with the potential for rapid assimilation and interpretation of the data. These project needs required the use of an adaptable, widely distributable data capture system, ideally using internet protocols for availability and ease [23].

The study designers elected to use The National Institutes of Health-funded Research Electronic Data Capture system (REDCap) developed through Vanderbilt University [23]. The REDCap system, described by the designers in a seminal article [26], is an open source program developed to solve problems around sharing research data with multiple collaborators at multiple sites via high-speed network sharing, while maintaining a high degree of security. The conceptualization of research while incorporating the REDCap tool involves the Informatics team at the outset of the project design. They would demonstrate how REDCap works, including use of the web interface, security, validation, statistical export, and a data collection strategy for the project. Case report forms are created in a format familiar to the researcher, typically an excel spreadsheet, which are populated by the research team around the specific goals, data, and other project requirements. This then forms the foundation of a web-based electronic data collection application [26].

The next step in project design involves the user interface. Data on study variables can be entered into the web-based application by text field, drop down menu, or other .html object design, and then exported either in part or by whole for analysis. According to the designers:

“[The project] uses PHP + JavaScript programming languages, and a MySQL database engine for data storage and manipulation. Hardware and software requirements are modest, and the system runs on Windows/IIS and Linux/Apache web server environments” [26].

Each project also contains significant and flexible data useful to the project as a whole outside of the data collection and analysis. This includes a log of all data transfers, researcher rights, and any ancillary forms required such as consents. REDCap was developed and released within the Vanderbilt University research environment in 2004, and at the time of the paper [26] included 204 active projects, and the total number of subjects in all databases exceeded 17,000. In 2006, the REDCap project was released in a pilot program to partner institutions, which had grown to 27 total partners in 2009. According to the REDCap website [27], as of 2014, the total number of projects was 72,643, and the REDCap Consortium was composed of 1,108 institutional partners from 83 countries. The group had also developed an Online Designer via a web interface for easier access for remote partners, and they note that both surveys and databases can be created.

The designer group emphasizes the flexibility and portability of REDCap in terms of requirements. The hardware requirements are listed as a Web Server with PHP, MySQL Database Server, and SMTP email server (installed if emails need to be sent directly out of REDCap), and an optional file server [26]. These can be running on the same or separate computer[s]. There are no hard requirements for processing speed, memory, clock speed, or hard drive space, as the total program space is approximately 10 MB. 20 GB total are recommended to be dedicated to the web server and MySQL, which should suffice for approximately a year of use. Being open source, there is no cost for the program [26].

Prior to the implementation of the REDCap system for the entire DSM 5 field trials, two pilot studies were done at Johns Hopkins in the Community Psychiatry Outpatient Program, and in the Child and Adolescent Outpatient Program at the Johns Hopkins Bayview Medical Center [28]. These pilot studies took place over an 8 week period with between 10 and 20 patients per stratum, and, identical to the eventual field trials, included two study visits for test and retest validity separated by between 4 h and 2 weeks. On the adult side, the pilot included Major Depressive Disorder, Bipolar Disorder, Schizophrenia, Schizoaffective Disorder. On the Child and Adolescent side, the pilot included Major Depressive Disorder, Disruptive Mood Dis-regulation Disorder, Oppositional Defiant Disorder, Generalized Anxiety Disorder. The total number of adult patients was 100, and the total pediatric patients was 50.

The REDCap system could flexibly be developed to suit the needs of the study or survey. In pilot field trials, the main modifications were around permitting the embedded research coordinator to oversee the study progress and contained a program that facilitated communication of the data entered in real time, as well as, in communicating the data with a central management system. The types of data that the patients (or parents) could enter were composed of self-administered and clinician scored metrics. These included proposed DSM 5 cross cutting symptom measures, the World Health Organization Disability Assessment Schedule II, and the Personality Inventory for DSM-5. Similarly, clinician driven data consisted of the proposed DSM 5 metrics for Suicide Risk in Teens, Suicide Concerns in Adults, Psychosis, Early Development and Home Background, Clinical Utility and a 6 item World Health Organization Disability Assessment Schedule [28].

Anonymous subject tracking was managed by assigning Patient Identification Numbers (PID) and Clinician Identification Numbers (CID). The CID data was used as a login for the system, and permitted data entry, access to reports and blindness to the ratings of other clinicians. The Research Coordinator was permitted access to a Patient Research Screening Form and to patient tracking forms, and the component on REDCap was designed to allow the Coordinator to identify additional fields to make the recruitment process more efficient. The Research Coordinator assisted patients and parents in the self-administered section of the assessment, and then in real time was able to download the results and make them available for study clinicians to review prior to the diagnostic assessment of the patient.

The pilot study results regarding the inclusion of REDCap was felt to be successful, with findings that most clinicians responding that a single 2–3 h training session combined with sufficient practice would result in feeling comfortable managing the data entry system. The REDCap system by necessity compelled clinicians to enter data via a checklist to qualify or disqualify for diagnoses and most study participants felt that greater automation in the entry of data would be helpful. Finally, at the time of the pilot study, a feature of REDCap which makes it attractive for study design—the coordination and communication of workflow between patients, clinicians, and research coordinator—was not available, and a second calendar system was implemented to help with the logistics of scheduling to enhance recruitment [28].

In summary, the logistical challenges around creating and coordinating a multi-site system for surveying and collecting data across thousands of patients and hundreds of providers, research coordinators, and analysts was solved with the involvement of REDCap. The technological tool to assist with data collection and a central data management function elevated psychiatry beyond the ancient system of one provider to one patient, and created a wealth of possibilities for how to use this data beyond the research for DSM 5. It remains to be seen if the flexibility of this instrument could be further utilized across providers and entities to assist with diagnosis, treatment, and research.

10.3 Using Technology Tools in Support of DSM-5 in Clinical Practice

The introduction of DSM-5 has provided an excellent opportunity for better integration of technology into clinical practice to enhance patient care. Although there is much promise to improving clinical flow and the quality of patient care in psychiatric setting by integrating technology, there are aspects of psychiatric practice that have made such integration inherently difficult. The relatively personal, private and subjective nature of psychiatric care have made such integration a challenge. The increasing complexity of systems of psychiatric care combined with improvements in the technology used in psychiatric diagnosis and the proliferation of more objective measures in clinical practice have lead to an increased desire to integrate technology into clinical practice.

The introduction of DSM-5 and the included assortment of measures to help quantify psychiatric illness and qualify improvement provide several opportunities to use technology to help the DSM-5 be more clinically relevant than any of the previous versions. The implementation of the data collection and coordination part of the DSM 5 field trials suggest ways in which the current tools available for clinicians could be made available to enhance accuracy and reliability of diagnosis, as well as communication of diagnosis among treatment entities.

Several areas could be of interest to developers and clinicians. The first could be developing tools to assist clinicians with diagnosis. This would be an opportunity to increase specificity and reliability of psychiatric diagnoses given in clinical setting and provide “decision-support” at the point of care for clinicians to lead to a more accurate diagnosis. This diagnosis could then be bridged to a menu of evidence based options based on accepted guidelines. Furthermore, these clinically validated diagnoses could be used for registries or for the basis of performance based measures.

These diagnoses could also be collected and used in psychiatric epidemiologic databases and possibly treatment outcomes databases. The data could be used to track and improve psychiatric care at the population level. Further, the specific diagnosis information could then coordinate with a computerized medical record and be used in communication with treatment entities outside of the home institution or potentially outside of psychiatry (such as primary care or other subspecialties).

Second, the digital tools could be useful for assisting the clinician to make co-morbid diagnoses as well as track response to treatment and evolution of the psychiatric illness. DSM-5 rating scales for diagnostic assessment of other conditions, such as functioning, degree of impairment, and suicide risk among others could prove invaluable. If available at the point of care or if previously completed by the patient, these rating scales could provide a valuable enhancement to the clinical encounter. There would be many ways that the cross-cutting assessments and rating scales that are provided in DSM-5 could be integrated in the clinical encounter with technology including applications on mobile devices or integration within the medical record. Having the results of these assessments and scales at the point of care would tremendously benefit patient care by helping to make diagnoses that may not be manifestly clear by the presentation as well as by helping to track treatment progress. Data that can provide the clinician an objective signal regarding symptom clusters that require more attention could help prevent adverse outcomes.

Practical Scenario

Mrs. X is a 45 y/o female that has been seeing her psychiatrist Dr. H for several years. She has a history of Major Depressive Disorder which has proven to be recurrent. She has been well maintained on medication and psychotherapy for several years. Her current episode began shortly after she lost her job. Dr. H has used symptoms scales filled out by Mrs. X on a tablet computer while she was in the waiting room. This tablet had a program that integrated with Dr. H’s electronic medical record so that was able to see her depression scores at this visit as compared to the last visit. He was able to see that the depression scores were worse and was able to identify that the current treatment plan may not be sufficient. He was also able to evaluate depression symptom cluster scores and he noted that she has been sleeping particularly poorly and had elevated scores in suicidal ideation. Dr. H was able to tailor his interview to be sure to address both of those issues. Further he was able to evaluate the insomnia scale that had also completed and was able to use that information for purposes of the differential diagnosis of the trouble sleeping.

Technology could also lead to enhancements of clinical care with the use of DSM-5 for interview support. The Cultural Formulation Interview (CFI) included in DSM-5 could be integrated with an electronic medical record to not only enhance the sensitivity and the accuracy of the interview but also with its documentation. Thus the CFI may be used as an interview aid as well as a documentation template. This may lead to enhanced effectiveness of care provided.

Housing rating scales and fields with inputtable results from diagnostic interviews within REDCap also suggests that greater diagnostic accuracy is possible. The transition from the DSM-IV to the DSM 5 is potentially fraught with confusion regarding the nuances of diagnosis from one edition to another, but with the safety net of the diagnostic criteria embedded in the REDCap interface, omitting elements of diagnosis is less likely. This would be a helpful IT solution to ensure that practices do not incur such coding risk to avoid penalties if billing and documentation are audited.

It is also clear that REDCap is capable of managing the comorbidity implicit in real world medical and psychiatric diagnosis from its handling of the comorbidity data from the DSM 5 field trials. REDCap and other diagnostic tools could be a valuable assistant in considering what diagnoses should be considered concurrently given the potential overlap in diagnosis. The DSM 5 has also explicitly stated that the text will be available as a subscription, and that this will make the book “readily adaptable to future scientific discoveries and refinements in its clinical utility” [29]. While the DSM-IV underwent only one major revision (the DSM-IV TR in 2000), the presence of a web-based subscription creates the possibility that changes may occur more rapidly in the future, reinforcing the need to keep up to date with potential changes, and therefore making integration with a readily updateable survey and database for diagnostic interview assists valuable to the busy clinician.

The converse to this scenario is also possible. If a centrally managed database of information from clinical interviews is created across institutions, the amount of data and ability for the tool to manage and analyze creates opportunities for vast trials powered to examine progressively smaller variations and subtypes of diagnosis in real world circumstances. In this way, the collection of interview data could influence the development of an expert consensus text (the DSM), and a discussion based format in which dynamic results could be assessed and viewed by users throughout the world. For example, the development of a new designer drug in a small region has been known to become disseminated widely throughout the country causing public health problems. Some recent examples include synthetic cannabis and bath salts [30].

Various studies have attempted to capture the public health impact, notably a report from SAMHSA (Substance Abuse and Mental Health Services Administration) that abuse of synthetic cannabis accounted for 11,406 Emergency Room visits in 2012 [31]. Could a nationally shared database of information regarding clinical interviews and experiences have enabled physicians and policy makers to act more rapidly to prevent the enormous impact of designer drugs in the US?

At the same time, substantial obstacles exist to the creation of a broadly available data sharing technology tool. In the arena of medicine, privacy concerns regarding the electronic mode of communication abound [32]. These concerns center around the nature of privacy regulations, the feasibility of secure data exchange, the security of personal devices and cloud computing, and social media. For example, the ability to store large amounts of data in compact devices, or in data accessible via internet connection such as a patient-provider connection website raises the possibility of data loss or theft, which in the case of medicine would constitute a large breach in confidentiality and privacy.

Given the role that technology has played in the development of the DSM 5, and the potential for the effect on the field of psychiatry, tools currently available to the psychiatric health care consumer and provider are affecting the current environment of diagnosis and practice. In an article in Clinical Psychiatry News, [33] the board of directors of the Anxiety and Depression Association of America (ADAA) is developing a rating system for mental health apps available for smartphone, and will be available on the associations website. For example, the National Center for PTSD is distributing an app created by the National Center for Telehealth and Technology, the Center for Deployment Psychology, and the National Center for PTSD, entitled “Mobile App: Prolonged Exposure Coach” [34]. The app is intended to be a companion tool for mobile devices that is intended to facilitate the evidence based treatment for PTSD, Prolonged Exposure, and serve as an extender for the therapist when not in session. The app features PTSD symptom tracking, psycho-education videos including common reactions to trauma, recording and playback of prolonged exposure treatment sessions, availability of homework forms and a record of completed tasks, and an “interactive breathing retraining coach”. The app is free, and privacy concerns are handled via a disclaimer with instructions on the app’s webpage. Notably, the user is instructed that the data are only “as safe as the phone/device itself”, and that storing or sharing data do not fall under HIPAA laws until the data are transmitted or shared with a mental health provider.

Another area of interest is in substance abuse treatment, where several companies have developed devices that either connect to smartphones/devices via the audio jack or usb connection, or bluetooth, that will approximate a breathalyzer reading [35]. These devices synchronize with an available app to track Blood Alcohol Levels over time, include blood alcohol levels in texts, and gain a real-time approximation of a blood alcohol level while drinking. Despite the comparative inaccuracy of these devices compared to a roadside breathalyzer available to a police officer, their relatively inexpensive cost may help them be more widely available. In addition, the idea is less to gain a blood alcohol concentration of high accuracy, and more to give the consumer objective information regarding blood alcohol rises and falls that may help provide them with better decision-making ability. If such data were available to a clinician, it may also aid in detection of substance related diagnoses apt for treatment.

Electronic Health Records available from vendors also tout the benefits of improved ability to enter rating scale data directly into patient records, asserting that the benefits of rating scale presence in the medical record results in improved quality outcomes and enhanced compensation from insurance, and attributing the lack of common practice in community settings is around access [http://www.patienttracemr.com/psychiatric-rating-scales/ for an example]. Vendors that tie electronic records to a billing and coding system offer the additional advantage of communicating rating scale results directly to an insurance company. Rating scales are widely seen as helpful adjuncts in psychiatric diagnosis [36–38].

Overall, the sheer technology utilized in developing DSM-5 has greatly advanced our field and the validity of our diagnoses. There are many additional benefits to be derived from the greater adoption of technology to improve clinical care. While we must balance the importance of confidentiality and avoid the creation of a cookbook mentality to diagnosis and treatment, the potential for dramatically improving care is difficult to argue.