Introduction

There is increasing awareness of the importance of integrated mental health care services and the need for effective psychiatric screening procedures within HIV testing and treatment centres [1,2,3,4]. Rates of psychological distress and psychopathology are marked among persons living with HIV [5,6,7,8,9] and among those seeking HIV testing [10, 11]. Mental health problems have an enduring deleterious impact on physical health, quality of life, and social and occupational functioning. Mental health problems are also associated with an increased risk of HIV infection and with poor adherence to anti-retroviral therapy (ART) [12]. Consequently, there have been calls to establish suitable screening procedures at HIV testing and treatment sites in order to identify individuals who are likely to meet diagnostic criteria for a CMD and who require referrals for clinical assessment by a mental health professional [10]. It may, however, not be practical or even desirable to screen all HIV test seekers for mental health problems in low resource settings; the lack of reliable screening instruments for use in these settings and the paucity of suitable mental health referral agencies makes it potentially unworkable to establish efficient screening and referral systems in HIV testing centres, especially if the screening instruments are not sensitive and specific [13]. Research has already established the utility of a range of disorder-specific screening tools for identifying major depressive disorder (MDD), generalised anxiety disorder (GAD), posttraumatic stress disorder (PTSD), and alcohol use disorders (AUD), among persons seeking HIV testing [14,15,16]. It is, however, potentially cumbersome and time-consuming to use multiple disorder-specific screening instruments to identify patients who need further psychiatric assessment, especially in busy resource-constrained HIV testing sites in low- and middle-income countries, like South Africa. It is within this context that we wished to establish whether the short version of the Hopkins Symptom Checklist (HSCL-25) could be used in HIV testing sites to identify people likely to have CMDs and who require further psychiatric assessment.

Screening for Common Mental Disorders

There are several advantages to utilising well-validated screening instruments for CMDs in primary health care settings, including the fact that such practices may help to create an integrated person-centred health care system and may assist in closing the mental health treatment gap [17,18,19]. Research suggests that health professionals working in primary health care settings have difficulty diagnosing mental disorders as a result of inadequate knowledge of diagnostic criteria, uncertainty about what to ask in order to elicit psychiatric symptoms, and time constraints [20]. Screening can overcome some of these barriers by enabling health workers who do not have specialist mental health training, to efficiently and accurately identify patients who would benefit from further psychiatric assessment.

The use of mental health screening instruments in primary health care settings is not without challenges and potential shortcomings [21]. First, it can be difficult to identify a screening instrument that has good sensitivity (i.e. a high probability of correctly identifying individuals with a mental disorder) and good specificity (i.e. a high probability of correctly identifying individuals who do not have a mental disorder) [22]. In a perfect screening instrument both sensitivity and specificity would equal 1 and the optimal cut off point would maximise both values [23]. In practice, however, there is often a trade-off between specificity and sensitivity with clinicians typically favouring sensitivity when screening for highly infectious illnesses or conditions which are life-threatening and/or have serious complications, while specificity is favoured when it is difficult or costly to make referrals and clinicians want to avoid generating high numbers of false positives [24]. Second, the use of mental health screening instruments for the indiscriminant screening of all patients may not be appropriate in low-resource settings where inadequate and inaccessible psychiatric services hamper the referral of individuals who screen positive. The use of screening tools which lack specificity can generate an inordinate number of false positive referrals and hence place an unnecessary burden on already scarce psychiatric services [13]. Third, self-report screening tools have been critiqued for being less reliable and valid than more structured clinician administered mental health screening instruments [22]. However, the use of clinician administered instruments is dependent on the ready availability of suitably trained clinicians, which may be limited in low-resource primary health care settings. Finally, it can be challenging to determine the optimal length of a screening instrument; overly inclusive screening instruments can be long and cumbersome to administer, while ultra-short instruments may have limited utility to accurately and consistently identify patients in need of further assessment [25]. For all of these reasons, there should be judicious use of mental health screening instruments within HIV testing sites.

In spite of the potential limitations of screening instruments, some advances have been made towards establishing the utility of disorder-specific screening instruments for CMDs in primary health care settings generally [25,26,27,28,29], and in specific health care settings, such as HIV treatment centres [30,31,32]. Trans-diagnostic screening instruments, such as the K10 and Substance Abuse and Mental Illness Symptom Screener (SAMISS), have also been validated for use in detecting CMDs among HIV-infected individuals in South Africa [33, 34]. Comparatively less attention has been paid to validating screening instruments for use among persons seeking HIV testing. Notable exceptions are the recently published studies reporting on the utility of the Posttraumatic Stress Scale-Self-report (PTSS), the Beck Depression Inventory (BDI), the Beck Anxiety Inventory (BAI), and the Alcohol Use Identification Test (AUDIT) to identify persons seeking HIV testing who likely meet diagnostic criteria for PTSD, MDD, GAD and AUD respectively. Research suggests that the Posttraumatic Stress Scale-Self-report can identify PTSD among HIV test seekers with a sensitivity of 0.76 and a specificity of 0.78 (positive predictive value (PPV) = 0.24, negative predictive value (NPV) = 0.97) [15]. The BDI-I predicts MDD among HIV test seekers with 0.67 sensitivity and 0.67 specificity (PPV = 0.25, NPV = 0.92) [14], while the BAI identifies cases of GAD with a sensitivity and specificity of 0.82 and 0.80, respectively (PPV = 0.13, NPV = 0.99) [16]. Similarly the AUDIT identifies cases of AUD among persons seeking HIV testing with a 0.81 sensitivity and 0.81 specificity (PPV = 0.51, NPV = 0.95) [16]. While there is some evidence to support the use of these disorder-specific screening instruments in HIV testing sites, it may not be practical or efficient to ask HIV test seekers to complete a lengthy battery of tests consisting of multiple screening instruments. It may also not be feasible for clinic staff to have to score and interpret multiple disorder-specific screening instruments, each of which has different optimal cut-off points and scoring algorithms. These concerns prompt questions about the potential utility of brief trans-diagnostic screening instruments, such as the Hopkins Symptom Checklist (HSCL), to identify HIV test seekers who require further psychiatric assessment in resource constrained settings, like South Africa.

Hopkins Symptom Checklist

The Hopkins Symptom Checklist (HSCL) is a selfreport symptom inventory, which asks about a range of complaints typically associated with CMDs. The items are representative of the symptom dimensions commonly observed among outpatients in primary health care settings (namely somatization, obsessive-compulsiveness, interpersonal sensitivity, anxiety and depression). These dimensions have been consistently identified in repeated factor analyses (i.e. a statistical method used to identify the smallest number of underlying variables) in a wide range of populations [35]. The HSCL is available in two formats; a longer version consisting of 90 items (HSCL-90) and a short version consisting of 25 items (HSCL-25).

The HSCL-25 consists of 25 items, each of which is scored on a scale of 0 (Not at all) to 4 (Extremely), thus yielding a potential total score between 0 and 100. It consists of two parts; part I has ten items assessing typical anxiety symptoms, and part II consists of 15 items assessing symptoms of depression. It is thus possible to calculate a HSCL-25 total score (the sum of all 25 items), an anxiety subscale score (sum of the items in part I), and a depression subscale score (sum of the items in part II).

Studies in several populations have consistently shown that the total score on the HSCL is highly correlated with severe emotional distress of unspecified diagnosis, and the depression score is correlated with major depression as defined by the Diagnostic and Statistical Manual (DSM) of the American Psychiatric Association [36]. The HSCL-25 has been used in population-level ecological studies of mental disorders [37], and in a range of diverse settings to identify individuals who require psychiatric care, including asylum seekers [38], traumatized refugees [39], former political detainees in South Africa [40], patients seeking medical care in primary health care settings [41], as well as among populations affected by war [42] and other post-conflict populations in low- and middle-income countries [43]. It has also been adapted and translated for use in culturally and linguistically diverse regions of the world, including Asia [44, 45], the Middle East [41], Africa [46], and the former Yugoslavia [47]. The instrument is also available in a number of European, African and Asian languages [35, 38, 41, 42, 48,49,50,51,52,53,54,55]. The HSCL-25 has also been utilised in studies of psychological distress and psychopathology among pregnant women with HIV in Tanzania [56] and persons living with HIV in South Africa [48, 57]. Furthermore, it has been used in multinational studies to assess the prevalence of psychiatric symptoms among persons living with HIV [58]. The HSCL has not, however, been extensively validated as a screening instrument for CMDs among persons seeking HIV testing.

The aim of this study was to establish the utility of the short version of the Hopkins Symptom Checklist (HSCL-25) to correctly identify CMDs among persons seeking HIV testing. We chose to use the HSCL-25 rather than the K-10 or SAMISS, given research showing that the K-10 only demonstrated “agreeable sensitivity and specificity” (p. 1163) [33] and that the SAMISS demonstrated “moderate specificity” (p. 1136) [34] in detecting CMDs among HIV-infected individuals in South Africa. We were also interested in establishing the utility of the HSCL-25 for use in HIV-testing sites given that this instrument was develop specifically for use in primary health care settings, it is available in a large number of languages, is relatively quick to administer (typically taking < 10 min to complete), and has previously been shown to have good utility as a trans-diagnostic mental health screening tool.

Methods

Sampling

Data for this cross-sectionals study were collected from a convenience sample of 500 individuals seeking HIV testing in a peri-urban area of the Western Cape Province of South Africa, as part of a larger study to investigate CMDs and psychological adjustment among individuals seeking HIV testing [59].

Persons seeking HIV testing were invited to enrol in the study prior to receiving their HIV-test. To be eligible for the study participants had to be 18 years or older, have capacity to give informed consent and be able to understand English or Afrikaans. Participants were excluded if they were under 18, lacked capacity or were not conversant in English or Afrikaans.

Data Collection

The following data were collected on Lenovo tablets in a confidential setting by trained data collectors under the close supervision of two registered psychologists:

  1. (1)

    Demographic variables Participants were asked their age and home language. They were also asked how they self-identified in terms of gender and race.

  2. (2)

    CMDs Participants were assessed to establish if they met diagnostic criteria for MDD, persistent depressive disorder (PDD), GAD, PTSD and AUD, using the research version of the Structured Clinical Interview (SCID) adapted to be compliant with the 5th edition of the DSM [59, 60].

  3. (3)

    Hopkins Symptom Checklist Participants were asked to complete the HSCL-25.

All interviews were audio recorded and quality checks were conducted to ensure adherence to the study protocol and the accuracy of the data collected.

Data Analysis

Data were cleaned, checked and imported into SPSS for analysis. Descriptive statistics were used to determine the sample characteristics, prevalence estimates for CMDs, and the range and mean scores on the HSCL-25. Receiver operating characteristic (ROC) curve analysis was employed to determine the optimal cut-off point, specificity, sensitivity, PPV and NPV for the HSCL-25’s ability to identify individuals with a CMD. We also used ROC curve analysis to establish the utility of the HSCL-25 total scores, depression subscale scores, and anxiety subscale scores to detect MDD, PDD, GAD, and PTSD, using the SCID as the gold standard for determining psychiatric caseness. We sought to optimise sensitivity and specificity, by selecting the cut-off scores which corresponded to the smallest absolute difference between sensitivity and specificity [23].

Ethics

Ethical approval for this study was obtained from the Health Sciences Research Ethics committee at Stellenbosch University. Permission to conduct the study was also obtained from the Western Cape Department of Health. Written informed consent was obtained from all participants prior to data collection, and data were securely stored in a manner that protected the identity of participants. Participants who were identified as being in psychological distress were referred to appropriate community based psychological services.

Results

Sample Characteristics

Just over half of the sample (N = 500) self-identified as female (51.6%). The mean age of the participants was 36 years (range 18–71, SD = 12.2). In terms of race, 72.6% identified as coloured (an official term used in South Africa for population classification), 26.2% Black-African, and 0.8% White. The majority of the sample (69.0%) were Afrikaans speaking, while 6.0% were English and 19.6% were isiXhosa speaking. The mean total score on the HSCL-25 was 46 (range = 25–97, SD = 17.1). Scores on the depression subscale ranged from 15 to 60, with a mean of 28.6 (SD = 11.4) and scores on the anxiety subscale ranged from 10 to 39, with a mean of 17.4 (SD = 6.4).

The prevalence estimates with 95% confidence intervals (95% CI) for CMDs in the sample were as follows: MDD 14.4% (11.4–17.8), PDD 7.2% (5.1–9.8), GAD 3.4% (5.1–9.8), PTSD 5.0% (3.2–7.3), and AUD 19.6% (16.2–23.4). A total of 37.0% of the sample met diagnostic criteria for one CMD, 7.6% met criteria for two disorders, and 2.8% met criteria for three disorders.

Receiver Operating Characteristic Curve Analysis of the HSCL-25

The ROC curve in Fig. 1 shows the performance of the HSCL-25 in detecting caseness of any CMD. The area under the curve (AUC) of 0.80 (95% CI 0.75–0.83) indicates that the HSCL-25 is moderately accurate in identifying individuals with a CMD. At a total-score cut-off point of 43, the HSCL-25 identified individuals who met diagnostic criteria for a CMD, with a sensitivity of 0.69% (95% CI 0.62–0.76) and specificity of 0.70 (95% CI 0.65–0.75). The PPV was 0.57 (95% CI 0.52–0.62) and the NPV was 0.80 (95% CI 0.76–0.83). These data indicate a 57% probability that individuals who scored above 43 on the HSCL-25 would likely meet diagnostic criteria for a CMD, and an 80.0% probability that individuals scoring below 43 would not meet the criteria for a disorder. Detailed data for the co-ordinates of the ROC curve, showing how the optimal cut-off point was established, are available as supplementary material on request.

Fig. 1
figure 1

Receiver operating characteristic curve for the utility of the Hopkins Symptom Checklist-25 to identify individuals with a common mental disorder among people seeking HIV testing (n = 500)

Full size image

The results of the ROC curve analysis of the utility of the HSCL-25 to identify MDD, PDD, GAD, PTSD and AUD are shown in Table 1. Optimal cut-off points were established using the total HSCL-25 score and the depressive scale sub-score (for depressive disorders) and the anxiety subscale scores (for anxiety disorders and PTSD). Figures showing the ROC curves for each of the analyses presented in Table 1, along with detailed data showing how the optimal cut-off points were determined, are available on request as supplementary material.

Table 1 The results of the ROC curve analysis of the utility of the eHSCL-25 to predict common mental disorders among persons seeking HIV testing (N = 500)
Full size table

Discussion

The finding that 37.0% of HIV test seekers reported at least one CMD is consistent with literature showing that between 20 and 36% of patients in primary health care outpatient settings typically have mental health problems [20, 61,62,63]. Our data supports calls to establish suitable mental health screening procedures and psychiatric referral pathways within primary health care settings generally [17], and HIV testing centres specifically. In order to establish effective mental health screening practices in HIV testing sites it will be necessary to identify user-friendly, valid screening instruments for this population. Our analysis suggests that the HSCL-25 has limited utility as a valid brief trans-diagnostic screening tool to identify individuals who are likely to have a CMD; it is only moderately accurate and 43% of the cases identified at the optimal cut-off score will be false positives, potentially generating a high number of unnecessary psychiatric referrals. This compares poorly to the SAMISS which identified CMDs among HIV infected individuals in South Africa with a specificity and sensitivity of 0.78 (95% CI 0.73–0.83) and 0.85 (95% CI 0.73–0.93) [34].

The analysis we have presented suggests that the HSCL-25 can be used to screen for depressive illnesses (both MDD and PDD) in populations of persons seeking HIV testing. It is noteworthy that in detecting cases of MDD, the depression subscale of the HSCL-25 (sensitivity = 0.81, specificity = 0.78, PPV = 0.39, NPV = 0.96) performed remarkably better than previous reports on the utility of the BDI-I for use in this population (sensitivity = 0.67, specificity = 0.67, PPV = 0.25, NPV = 0.92). The depression subscale of the HSCL-25 is also shorter than the BDI-I. In terms of identifying GAD, the anxiety subscale of the HSCL-25 (sensitivity = 0.77, specificity = 0.78, PPV = 0.11, NPV = 0.99), performed less well than the BAI (sensitivity = 0.82, specificity = 0.80, PPV = 0.13, NPV = 0.99), although still within acceptable limits.

The finding that the HSCL-25 has utility as a screening instrument for PTSD among HIV test seekers, is consistent with a number of other studies that have demonstrated the utility of this instrument in screening for trauma related disorders in diverse populations [38, 42]. In terms of identifying cases of PTSD among HIV test seekers, the anxiety subscale of the HSCL-25 (sensitivity = 0.80, specificity = 0.79, PPV = 0.17, NPV = 0.99) compares favourably with disorder-specific screening instruments, such as the Posttraumatic Stress Scale–Self-report (sensitivity = 0.76, specificity = 0.78, PPV = 0.24, NPV = 0.97).

It is noteworthy that the HSCL-25 demonstrated low levels of sensitivity and specificity in detecting cases of AUD. This is not altogether surprising given that the instrument does not ask directly about substance use. Crucially this finding highlights the fact that if the HSCL-25 is used to screen for CMDs among HIV test seekers, it will be necessary to augment it with substance use specific screening items, especially given the finding in previous studies of prevalence estimates of 19.8% for AUD in this population [10]. To this end, future studies could establish the utility of screening instruments such as the CAGE Screening Test for Alcohol Use Disorders and AUDIT to be used in conjunction with the HSCL-25. Both the CAGE and AUDIT are short instruments which could be included with the HSCL-25 without significantly increasing the time taken to screen patients.

The HSCL-25 makes use of a Likert scale and relies on the use of patient self-report forms. This may make the instrument inaccessible and difficult to understand for illiterate individuals and those with low levels of education. Research suggests that there are also important cultural differences in response patterns to Likert scale questionnaires [64]. It will be important for future research to establish how user-friendly the HSCL-25 is and how patients in different cultural settings experience its use, before the instrument is widely adopted for mental health screening at HIV-testing sites.

Limitations

Data were collected via self-report measures at one non-medical testing site in the Western Cape Province of South Africa. Only individuals conversant in English or Afrikaans were included in the sample. This limits the generalisability of these results to other settings and highlights the need for these findings to be replicated in samples drawn from HIV testing sites in other areas. It is also a limitation that we did not assess symptoms of bipolar spectrum disorder and thus our estimates of MDD may be marginally inflated because they might include some individuals with a bipolar spectrum disorder.

Conclusion

There is an emerging debate about the merits of trans-diagnostic approaches to screening and intervention for CMDs, versus disorder-specific approaches [65,66,67]. This is an important issue to consider in the context of implementing mental health screening and intervention services in both HIV testing and HIV treatment settings, especially in resource-constrained environments where psychiatric services are scarce and oversubscribed. The prevalence data we collected suggest that there is a need to identify brief trans-diagnostic instruments with high levels of sensitivity and specificity for screening HIV test seekers for CMDs. Our analysis suggests that while the HSCL-25 holds some promise as a valid brief trans-diagnostic screening tool, its low PPV may make it unsuitable for use in low-resource environments, like HIV testing sites in South Africa. More work is required to identify user-friendly trans-diagnostic mental health screening instruments for use in HIV testing sites.