Introduction

Anemia is a common global cause of morbidity and mortality, affecting up to a third of the world’s population and responsible for approximately 8% of nonfatal health loss from all diseases worldwide [1, 2]. As with many health metrics, anemia disproportionately affects persons living in resource-limited settings, with prevalence rates five times higher in low-income countries (42.8%) than high-income countries (9.1%) [3]. Overall, an estimated 1.6 billion people are anemic with the greatest burden among preschool age children (293 million) and pregnant women (56 million) [3]. In sub-Saharan Africa, 64.6% of children less than 5 years of age are anemic [3, 4].

The etiology of anemia is multifactorial, but sickle cell disease (SCD) is arguably the most important cause that contributes to early childhood mortality. Globally, over 400,000 infants are born with SCD annually, with a majority of affected births (> 75%) occurring in the resource-limited settings of sub-Saharan Africa [5]. Without the availability of appropriate diagnostics, most children with SCD will die before they reach 5 years of age, usually before the correct diagnosis is made [6,7,8]. Delayed diagnosis, failure to diagnose, and misdiagnosis all lead to suboptimal use of a limited blood supply and higher morbidity and mortality [9,10,11].

The World Health Organization (WHO) and the National Institutes of Health have called for technology development that can be deployed in low-resource settings [12, 13]. The development of simple, inexpensive, and accurate point-of-care (POC) diagnostics for both anemia and SCD could significantly shorten the time to diagnosis and allow critical interventions to reduce both morbidity and mortality from these two global health challenges. Several new low-tech POC assays are on the horizon for diagnosis of both anemia [14, 15] and for SCD [16,17,18,19,20,21,22]. Ease of use and interpretation is essential for wide-scale implementation. We previously validated the accuracy of a prototype of a POC color-based assay for the diagnosis of anemia [14] and a POC immunoassay for the diagnosis of SCD [20], but the assays were performed and results interpreted by experienced investigators in a controlled laboratory setting within the USA. In the current study, we aimed to field test both of these POC assays in the hands of local end-users working in the outpatient clinics and inpatient wards of a busy referral hospital in sub-Saharan Africa. Our objectives were to examine the diagnostic accuracy of both assays compared to their respective gold standards in a real-world situation and to determine the interobserver reliability between experienced and inexperienced end-users.

Methods

Study design

The Robust Assays for Point-of-care Identification of Disease (RAPID) study was a prospective, blinded comparison of two different POC assays against their respective laboratory-based gold standards. The study was designed to prospectively collect a blood sample from children at risk for anemia or SCD and aimed to (1) compare results using a POC assay for the diagnosis of SCD (Sickle SCAN™, Biomedomics, Inc., Research Triangle Park, NC) to hemoglobin electrophoresis results and (2) compare the results obtained from a first-generation POC color-based assay to estimate hemoglobin concentration with results obtained from an automated hematology analyzer. The study was conducted from February 2016 to February 2017 at Bugando Medical Centre (BMC) in Mwanza, Tanzania, located in the northwestern part of the country on the shores of Lake Victoria. Mwanza is the second largest city in Tanzania with a population of over 700,000, and BMC is a 900-bed referral and teaching hospital that serves a catchment area of 13 million people in the seven surrounding regions.

Study objectives

The goals of the RAPID study were to obtain data regarding the accuracy and simplicity of an SCD POC lateral flow assay and a first-generation POC color-based assay, in a real-world clinical setting within sub-Saharan Africa. The Joint BMC and Catholic University of Health and Allied Sciences (CUHAS) Research and Ethics Committee, the Tanzanian National Institute for Medical Research, and the Institutional Review Board at Cincinnati Children’s Hospital Medical Center approved the study protocol. The primary study objective for the SCD POC assay was to compare results with “gold standard” hemoglobin electrophoresis results, to identify the sensitivity and specificity of the POC assay to diagnose SCD. The primary study objective for the hemoglobin POC color-based assay was to provide pilot results from field-use including accuracy and information on ease of use to inform the design of packaging, color scale, and formulation decisions in subsequent generations.

Selection and recruitment of participants

Children ≤ 21 years old attending the outpatient sickle cell clinic, visiting the emergency department, or hospitalized in the pediatric ward and suspected to have anemia or SCD, were invited to participate. Written informed consent was obtained from a parent or guardian, and assent was obtained from any child ≥ 10 years old. Consent and assent were obtained in the local language of the participant or parent/guardian providing consent (Swahili or English).

Point-of-care assays

A prototype of the hemoglobin POC assay was previously described [15]. It is a visual, color-based assay that relies upon the reaction between hemoglobin, hydrogen peroxide, and 3,3′,5,5′-tetramethylbenzidine. The assay results range from a bluish color (lower hemoglobin concentration, more anemia) to a reddish color (higher hemoglobin concentration, less anemia). Figure 1 illustrates the hemoglobin range (2.5–9.1 g/dL) and color spectrum of this POC assay. Reagents for the anemia assay were prepared in the USA and hand-delivered to the study site approximately every 3 months. The reagent was prepared in bulk, and 0.5 mL was aliquoted into 1.5 mL Eppendorf tubes covered with foil until use, to prevent exposure to light, which can affect the chemical reaction. Reagents were stored at mean ambient temperature (about 25 °C) at the clinical site but out of direct sunlight in the site investigator’s office.

Fig. 1
figure 1

Color spectrum of the point-of-care hemoglobin assay. The chemical reaction between hemoglobin, hydrogen peroxide, and 3,3′,5,5′-tetramethylbenzidine was designed to identify moderate and severe anemia with a hemoglobin range of 2.5 to 9.1 g/dL

Full size image

The SCD POC assay is a lateral flow chromographic qualitative immunoassay for the rapid determination of the presence or absence of HbA, HbS, and HbC; this design is intended to diagnose the common forms of sickle cell disease [16]. The testing cartridge has four detection bands, including a distal control band that appears when the sample has flowed to the end of the test strip (Fig. 2). The presence of normal (HbA) or variant (HbS or HbC) hemoglobin is indicated by a blue line in the specific region indicated on the device. Each kit includes a capillary sampler (disposable plastic pipette with a black line at the 5 μL point), a tube prefilled with 1.0 mL of buffer, and a testing cartridge. The SCD POC assays were procured in boxes containing 20 kits from the USA distributor and hand-delivered to the study site approximately every 6 months. Before use, the boxes of kits were stored at ambient temperature but out of direct sunlight in the site investigator’s office.

Fig. 2
figure 2

Performance of the point-of-care sickle cell disease assay. After mixing the sample with the buffer and placing on the application pad, the presence of HbA, HbS, and HbC are indicated by a darkened horizontal band. Results are easily interpreted within 5 min

Full size image

Study procedures

At the time of a study participant’s routine phlebotomy, venous blood was collected in an EDTA tube for performing the laboratory tests, while a finger prick sample of capillary blood was also collected for the POC assays. For the hemoglobin POC assay, an end-to-end capillary tube was used to collect exactly 10 μL of blood. Any excess blood at the end of the capillary tube was wiped off prior to inserting the tube into the Eppendorf tube prefilled with reagents. The Eppendorf tube was gently inverted, and the contents were thoroughly mixed. After 60 s, the Eppendorf tube was held against a white background, and the color of the mixture was compared to a printed copy of the color scale (Fig. 1). If the color of a sample was between two printed colors, the scorer estimated the hemoglobin concentration using a value that was between the hemoglobin values assigned to the printed colors.

For the SCD POC assay, a small disposable pipette included in the kit was used to collect 5 μL of capillary blood, which was added to the tube prefilled with buffer. The container was mixed gently by inverting the tube three times, to allow lysis of erythrocytes and release of hemoglobin. Per manufacturer’s recommendation, the first three drops of the lysate were discarded, and the following five drops applied to the test cartridge. Five minutes after sample application, two persons visually scored each sample independently for the presence or absence of the four potential bands (HbA, HbS, HbC, and control). If there was difficulty obtaining capillary blood from a finger prick, venous blood from the EDTA tube was used for both POC assays.

Both POC assays were performed at the bedside and then scored by two independent scorers who were blinded to each other’s results and to the gold standard laboratory results. Scorer 1 was a physician and study team member who was familiar with the assays, while Scorer 2 was one of the local healthcare staff and included pediatricians, intern doctors, medical students, and nurses. Limited training on the use and interpretation of the POC assays was provided, in order to evaluate the performance of the POC assays as they might actually be used in the clinical setting. The providers were shown a half-page description of how to perform the color-based hemoglobin assay, as well as the package insert for the SCD POC assay and then observed the performance of each test once by a co-investigator.

The hemoglobin concentration was determined in the laboratory as a full-blood count using a Mindray BC-3200 Auto Hematology Analyzer (Mindray Bio-Medical Electronics Co., Ltd., Shenzhen, China). Sickle cell status (trait or disease) was determined in the laboratory using an alkaline bath agarose gel hemoglobin electrophoresis technique (Biotec-Fischer GmbH, Reiskirchen, Germany). Both the hemoglobin electrophoresis and the full-blood count were completed in the main laboratory by hospital personnel independently of the POC assays. The results of the POC assays were not available to the technicians who performed and reported the laboratory results.

Statistical analysis

All statistical analyses were performed using Stata 14.2 (StataCorp, College Station, Texas). A p value of less than 0.05 was considered significant. Pearson’s correlations and linear regression models with prediction limits were used to assess associations between the hemoglobin concentration obtained from the POC color-based assay and the hemoglobin concentration obtained from the standard hematology analyzer. The Bland-Altman method was used to calculate the mean difference in hemoglobin obtained by the POC assay and the automated hematology analyzer, including the 95% limits of agreement. For the SCD analysis, the presence or absence of HbA, HbS, and HbC as detected on the POC assay was compared to the result obtained by hemoglobin electrophoresis. Sensitivity and specificity were calculated for each individual observer and in aggregate. Interobserver agreement was calculated using Fleiss’ kappa statistic.

Results

Enrolment and demographics

A total of 752 participants consented to the Robust Assays for Point-of-care Identification of Disease (RAPID) study. Primary case report forms were misplaced for 7, so the final dataset included 745 participants. The mean (±SD) age was 5.2 ± 4.2 years, ranging from 1 day old to 20 years old. Most (58.3%) were less than 5 years of age, while only 15.3% were greater than 10 years of age. Participants were primarily recruited from the inpatient hospital wards (58.2%) and the sickle cell clinic (38.5%) of Bugando Medical Centre (BMC) in Mwanza, Tanzania. Fewer were recruited from the general pediatric outpatient clinic (2.2%) or emergency department (1.1%) of BMC. Nearly 1 in 5 (19.1%) reported receiving a blood transfusion within the prior 3 months. Two individuals served as Scorer 1 and interpreted 99% of the assays. Scorer 2 included 136 different Tanzanian pediatricians, interns, students, and nurses at BMC. The median number of assays interpreted by each of these individuals was 2, and 85% interpreted fewer than 10 assays.

Performance of SCD POC assay

Among the 745 participants included in the final analysis, 652 (87.5%) had results available from at least one SCD POC assay and hemoglobin electrophoresis (92 did not have a confirmatory electrophoresis result, while the POC assay result was not available for 1 participant). Given the rarity of HbC in East Africa, 3 patterns were observed: (1) HbA present and HbS absent, consistent with a normal HbAA, identified in 214 (32.8%) participants; (2) both HbA and HbS present, suggesting either sickle cell trait or SCD with a recent blood transfusion, identified in 119 (18.3%); (3) HbS present and HbA absent, consistent with a diagnosis of SCD, identified in 319 (48.9%). Over half (57%) of children with both HbA and HbS identified had received a transfusion recently, suggesting that they might have an underlying diagnosis of SCD, rather than sickle trait.

The SCD POC assay had excellent sensitivity and specificity for the identification of the HbS-only pattern, consistent with SCD (homozygous HbSS). For Scorer 1, who was more experienced with the assay, the sensitivity was 99.4% (95% CI 97.7–99.9%) and specificity was 94% (95% CI 90.9–96.3%). For the less-experienced Scorer 2, the sensitivity was 98.1% (95% CI 95.9–99.3%) and specificity was 91.1% (95% CI 87.5–94.0%). Table 1 details the sensitivity, specificity, and positive and negative predictive values with the associated 95% confidence intervals. Data are provided for the Scorer 1 and Scorer 2 roles individually as well as in aggregate.

Table 1 Diagnostic test accuracy for the point-of-care sickle cell disease assay
Full size table

Of the 644 samples that had 2 SCD POC scores in addition to a hemoglobin electrophoresis result, there were 45 (7%) samples where at least 1 scorer reported a result discordant with the electrophoresis result. In about half of these cases (24 of 45), both scorers made the same call. The most common discordant result (30 of 45) was only HbS identified on POC assay, but both HbA and HbS scored on hemoglobin electrophoresis. Interobserver agreement was excellent with a Kappa statistic of 0.95 (Table 2).

Table 2 Interobserver agreement for the point-of-care sickle cell disease assay
Full size table

Performance of hemoglobin POC assay

Hemoglobin POC color-based assay results were available for 429 participants, and comparison hematology analyzer results were available for 376 of these samples. The mean hemoglobin concentration (±SD) measured by automated hematology analyzer was 7.3 (± 2.7) g/dL. Nearly half of the samples (170/376 = 45%) met WHO criteria for severe anemia in young children (Hb ≤ 7 g/dL) and 75 samples (20%) had very severe anemia with Hb ≤5 g/dL.

Since the maximum hemoglobin concentration measured by the color-based assay is 9.1 g/dL (Fig. 1), 72 samples with measured Hb > 9.1 g/dL were excluded from the primary analysis of hemoglobin determination but were included in the secondary analyses evaluating the ability to detect severe anemia. The absolute difference (mean ± SD) between the POC assay and the hematology analyzer result was 1.0 (± 0.9) g/dL for Scorer 1 and 1.1 (± 0.9) g/dL for Scorer 2. There was a moderately strong correlation between the hemoglobin POC assay and the measured hemoglobin concentration (r = 0.65 for Observer 1 and 0.64 for Observer 2). Figure 3a illustrates the correlation in aggregate with a combined correlation coefficient (r) of 0.65. Figure 3b is a Bland-Altman plot illustrating the correlation according to hemoglobin concentration for both scorers in aggregate. Despite variation compared to the gold standard, the interobserver agreement for the POC assay was high with a Pearson’s correlation coefficient (r) of 0.96 (Fig. 3c).

Fig. 3
figure 3

Hemoglobin concentration comparisons between gold standard hematology analyzer and color-based point-of-care hemoglobin assay. Panel A illustrates the correlation between the gold-standard hemoglobin concentration and the hemoglobin concentration measured by the color-based POC assay (r = 0.65). Panel B is a Bland-Altman plot demonstrating no clear differences in the accuracy of the POC assay across hemoglobin concentrations. Panel C demonstrates the excellent interobserver agreement for the POC assay (r = 0.96)

Full size image

In addition to analyzing the correlation to measured hemoglobin concentration, we also analyzed the ability for the POC assay to detect both severe (Hb ≤7 g/dL) and very severe (Hb ≤5 g/dL) anemia. There were 170 samples with severe anemia and 75 samples with very severe anemia as measured by the automated hematology analyzer. The sensitivity of the color-based test for WHO-defined severe anemia (Hb ≤ 7 g/dL) was 83.2% (95% CI 78.8–87%) with specificity of 74.5% (95% CI 69.9–78.6%). The sensitivity of the POC anemia assay for very severe anemia (Hb ≤5 g/dL) was 50.3% (95% CI 42–59.6%) with specificity of 81.3% (95% CI 75.5–86.2%). These data are reported in aggregate for both scorers, but there were no significant differences in sensitivity or specificity when analyses were performed separately.

Discussion

Anemia and SCD are among the world’s most common and most serious health conditions. Early and accurate diagnosis of both is essential in order to facilitate timely lifesaving interventions. In this study, we describe a true field study experience using novel POC assays for the bedside diagnosis of both anemia and SCD in the hands of local healthcare providers in Mwanza, Tanzania. The SCD POC assay provided excellent sensitivity (> 98%) for detecting SCD, even in the hands of inexperienced end-users who were relatively unfamiliar with the assay. Despite this inexperience, it is important that these local end-users achieved the same degree of accuracy as highly trained laboratory staff in the USA [16, 20]. A small study in Nigeria where only 16 of 57 samples had an abnormal hemoglobin pattern reported similar results using experienced study staff [23]. RAPID is the largest study of these POC assays reported to date, enrolling > 700 participants and utilizing > 100 minimally trained local end-users, yet retaining excellent accuracy for the immediate diagnosis of SCD.

This simple but accurate SCD POC assay could vastly expand access to SCD testing in low-resource settings where the majority of people with SCD are born. Currently, in most high-burden African countries, where up to 1–2% of all infants are born with SCD, newborn screening (NBS) programs are largely absent, and the diagnosis of SCD can only be made in private clinics or large tertiary care medical centers with laboratories. Even where these diagnostics are available, the laboratory turnaround time is days to weeks, resulting in delays and significant difficulties in locating affected persons once the diagnosis is made. The only “low-tech” alternatives until now have been light microscopy or a sickling testing, both of which are inaccurate and still require a functioning laboratory. Also, it is unfortunately common for high-tech equipment such as high-performance liquid chromatography (HPLC) or capillary zone electrophoresis (CZE), which is often philanthropically donated, to break down and lose functionality due to lack of finances for maintenance and repair of damaged equipment or restocking expensive reagents. The lack of or significant delay in diagnosis of SCD is truly a matter of life and death for the affected patients. This novel SCD POC assay could change the balance of that equation.

Our findings support the idea that this POC assay could be introduced as a screening tool for identification of SCD early in life, even in distant health outposts without immediate laboratory confirmation. Rural health personnel already use similar POC assays for malaria and HIV, and we have demonstrated that an SCD POC assay can also be easily and accurately interpreted. Implementation on a wider scale would require end-users to interpret results within the proper clinical context. Specifically, healthcare providers must understand that a result of HbA and HbS could represent either sickle cell trait or recent blood transfusion in a child with SCD; of course, this dilemma is also relevant for the proper interpretation of laboratory-based hemoglobin electrophoresis results. Wherever such screening occurs, but particularly if performed in the most rural areas, patients with HbS only (or HbA and HbS with index of clinical suspicion) should be referred to a larger center that provides confirmatory testing and comprehensive care for SCD.

In the USA, UK, and much of Europe, NBS for SCD is routinely performed for all infants and allows for early diagnosis and more importantly, early intervention. Diagnosis of SCD is routinely and accurately performed using a variety of techniques, including hemoglobin electrophoresis, CZE, HPLC, and even DNA-based genetic methods. With the combination of early diagnosis and associated preventive care such as prophylactic penicillin and pneumococcal immunization, it is now rare for a child with SCD to die before reaching adulthood with reported survival well above 90% in the USA and UK [24, 25]. Even in the lower resourced setting of Jamaica, the addition of NBS and early care for infants affected by SCD has substantially reduced morbidity and mortality [26].

In our study, the first-generation hemoglobin POC assay was less accurate but still estimated the hemoglobin concentration within ~ 1 g/dL. The sensitivity and specificity to diagnose severe anemia (Hb < 7 g/dL) were modest (83.2 and 74.5%). More importantly, its sensitivity to diagnose very severe anemia (Hb < 5 g/dL), the level at which a blood transfusion is indicated in many resource-limited settings was relatively low (50.3%) with moderate specificity (81.3%). The POC assay did perform better than using clinical signs such as conjunctival, palmar, and nailbed pallor to diagnose anemia, as clinical signs have a reported sensitivity of 52% and specificity of 75% for diagnosing anemia < 11 g/dL [27]. However, with a sensitivity of 50.3%, many children with very severe anemia would have a false-negative result, be improperly diagnosed to have a hemoglobin > 5 g/dL, and miss out on a potentially life-saving blood transfusion.

The moderate accuracy of the POC assay was most likely the consequence of reagent instability resulting in color drift away from the preprinted color scale. The process of disbursing a large quantity of reagent and aliquoting into individual samples on-site allowed excess exposure to air. Furthermore, reagent interaction with plastic-lined Eppendorf tubes degraded the color quality. A second-generation POC assay is now under development that has incorporated these realities into its design for a low-resource setting. While an inexpensive and accurate test for hemoglobin is sorely needed, the true utility of this POC assay is yet to be established. In addition to validating the second-generation POC assay, future testing may focus on its use as a screening test primarily for severe anemia, to triage patients warranting higher level acuity and blood transfusion; a prospective comparison to similar low-cost POC assays such as the WHO Hemoglobin Color Scale is warranted.

Our study has several limitations that should be noted. In RAPID, a single-hemoglobin electrophoresis result for each participant was used as the gold standard for identification of sickle trait or disease, but abnormal results were not confirmed using a second testing modality as commonly performed in high-resource settings. Similarly, the measured hemoglobin concentrations were not confirmed, and the hematology analyzer results could have been inaccurate due to clerical or technical issues. Finally, for transfused patients with SCD, even small amounts of HbA might be detected with lower sensitivity by POC than by hemoglobin electrophoresis. Despite these potential limitations, our results are encouraging and should lead to studies to improve and optimize the utility of these assays.

In conclusion, the RAPID study provides essential real-world field data for two novel POC assays that had been previously evaluated primarily in controlled settings and scored primarily by USA-based investigators. We demonstrate that implementation of a POC assay to screen for SCD in low-resource settings is both feasible and highly sensitive and retains a high degree of accuracy even with a wide variety of mostly inexperienced end-users. The first-generation hemoglobin POC assay has been further modified, and evaluation of subsequent generations is necessary before widespread implementation, possibly with automated color scoring rather than visual estimates. Taken together, however, these results are highly encouraging for the problems of anemia and SCD in sub-Saharan Africa, where simple, inexpensive, and accurate POC tests are sorely needed and offer the potential to save lives.