1 Introduction

Anemia is frequent in emergency room (ER) patients, especially in those with trauma or severe bleeding. Profound anemia in conjunction with hypovolemia can decrease oxygen delivery with resultant shock and tissue hypoxia. Low red blood cell count can also severely impede coagulation [1]. The gold standard of hemoglobin level measurement is the cyanohemoglobin method performed with automated hematology analyzers. Unfortunately, this method has several disadvantages: the test must be completed in a laboratory, it is time consuming and costly. Because time-sparing analytical methods might be of vital importance in the emergency room, several point-of-care devices (POCT) as for instance HemoCue and Gem Premier 3000 are often used for hemoglobin estimation. HemoCue is based on azidmethemoglobin reaction in testing cuvette and the hemoglobin level is measured using dual wavelength photometry. Gem Premier 3000 measures hemoglobin level via conductometry.

The immediate availability of measured value via POCT is outweighed by decreased accuracy compared with standard laboratory method. The reliability of different POCT devices in surgical or intensive care unit population was tested previously with contradictory results [2–20]. As intravascular volume is mostly well maintained during scheduled surgical procedures or intensive care, the performance of POCT devices might differ from under-resuscitated bleeding patients presenting in the ER. The accuracy of named POCT devices in this setting remains unclear.

The aim of our study was to test the accuracy of two commonly used POCT devices in terms of absolute hemoglobin level estimation and in terms of clinical usefulness in making of transfusion decision during routine workflow of ER.

2 Materials and methods

This was a prospective observational study conducted in the Emergency room of the Department of anesthesia and intensive care medicine at the Charles University Teaching Hospital in Plzeň between April 2012 and June 2012. The study was approved by the local ethical committee; given its observational nature, the requirement for written informed consent was waived by this committee.

All consecutive patients (n = 99) admitted to our ER in screening period with hemoglobin level measurement indicated were included in the study. No age, sex or other restrictions were applied. According to the reason of admission the patients were divided into three groups. Patients presenting with any form of blunt or penetrating trauma (polytrauma, compound trauma, or severe monotrauma as well as a head injury) were included in the “TRAUMA” group. Patients with either external or internal bleeding without severe trauma (gastro-intestinal, obstetric or bleeding of other origin) were marked as “BLEEDING”. Finally patients presenting with any form of organ dysfunction without significant bleeding (cardiac arrest, severe respiratory insufficiency, coma etc.) were included in the “ORGAN DYSFUNCTION” group.

Blood samples for hemoglobin level estimation (one sample per patients at the admission) were taken either from a vein (femoral or on the forearm) or from an artery (radial or femoral). An educated ER nurse performed the blood taking. The first 10 ml of blood drafted from the vessel was discarded in order to exclude dilution. Samples were consecutively taken directly into analyzer-specific containers: a 3 ml Vacuette® K3EDTA tube (Greiner Bio-one, VWR International, Radnor, U.S.A.), 2 ml heparinized syringe PICO™ (Radiometer Medical, Bronshoj, Denmark) and to the HemoCue® specific microcuvette (HemoCue, Angelholm, Sweden). The hemoglobin level was measured using each of the three devices. Two POCT devices, HemoCue® 201+ (HemoCue, Angelholm, Sweden) and a blood gas analyzer GEM® Premier™ 3000 (Instrumentation Laboratory Company, Lexington, U.S.A.) were operated by the emergency room nurse during their routine workflow immediately after blood taking. Simultaneously, the blood samples taken to Vacuette K3EDTA were transported at room temperature to the central laboratory and analyzed for reference value of hemoglobin using the certified Beckman Coulter® LH 750 (Beckman Coulter Inc., Miami, Florida—manufacturer´s specification: operating range 0–999 g/l, reportable range 0–250 g/l, coefficient of variation ≤0.8 %, accuracy–mean difference ±2 g/l, mean percent difference ±3.0 %) operated by the trained laboratory personnel. All three devices were regularly maintained and controlled according to the manufacturer´s recommendations.

2.1 Statistical analysis

Categorical data are expressed as number and percentage. Quantitative data are reported as mean values and standard deviation (SD) if normally distributed and as median values and interquartile range if the distribution is non-normal. The POCT devices were tested by following methods. First, a pairwise comparison between each of the POCT devices (HbHC and HbGEM) and standard comparator (HbLAB) was performed and correlation between methods was assessed by calculating coefficient of determination (r2). Second, the Bland–Altman analysis was performed. The mean of differences (bias), standard deviation of differences (SD) and limits of agreement (mean ± 2 × SD) were calculated. In accordance with previous works, the calculated difference within ±10 g/l between any two pairs of methods were considered to be clinically acceptable [3, 16, 18]. Further on, the mean differences (bias) of hemoglobin values calculated as (HbLAB–HbHC or HbLAB–HbGEM) were compared using ANOVA and paired or unpaired t test. All named calculations were performed for the whole population and in predefined subgroups based on reason for admission (trauma, other bleeding and organ dysfunction) and based on blood sample origin (arterial, venous). The p < 0.05 was considered statistically significant. All statistical analyses were performed using MedCalc software version 12.1.4. (MedCalc Software, Ostend, Belgium).

The clinical utility of both POCT devices was tested by indication or contraindication of the transfusion requirement using the error grid analysis proposed by Morey and colleagues [21]. Paired Hb values provided by test devices and the reference method were plotted using the three zones, which takes into account the clinical significance of the difference. According to the original methodology zone A should contain 95 % of all measurements and means clinically acceptable difference without affecting the transfusion indication—the deviation of the tested device is ±10 % from the HbLAB. Zone B (optimally less than 5 % of all measurements) is the area between zone A and zone C. It is defined as the area of significant errors. Finally, Zone C is defined as the area of major therapeutic errors. It is the zone in which either the HbLAB value >100 g/l (transfusion contraindicated) is associated with HBHC/GEM <70 g/l (transfusion indicated) or opposite situation (HbLAB <70 and HBHC/GEM >100 g/l) occurs. No measurement is allowed to be in zone C. The 95 % confidence interval (CI) for each zone was calculated using exact Pearson–Clopper intervals. Unlike the original authors who used the 60–100 g/l thresholds, according to the ASA 2006 practice guidelines for transfusion [22], we have used hemoglobin values of 70 and 100 g/l based on updated European guideline [23].

3 Results

In total, 292 measurements were performed in 99 patients recruited in the study. In 243 (83 %) measurements, venous blood samples and in 49 (17 %) arterial blood samples were used. In two subjects, no HemoCue value was obtained (device temporary unavailable) and in three subjects no GEM Premier 3000 value was obtained due to lack of time.

The age range of participating patients was from 4 to 88 years with an average and SD of 50 ± 22 years, with 23 (23 %) being female. The study population consisted of 64 patients admitted with trauma, 14 patients with severe bleeding and 21 patients with organ dysfunction. The reference HbLAB ranged from 27 to 175 g/l. The mean HBLAB, HBHC and HBGEM values were 111 ± 30, 115 ± 33 and 110 ± 28 g/l respectively. Tables 1, 2 summarize the descriptive characteristics of the patients’ population and their subgroups.

Table 1 Descriptive characteristics of patient population and the mean Hb levels in each group
Full size table
Table 2 The number of measurements and mean Hb levels in arterial and venous blood sample group
Full size table

3.1 HemoCue versus automated hematology analyzer

There was a positive correlation between HemoCue and automated hematology analyzer (r2 = 0.93, p < 0.001). The two methods comparison using the Bland–Altman methodology showed mean difference and limits of agreement −3.7 g/l (−20.9 to 13.5). (Tables 3, 4; Fig. 1). These results were similar also in predefined subgroups (trauma, bleeding and organ dysfunction) or based on blood sample (arterial and venous), as shown in Table 3. The clinically acceptable limits of ±10 g/l difference were fulfilled in 75 (77 %). According to Morey´s methodology Zone A, B and C encompassed 93 % (95 % CI 87–98 %), 7 % (95 % CI 3–14 %) and 0 % (95 % CI 0–4 %) of data pairs (Fig. 2).

Table 3 Comparison of POCT devices and hematology analyzer
Full size table
Table 4 Comparison of POCT devices and hematology analyzer according to blood sample
Full size table
Fig. 1
figure 1

Bland–Altman plot of the HemoCue versus hematology analyzer

Full size image

3.2 GEM Premier 3000 versus automated hematology analyzer

Positive correlation between GEM Premier 3000 and automated hematology analyzer was observed (r2 = 0.86, p < 0.001). The Bland–Altman mean difference and limits of agreement were 2.5 g/l (−18.6 to 23.5) (Tables 3, 4; Fig. 3). The mean difference of HbLAB against HbHC or HbGEM significantly differed with HbGEM systematically under- and HbHC overestimating the HbLAB values. This was especially true for GEM measurements in trauma patients and arterial blood samples (Table 3). The results of the subgroup analysis are summarized in Tables 3, 4. Clinically acceptable difference (±10 g/l) was found in 77 (80 %) of patients. The results of the error grid analysis (Fig. 4) were out of recommended ranges [21] with Zone A, B and C encompassing 91 % (95 % CI 83–86 %), 8 % (95 % CI 4–15 %) and 1 % (95 % CI 0–5 %) respectively.

Fig. 2
figure 2

Paired hemoglobin values provided by HemoCue and the hematology analyzer plotted using three zones error grid. Zone A (green) represents the area of clinical acceptable difference, zone B (yellow) represents the area of significant errors, although their magnitude is not as significant as in zone C (red) which represents the area of major therapeutic errors leading to unnecessary or delayed transfusion

Full size image
Fig. 3
figure 3

Bland–Altman plot of the GEM Premier 3000 versus hematology analyzer

Full size image
Fig. 4
figure 4

Paired hemoglobin values provided by GEM Premier 3000 and the hematology analyzer plotted using three zones error grid. Zone A (green) represents the area of clinical acceptable difference, zone B (yellow) represents the area of significant errors, although their magnitude is not as significant as in zone C (red) which represents the area of major therapeutic errors leading to unnecessary or delayed transfusion

Full size image

4 Discussion

In our study, the levels of hemoglobin assessed during routine ER work flow by two different POCT devices (HemoCue and GEM Premier 3000) showed correlation with reference laboratory measurement. However, their clinical applicability might be confounded by low accuracy and precision. In addition, their ability to correctly indicate the red blood cell transfusion remains disputable.

There are several studies which contrast with our results. Both devices were assessed previously (HemoCue [3, 9, 14, 17, 20, 24] or GEM Premier 3000 [13]) mostly with positive results. However, also negative studies were published [4, 5, 8]. The reliability of measurements was assessed using standard correlation and by the Bland–Altman method in most of these studies. The aim of our study was to test the reliability under normal workflow in patients admitted solely to the emergency department. The population studied so far was recruited in the operating room or in the intensive care. In two studies only, patients presented in the ER with acute bleeding were also included, but contributed with limited part (one-third) into the mixed population of post-anesthetic unit and/or intensive care unit [16, 20]. The conditions in the ER patients can significantly differ from relatively stable patients during surgery or intensive care. First, the blood loss in the ER patients is usually ongoing and in the moment of admission often not well replaced, patients are usually hypothermic, with leukocytosis etc. Quite recently, Gayat tested the pulse oxymetry derived hemoglobin values with central laboratory under similar conditions [25]. In this study, the continuous POCT device failed to be accurate or precise in bleeding patients under emergency conditions.

The person of the operator who did the measurement could also play an important role. It has been pointed out that better results were obtained by the laboratory scientific officers and by single users in comparison with practice nurses and multiple users [15, 26, 27]. In certain studies the measurement was done by a person intended exclusively for this purpose [3] or all the samples were sent into the central laboratory where they were analyzed with POCT devices used by a single, trained operator [13]. This fact decreases the influence of analytical risk on one side, but on the other side is not clinically relevant. In our study the measurements were done by the well trained nurses during their standard workflow presenting real conditions of the emergency room.

Appropriate sampling site of blood for HemoCue measurement is another potential source of discordance. The use of capillary, arterial or venous blood is described in the literature. However according to some limited data [4, 8] and given the acute bleeding patient physiology we deem the use of capillary blood in the ER patients prone to error and replaced it by arterial or venous samples. In addition, this enabled us to test all devices using the same sample. We found no difference in accuracy between arterial and venous blood samples.

When we judge the usefulness of POCT hemoglobin meters, the most important fact is not the ability of reaching correct absolute value, but rather the ability to indicate the transfusion whenever it is necessary. When we used the three zones error grid to assess the clinical accuracy of POCT devices, we found that the POCT devices were not enough accurate. According to the methodology proposed by Morey [21], <5 % of measurements should be in zone B and none should be in zone C. The results of our study (and their 95 % confidence intervals) do not fulfill these strict safety margins making both devices (and especially GEM Premier 3000) unreliable. This is in conflict with the recent study of Giraud [7]. But two facts have to be pointed out. First, they had just 2,3 % of measurements with HBLAB <80 g/l while in our study it was 18 % and second, they used the transfusion limits according to ASA 2006 guidelines [22] which are 100–60 g/l and thus more benevolent in comparison with the recent European guidelines [23].

5 Limitations

Our study poses several limitations. First, the number of patients is rather low, especially in the subgroup of patients with very low hemoglobin values. The study sample size was not calculated a priori, because the study ran as an audit of routine praxis. However, the sample collected after 3 month was comparable to previous published studies [5, 6, 18, 19]. Second, only one set of measurements does not allow us to address the trending ability of studied POCT devices. Including another set of measurements after fluid resuscitation and/or blood product administration may overcome both these limitations. Blood samples were taken during the admission to the ER reflecting the usual practice and population studied, but after the initial care the patients were usually quickly distributed from the ER to appropriate departments or ICUs for further treatment and we had no control over the further treatment. Also, the routine workflow precluded us to perform repeated measurements on one sample so the internal variability of each device remained untested. Finally we have not studied to what extent the false POCT reading has affected the real practice i.e. whether unnecessary transfusions were given or vice versa. However, we deem the Morey error grid methodology is robust enough to address this issue.

6 Conclusion

In our study, the levels of hemoglobin assessed during routine emergency room workflow by two different POCT devices (HemoCue and GEM Premier 3000) showed correlation with reference laboratory measurement, but their clinical applicability might be confounded by low accuracy and precision. Also their ability to correctly indicate the red blood cell transfusion remains disputable. For this reason, we deem these devices may help to lead the decision in the very urgent situations of massively bleeding patients, but for other occasions a more reliable measurement should be preferred. However, our data might be limited by the low number of included patients.