Introduction

Obstetric haemorrhage remains the leading cause of maternal mortality worldwide, and it is estimated that more than 132,000 women die globally as a direct result of haemorrhage every year [1], accounting for 11% of all maternal deaths [2]. Although maternal mortality due to haemorrhage has decreased dramatically in the past 50 years [3], it is still a major cause of maternal death in the UK, accounting for 14 deaths in the triennium 2003–2005 [4].

Visual estimation of blood loss is known to be inaccurate and subjective, and may be dependent on the level of prior clinical experience [5–7]. There is poor agreement between visual estimation by health-care professionals and actual blood loss as determined by laboratory methods [8]. Previous studies have shown that accuracy is volume dependent: there is a tendency to overestimate at lower volumes and to underestimate at higher volumes [5, 9], so much so that only one in nine women with postpartum haemorrhage (PPH) would be correctly diagnosed [5]. Errors in estimation were not confined to particular disciplines, as professionals from different specialities demonstrated similar inconsistencies [7, 10]. Moreover, experience did not appear to be a confounding variable: both medical students and senior obstetricians were equally inaccurate in their estimations [7]. Surveys of standard obstetric and midwifery textbooks reveal that there is little formal training on how to estimate blood loss, although several recent authors [7, 10] have been able to show post-didactic improvement in estimation by using simulated scenarios to educate health-care professionals. Moreover, little is known on how consistent health-care professionals are when asked to estimate a similar volume of blood on separate occasions, a factor which is relevant in cases when repeated small hemorrhages have occurred.

Accurate and consistent visual estimation of blood loss in the peripartum period is crucial in helping health professionals anticipate and prepare for the appropriate resuscitative measures.

The aims of this study were twofold:

  1. (a)

    to assess the accuracy of visually estimated blood loss by midwives and obstetricians and to determine if experience was a factor;

  2. (b)

    to assess the intra-observer test–retest reliability of visually estimated blood loss by the aforementioned health professionals

Methods

This was a prospective, single-blinded observational study involving medical and midwifery staff working in a London teaching hospital. Expired units of packed red cells donated from the regional blood bank were reconstituted to approximate normal haematocrit (35–40%). Blood loss scenes were simulated by pouring accurately measured volumes of the reconstituted blood onto absorbent maternity pads (Southern Syringe Services Ltd, Enfield, UK) (Fig. 1) placed at separate stations.

Fig. 1
figure 1

Absorbent maternity pads with reconstituted blood

Full size image

In the first part of the study, maternity pads were soaked with five accurately measured volumes (25, 50, 100 150 and 200 ml) and prepared by one of the authors (KM). Participants, who were blinded to these volumes, were invited to visually estimate the blood volumes using a simple response sheet.

For the second part of the study, the authors sought to assess intra-observer test–retest reliability when visually estimating the same volumes of blood. Eight accurately measured volumes of reconstituted blood (two sets of 50, 100, 150 and 200 ml) were poured onto maternity pads randomly placed at separate stations. Participants, who were blinded to the duplicated volumes, were requested to record their estimates, in order to evaluate the consistency of their estimation of similar volumes.

The two parts of the study were conducted on separate days in order to reduce observer bias and recall. At the conclusion of data collection, the unused blood and soaked Incopads were disposed off according to hospital infection policy and procedure.

For the first part of the study, the differences between the estimated and actual volumes (in ml) were obtained by subtraction and the percentage differences calculated. In the second part, the differences in ml as well as the percentage differences between the first and second estimates of the same volume were similarly calculated.

The Ethics Committee was informed of our study but felt that ethical approval was not required as this was an observational study, which did not involve patients.

Statistical analysis

The data were normally distributed when assessed using Shapiro-Wilks test and thus parametric statistics were used for analysis. Mean and standard deviation (SD) were used to describe the data while analysis of variance (ANOVA) was used to compare groups. Box and whisker (Tukey) plots were used to present the data graphically. Intra-observer reliability was assessed by the method described by Bland and Altman [11] whereby the mean of the two estimates was plotted against the difference between them. According to the British Standards Institute, adequate repeatability is assumed when 95% of the measurements lie within 2 SD of the mean difference between the measurements [12]. As recommended by Bland and Altman, the mean difference between the two estimates should ideally be close to zero. The clinical usefulness of the test is considered in relation to the observed variability with the limits of agreement being defined as the mean difference ±2 SD.

Results

Accuracy of volume estimation

Five consultants, eight specialist registrars (residents), 11 senior house officers (interns) and 23 midwives made 218 estimations in this part of the study. The mean percentage difference between the estimated and actual volumes is shown in Table 1.

Table 1 Mean and percentage differences between estimated and actual blood volumes (ml)
Full size table

For this part of the study, the data are initially presented graphically as box and whisker plots. Midwives had the highest median percentage difference between estimated and actual volumes, compared to trainees or consultants, indicating that they were least accurate. Consultants and senior house officers (interns) appeared to be the most accurate in their estimations, with the median percentage differences being the closest to zero. The median percentage differences between the estimated and actual blood volumes were above zero for all the four groups indicating an overall tendency to overestimate, with the consultant group having the narrowest inter-quartile range (Fig. 2). However, when the mean percentage difference between the estimated and actual blood volumes were compared for the four groups, there were no statistically significant differences among consultants, registrars (residents), senior house officers (interns) and midwives (ANOVA, P > 0.05). Thus, there appears to be no statistical relationship among speciality, level of obstetric experience and accuracy of estimation.

Fig. 2
figure 2

Box and whisker plot showing percentage difference between estimated and actual blood volumes stratified by speciality. Outliers are depicted as open circles

Full size image

When stratified by actual volumes used, the inter-quartile ranges became progressively narrower with increasing volume, suggesting that the participants were more accurate with the estimation of the larger volumes (Fig. 3). Although there was a general tendency to overestimate, visual estimation was most accurate with 200 ml, as the median percentage difference between estimated and actual volume was closest to zero. Another way of expressing the magnitude of inaccuracy was to indicate the difference between estimated and actual volumes as positive deflection irrespective of whether it was an underestimation or overestimation (i.e. an underestimation of –100 ml was classified as an inaccuracy of 100 ml). This also showed that the accuracy of visual estimation increased with larger volumes with the median differences being closer to zero and the inter-quartile ranges narrower (Fig. 4).

Fig. 3
figure 3

Box and whisker plot showing the percentage difference between estimated and actual volume stratified by actual volumes used. Outliers are depicted as open circles

Full size image
Fig. 4
figure 4

Box and whisker plot showing percentage difference between estimated and actual volume with difference expressed as positive deflection. Outliers are depicted as open circles

Full size image

When subgroup analysis was performed, for larger volumes such as 200 ml, the mean estimate was 222.07 ml (±12.88), with 12 out of 29 participants (41.4%) being within 20% of the correct volume. With smaller volumes such as 25 ml, the mean estimated volume was 41.94 ml (±23.52), with 32.3% of the participants within 20% of the actual volume. The magnitude of difference between actual and estimated loss was greater with the smaller volumes where blood loss could be visually overestimated by a factor of 540% or underestimated by 87%.

Test–retest reliability

A total of 43 doctors (of differing levels of experience) and midwives participated in the test–retest reliability of visual estimation, which was assessed using the method described by Bland and Altman. For the 200 ml Incopad estimation, the mean difference between the two estimates as plotted by the Bland Altman method was –153.17 ml with 95% confidence intervals (limits of agreement) ranging from –728.95 to +422.60 ml (Fig. 5). Thus for this volume, the second estimate could be 728.95 ml less or 422.60 ml more than the first. With the 50 ml Incopad, the mean difference between the two estimates was –7.38 ml with the limits of agreement ranging from –114.90 to +100.23 ml as plotted in Fig. 6.

Fig. 5
figure 5

Bland–Altman plot for repeatability for 200 ml Incopad

Full size image
Fig. 6
figure 6

Bland–Altman plot for repeatability for 50 ml Incopad

Full size image

For the two smaller volumes of 50 and 100 ml, 95% of the data lay within two SD of the mean difference, thus fulfilling the British Standards Institute criterion for acceptable repeatability. However, visual estimation of the larger volumes (150 and 200 ml) had poor repeatability as only 88 and 93%, respectively, of the data were within 2 SD of the mean difference.

Discussion

This small observational study was prompted by a maternal death in our unit from placenta praevia. The patient had bled small amounts regularly onto sanitary pads, thus making estimation of cumulative blood loss difficult. We have therefore used small volumes of blood (of between 25 and 200 ml) for this study, as we were especially interested in assessing the accuracy and test–retest reliability of visual estimation of such volumes of blood by health-care professionals.

It was interesting to note that with the volumes assessed in this study, there was an overall composite tendency to overestimate. This is in contrast to findings by previous authors [5–8, 10], who noted that visually estimated blood loss was generally less than the actual measured volume and the magnitude of underestimation increased with progressively larger blood loss. These authors, however, reported findings that mainly pertained to volumes of greater than 500 ml, whereas our study was specifically concerned with smaller volumes of up to 200 ml. If we were to compare “like with like”, then subset analysis of the volumes of below 200 ml in the studies by Dildy [9] and Bose [10] revealed data that were broadly similar to ours. It is possible that our participants’ tendency to overestimate was influenced by their previous anecdotal experience with adverse outcomes following underestimation of blood loss as well as by comments in the recent Confidential Enquiry into Maternal and Child Health (CEMACH) Reports [4, 5] that suboptimal care in maternal deaths was often attributed to underestimation and failure to recognize significant haemorrhage. Thus, they may have adjusted their estimates generically based on these previous publications.

When data were plotted graphically using box and whisker plot, consultants and senior house officers (interns) appeared to be more accurate in their estimations compared to specialist registrars (residents) and midwives (Fig. 2). However, all four professional groups had statistically similar mean percentage difference between visually estimated and actual blood volumes (ANOVA, P > 0.05), confirming previous findings [9, 10] that senior obstetricians were statistically no more accurate than trainees or midwives.

Our study suggests that accuracy is volume dependent, with the smaller volumes being less accurately estimated than larger volumes (Fig. 2). The ranges of the difference between estimated and actual volumes was also greater with 25 and 50 ml compared to 150 and 200 ml; this data corroborate the findings of Duthie [8] and Dildy [9] who also noted more accurate estimations with volumes between 100 and 400 ml compared to volumes <50 ml. There is no satisfactory explanation as to why this is the case, although we could postulate that clinicians are more used to seeing volumes of 200–400 ml, which are the usual estimated loss in normal deliveries.

Major recommendations from recent CEMACH Reports [3, 4] relate to the importance of accurate quantification of blood loss as this influenced the degree of resuscitation, the volume of fluid restoration and blood transfusion required. Sadly, our study confirmed that visual estimation by front-line health-care workers was not only inaccurate but also inconsistent. We are not aware of any previous study whereby the test–retest reliability of visual estimation has been formally assessed. Even though the test–retest reliability was statistically acceptable for 25 and 50 ml, the limits of agreement were wide; in the case of 50 ml, the first estimate could be 122.3 ml less or 93 ml more than the second, which would make it unacceptable in clinical practice.

In simulated clinical scenarios [9, 10, 13], it appeared that clinicians were more accurate in estimating blood volumes in containers such as kidney dishes and bed pans compared to swabs or bed linen, a factor that needs to be taken into account in clinical practice when pregnant women are more likely to bleed onto pads and bed linen. For the smaller volumes described in our study, we found that the gravimetric method of weighing soaked pads using electronic scales (Marsden Weighing Co., Reading, Berkshire, UK) was a more accurate and reproducible way of estimating blood loss with a coefficient of variation of 7.57%. The use of standardized maternity pads and sanitary towels that could be weighed subsequently would be a more robust method of assessing cumulative blood loss in cases where patients have repeatedly bled small volumes.

Although precise quantitative methods of estimating blood loss such as using colorimetry or 51Cr-tagged erythrocytes have been described [14], these are not practical techniques that can be used rapidly in the labour ward. The use of calibrated blood collection drapes (attached to the lower part of a delivering mother’s body), such as the BRASSS-V drape [15, 16], is still being evaluated in multi-centred international settings, but this is probably more pertinent to blood loss in the third stage of labour rather than haemorrhage per se. Thus visual estimation, though inaccurate and inconsistent, is likely to remain the most rapid and accessible way of estimating loss in clinical practice. Although we did not include it as part of this exercise, blood loss assessment skills can be taught and post-didactic improvements have been demonstrated following educational sessions using clinical reconstruction scenarios [9, 10, 17]. Given the dramatic overestimations demonstrated in our study, these educational sessions would be crucial in helping clinicians improve their visual estimation skills.

We emphasize that no single visual parameter is accurate and clinical examination, vital signs monitoring and measurement of laboratory parameters such as haemoglobin concentration, platelet count and coagulation screen remain important in the overall assessment of obstetric haemorrhage.

What is already known about the topic

In general, visual estimation of blood loss is poor with an overall tendency to underestimate.

Educational sessions with simulated scenarios would help improve visual estimation skills.

What this study contributes to the topic

For volumes of 200 ml or less, there is a tendency to overestimate.

Test–retest reliability and consistency when visualizing the same volume is statistically, but not clinically, acceptable.