Abstract
Objectives
To evaluate the observer accuracy and intra-observer test–retest reliability of visual estimation of blood loss by midwives and obstetricians.
Methods
This was a prospective, single-blinded observational study conducted at a London teaching hospital. The accuracy of visually estimating five maternity pads that had been soaked with 25, 50, 100, 150 and 200 ml of blood was assessed. The reproducibility in estimating the same volume (two sets of pads soaked with 50, 100, 150 and 200 ml of blood randomly placed at separate stations) was evaluated by asking participants to visually estimate these volumes.
Results
Although there is a tendency to overestimate, the mean percentage difference (estimated–actual volumes) was not significantly different among consultants, trainees and midwives. Visual estimations were especially inaccurate with smaller volumes, which could be overestimated by up to 540%. Test–retest reliability was poor for the larger volumes but statistically acceptable for the smaller volumes, although the difference between the two estimates of the same volume could be as much as 300%.
Conclusions
Visual estimations were inaccurate by health-care professionals who have a tendency to overestimate. Experience did not appear to have a confounding effect on accuracy. Further training in visual assessment skills is necessary in order to improve the clinicians’ estimation.
Similar content being viewed by others
Avoid common mistakes on your manuscript.
Introduction
Obstetric haemorrhage remains the leading cause of maternal mortality worldwide, and it is estimated that more than 132,000 women die globally as a direct result of haemorrhage every year [1], accounting for 11% of all maternal deaths [2]. Although maternal mortality due to haemorrhage has decreased dramatically in the past 50 years [3], it is still a major cause of maternal death in the UK, accounting for 14 deaths in the triennium 2003–2005 [4].
Visual estimation of blood loss is known to be inaccurate and subjective, and may be dependent on the level of prior clinical experience [5–7]. There is poor agreement between visual estimation by health-care professionals and actual blood loss as determined by laboratory methods [8]. Previous studies have shown that accuracy is volume dependent: there is a tendency to overestimate at lower volumes and to underestimate at higher volumes [5, 9], so much so that only one in nine women with postpartum haemorrhage (PPH) would be correctly diagnosed [5]. Errors in estimation were not confined to particular disciplines, as professionals from different specialities demonstrated similar inconsistencies [7, 10]. Moreover, experience did not appear to be a confounding variable: both medical students and senior obstetricians were equally inaccurate in their estimations [7]. Surveys of standard obstetric and midwifery textbooks reveal that there is little formal training on how to estimate blood loss, although several recent authors [7, 10] have been able to show post-didactic improvement in estimation by using simulated scenarios to educate health-care professionals. Moreover, little is known on how consistent health-care professionals are when asked to estimate a similar volume of blood on separate occasions, a factor which is relevant in cases when repeated small hemorrhages have occurred.
Accurate and consistent visual estimation of blood loss in the peripartum period is crucial in helping health professionals anticipate and prepare for the appropriate resuscitative measures.
The aims of this study were twofold:
-
(a)
to assess the accuracy of visually estimated blood loss by midwives and obstetricians and to determine if experience was a factor;
-
(b)
to assess the intra-observer test–retest reliability of visually estimated blood loss by the aforementioned health professionals
Methods
This was a prospective, single-blinded observational study involving medical and midwifery staff working in a London teaching hospital. Expired units of packed red cells donated from the regional blood bank were reconstituted to approximate normal haematocrit (35–40%). Blood loss scenes were simulated by pouring accurately measured volumes of the reconstituted blood onto absorbent maternity pads (Southern Syringe Services Ltd, Enfield, UK) (Fig. 1) placed at separate stations.
In the first part of the study, maternity pads were soaked with five accurately measured volumes (25, 50, 100 150 and 200 ml) and prepared by one of the authors (KM). Participants, who were blinded to these volumes, were invited to visually estimate the blood volumes using a simple response sheet.
For the second part of the study, the authors sought to assess intra-observer test–retest reliability when visually estimating the same volumes of blood. Eight accurately measured volumes of reconstituted blood (two sets of 50, 100, 150 and 200 ml) were poured onto maternity pads randomly placed at separate stations. Participants, who were blinded to the duplicated volumes, were requested to record their estimates, in order to evaluate the consistency of their estimation of similar volumes.
The two parts of the study were conducted on separate days in order to reduce observer bias and recall. At the conclusion of data collection, the unused blood and soaked Incopads were disposed off according to hospital infection policy and procedure.
For the first part of the study, the differences between the estimated and actual volumes (in ml) were obtained by subtraction and the percentage differences calculated. In the second part, the differences in ml as well as the percentage differences between the first and second estimates of the same volume were similarly calculated.
The Ethics Committee was informed of our study but felt that ethical approval was not required as this was an observational study, which did not involve patients.
Statistical analysis
The data were normally distributed when assessed using Shapiro-Wilks test and thus parametric statistics were used for analysis. Mean and standard deviation (SD) were used to describe the data while analysis of variance (ANOVA) was used to compare groups. Box and whisker (Tukey) plots were used to present the data graphically. Intra-observer reliability was assessed by the method described by Bland and Altman [11] whereby the mean of the two estimates was plotted against the difference between them. According to the British Standards Institute, adequate repeatability is assumed when 95% of the measurements lie within 2 SD of the mean difference between the measurements [12]. As recommended by Bland and Altman, the mean difference between the two estimates should ideally be close to zero. The clinical usefulness of the test is considered in relation to the observed variability with the limits of agreement being defined as the mean difference ±2 SD.
Results
Accuracy of volume estimation
Five consultants, eight specialist registrars (residents), 11 senior house officers (interns) and 23 midwives made 218 estimations in this part of the study. The mean percentage difference between the estimated and actual volumes is shown in Table 1.
For this part of the study, the data are initially presented graphically as box and whisker plots. Midwives had the highest median percentage difference between estimated and actual volumes, compared to trainees or consultants, indicating that they were least accurate. Consultants and senior house officers (interns) appeared to be the most accurate in their estimations, with the median percentage differences being the closest to zero. The median percentage differences between the estimated and actual blood volumes were above zero for all the four groups indicating an overall tendency to overestimate, with the consultant group having the narrowest inter-quartile range (Fig. 2). However, when the mean percentage difference between the estimated and actual blood volumes were compared for the four groups, there were no statistically significant differences among consultants, registrars (residents), senior house officers (interns) and midwives (ANOVA, P > 0.05). Thus, there appears to be no statistical relationship among speciality, level of obstetric experience and accuracy of estimation.
When stratified by actual volumes used, the inter-quartile ranges became progressively narrower with increasing volume, suggesting that the participants were more accurate with the estimation of the larger volumes (Fig. 3). Although there was a general tendency to overestimate, visual estimation was most accurate with 200 ml, as the median percentage difference between estimated and actual volume was closest to zero. Another way of expressing the magnitude of inaccuracy was to indicate the difference between estimated and actual volumes as positive deflection irrespective of whether it was an underestimation or overestimation (i.e. an underestimation of –100 ml was classified as an inaccuracy of 100 ml). This also showed that the accuracy of visual estimation increased with larger volumes with the median differences being closer to zero and the inter-quartile ranges narrower (Fig. 4).
When subgroup analysis was performed, for larger volumes such as 200 ml, the mean estimate was 222.07 ml (±12.88), with 12 out of 29 participants (41.4%) being within 20% of the correct volume. With smaller volumes such as 25 ml, the mean estimated volume was 41.94 ml (±23.52), with 32.3% of the participants within 20% of the actual volume. The magnitude of difference between actual and estimated loss was greater with the smaller volumes where blood loss could be visually overestimated by a factor of 540% or underestimated by 87%.
Test–retest reliability
A total of 43 doctors (of differing levels of experience) and midwives participated in the test–retest reliability of visual estimation, which was assessed using the method described by Bland and Altman. For the 200 ml Incopad estimation, the mean difference between the two estimates as plotted by the Bland Altman method was –153.17 ml with 95% confidence intervals (limits of agreement) ranging from –728.95 to +422.60 ml (Fig. 5). Thus for this volume, the second estimate could be 728.95 ml less or 422.60 ml more than the first. With the 50 ml Incopad, the mean difference between the two estimates was –7.38 ml with the limits of agreement ranging from –114.90 to +100.23 ml as plotted in Fig. 6.
For the two smaller volumes of 50 and 100 ml, 95% of the data lay within two SD of the mean difference, thus fulfilling the British Standards Institute criterion for acceptable repeatability. However, visual estimation of the larger volumes (150 and 200 ml) had poor repeatability as only 88 and 93%, respectively, of the data were within 2 SD of the mean difference.
Discussion
This small observational study was prompted by a maternal death in our unit from placenta praevia. The patient had bled small amounts regularly onto sanitary pads, thus making estimation of cumulative blood loss difficult. We have therefore used small volumes of blood (of between 25 and 200 ml) for this study, as we were especially interested in assessing the accuracy and test–retest reliability of visual estimation of such volumes of blood by health-care professionals.
It was interesting to note that with the volumes assessed in this study, there was an overall composite tendency to overestimate. This is in contrast to findings by previous authors [5–8, 10], who noted that visually estimated blood loss was generally less than the actual measured volume and the magnitude of underestimation increased with progressively larger blood loss. These authors, however, reported findings that mainly pertained to volumes of greater than 500 ml, whereas our study was specifically concerned with smaller volumes of up to 200 ml. If we were to compare “like with like”, then subset analysis of the volumes of below 200 ml in the studies by Dildy [9] and Bose [10] revealed data that were broadly similar to ours. It is possible that our participants’ tendency to overestimate was influenced by their previous anecdotal experience with adverse outcomes following underestimation of blood loss as well as by comments in the recent Confidential Enquiry into Maternal and Child Health (CEMACH) Reports [4, 5] that suboptimal care in maternal deaths was often attributed to underestimation and failure to recognize significant haemorrhage. Thus, they may have adjusted their estimates generically based on these previous publications.
When data were plotted graphically using box and whisker plot, consultants and senior house officers (interns) appeared to be more accurate in their estimations compared to specialist registrars (residents) and midwives (Fig. 2). However, all four professional groups had statistically similar mean percentage difference between visually estimated and actual blood volumes (ANOVA, P > 0.05), confirming previous findings [9, 10] that senior obstetricians were statistically no more accurate than trainees or midwives.
Our study suggests that accuracy is volume dependent, with the smaller volumes being less accurately estimated than larger volumes (Fig. 2). The ranges of the difference between estimated and actual volumes was also greater with 25 and 50 ml compared to 150 and 200 ml; this data corroborate the findings of Duthie [8] and Dildy [9] who also noted more accurate estimations with volumes between 100 and 400 ml compared to volumes <50 ml. There is no satisfactory explanation as to why this is the case, although we could postulate that clinicians are more used to seeing volumes of 200–400 ml, which are the usual estimated loss in normal deliveries.
Major recommendations from recent CEMACH Reports [3, 4] relate to the importance of accurate quantification of blood loss as this influenced the degree of resuscitation, the volume of fluid restoration and blood transfusion required. Sadly, our study confirmed that visual estimation by front-line health-care workers was not only inaccurate but also inconsistent. We are not aware of any previous study whereby the test–retest reliability of visual estimation has been formally assessed. Even though the test–retest reliability was statistically acceptable for 25 and 50 ml, the limits of agreement were wide; in the case of 50 ml, the first estimate could be 122.3 ml less or 93 ml more than the second, which would make it unacceptable in clinical practice.
In simulated clinical scenarios [9, 10, 13], it appeared that clinicians were more accurate in estimating blood volumes in containers such as kidney dishes and bed pans compared to swabs or bed linen, a factor that needs to be taken into account in clinical practice when pregnant women are more likely to bleed onto pads and bed linen. For the smaller volumes described in our study, we found that the gravimetric method of weighing soaked pads using electronic scales (Marsden Weighing Co., Reading, Berkshire, UK) was a more accurate and reproducible way of estimating blood loss with a coefficient of variation of 7.57%. The use of standardized maternity pads and sanitary towels that could be weighed subsequently would be a more robust method of assessing cumulative blood loss in cases where patients have repeatedly bled small volumes.
Although precise quantitative methods of estimating blood loss such as using colorimetry or 51Cr-tagged erythrocytes have been described [14], these are not practical techniques that can be used rapidly in the labour ward. The use of calibrated blood collection drapes (attached to the lower part of a delivering mother’s body), such as the BRASSS-V drape [15, 16], is still being evaluated in multi-centred international settings, but this is probably more pertinent to blood loss in the third stage of labour rather than haemorrhage per se. Thus visual estimation, though inaccurate and inconsistent, is likely to remain the most rapid and accessible way of estimating loss in clinical practice. Although we did not include it as part of this exercise, blood loss assessment skills can be taught and post-didactic improvements have been demonstrated following educational sessions using clinical reconstruction scenarios [9, 10, 17]. Given the dramatic overestimations demonstrated in our study, these educational sessions would be crucial in helping clinicians improve their visual estimation skills.
We emphasize that no single visual parameter is accurate and clinical examination, vital signs monitoring and measurement of laboratory parameters such as haemoglobin concentration, platelet count and coagulation screen remain important in the overall assessment of obstetric haemorrhage.
What is already known about the topic
In general, visual estimation of blood loss is poor with an overall tendency to underestimate.
Educational sessions with simulated scenarios would help improve visual estimation skills.
What this study contributes to the topic
For volumes of 200 ml or less, there is a tendency to overestimate.
Test–retest reliability and consistency when visualizing the same volume is statistically, but not clinically, acceptable.
References
Cameron MJ, Robson SC (2006) Vital statistics: an overview. In: Lynch CB, Keith L, Lalonde AB and Karoshi M (eds) A textbook of postpartum hemorrhage. Sapiens Publishing, Duncow, Kirkmahoe, Dumfrieshire, UK, Chap 3, pp 17–34
Ronsmans C, Graham WJ (2006) Maternal mortality: who, when, and why. Lancet 368(9542):1189–1200. doi:10.1016/S0140-6736(06)69380-X
Lewis G (ed) (2004) The confidential enquiry into maternal and child health (CEMACH). Why mothers’ die—2000–2002. The sixth report on confidential enquiries into maternal deaths in the United Kingdom. RCOG Press, London
Lewis G (ed) (2007) The confidential enquiry into maternal and child health (CEMACH). Saving mothers’ lives: reviewing maternal deaths to make motherhood safer: 2003–2005. The seventh report on confidential enquiries into maternal deaths in the United Kingdom. RCOG Press, London
Razvi K, Chua S, Arulkumaran S, Ratnam SS (1996) A comparison between visual estimation and laboratory determination of blood loss during the third stage of labour. Aust NZ J Obstet Gynaecol 36(2):152–154
Prasertcharoensuk W, Swadpanich U, Lumbiganon P (2000) Accuracy of the blood loss estimation in the third stage of labour. Int J Gynaecol Obstet 71:69–70. doi:10.1016/S0020-7292(00)00294-0
Meiser A, Casagranda O, Skipka G, Laubenthal H (2001) Quantification of blood loss. How precise is visual estimation and what does its accuracy depend on? Anaesthetist 50(1):13–20. doi:10.1007/s001010050957
Duthie SJ, Ven D, Yung GL, Guanng DZ, Chan SY, Ma HK (1991) Discrepancy between laboratory determination and visual estimation of blood loss during normal delivery. Eur J Obstet Gynecol Reprod Biol 38:119–124. doi:10.1016/0028-2243(91)90188-Q
Dildy GA, Paine AR, George NC, Velasco C (2004) Estimating blood loss: can teaching significantly improve visual estimation? Obstet Gynecol 104:601–606
Bose P, Regan F, Paterson-Brown S (2006) Improving the accuracy of estimated blood loss at obstetric haemorrhage using clinical reconstructions. BJOG 113(8):919–924. doi:10.1111/j.1471-0528.2006.01018.x
Blandt JM, Altman DG (1986) Statistical methods for assessing agreement between two methods of clinical measurements. Lancet 1:307–310
British Standards Institution Precision of test methods I: Guide for the determination and reproducibility for a standard test method 9BS 5497 (Part 1). BSI 1979, London
Buckland SS, Homer CSE (2007) Estimating blood loss after birth: using simulated clinical examples. Women Birth 20:85–88. doi:10.1016/j.wombi.2007.01.001
Kodkany BS, Dreman RJ (2006) Pitfalls in assessing blood loss and decision to transfer. In: Lynch CB, Keith L, Lalonde AB, Karoshi M (eds) A textbook of postpartum hemorrhage. Sapiens Publishing, Duncow, Kirkmahoe, Dumfrieshire, UK, Chap 4, pp 35–45
Patel A, Goudar SS, Geller SE, Kodkany BS, Edlavitch SA, Wagh K, Patted SS, Naik VA, Moss N, Derman RJ (2006) Drape estimation vs. visual assessment for estimating postpartum hemorrhage. Int J Gynaecol Obstet 93(3):220–224. doi:10.1016/j.ijgo.2006.02.014
Toledo P, McCarthy RJ, Hewlett BJ, Fitzgerald PC, Wong CA (2007) The accuracy of blood loss estimation after simulated vaginal delivery. Anesth Analg 105(6):1736–1740. doi:10.1213/01.ane.0000286233.48111.d8
Luegenbiehl D (1997) Improving visual estimations of blood volume on peripads. Am J Matern Child Nurs 22(6):294–298. doi:10.1097/00005721-199711000-00003
Conflict of interest statement
All authors declare that they have no competing interests.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Yoong, W., Karavolos, S., Damodaram, M. et al. Observer accuracy and reproducibility of visual estimation of blood loss in obstetrics: how accurate and consistent are health-care professionals?. Arch Gynecol Obstet 281, 207–213 (2010). https://doi.org/10.1007/s00404-009-1099-8
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00404-009-1099-8