Introduction

Diabetes is a growing public health problem in the United States [1]. Diabetes during pregnancy has serious adverse consequences for both mothers and their children. Prepregnancy diabetes mellitus (PDM), type 1 or type 2, accelerates maternal diabetes complications and increases risk for spontaneous abortions and birth defects [2]. Gestational diabetes mellitus (GDM) can lead to pregnancy-associated hypertension, fetal macrosomia, and cesarean delivery [3]. GDM recurs in up to 70% of subsequent pregnancies; 50% of women with GDM will develop type 2 diabetes within 5 years, and their children face increased risk for obesity and diabetes [3–5].

Effective public health surveillance of diabetes during pregnancy is needed to define the burden, identify mothers at risk and inform strategies to reduce the short-term and long-term impact of diabetes and its complications. It is important that surveillance sources distinguish between PDM and GDM because the two conditions differ markedly in terms of their impacts and potential interventions. Treatment for PDM requires tight glucose control prior to and during pregnancy, early screening for fetal abnormalities and neonatal follow-up care [2]. For GDM, treatment strategies include medical nutrition therapy, insulin and moderate physical activity [3]. GDM also presents a window of opportunity to prevent or delay type 2 diabetes among women at high risk and their families.

A challenge for GDM surveillance is that there is no consensus on screening or diagnostic criteria for GDM [5]. Russell, Carpenter, and Coustan’s recent review [6] found limited evidence to choose among different screening methods, or criteria. The US Preventive Services Task Force found insufficient evidence to warrant GDM screening at all [7]. However, there is mounting evidence that diabetes risk persists across generations via the intrauterine environment [8]. It is therefore worth noting that none of the reviews took into account the impact of GDM screening on type 2 diabetes prevention. It is also unclear when or how these reviews will influence practice. Currently, most prenatal care providers uniformly screen for GDM. For example, Ferrara et al. [9] found that 87% of women without pre-pregnancy diabetes were screened for GDM in a large northern California health plan.

Birth certificate and hospital discharge data are population-based, routinely available and economical to obtain and analyze. Both can be used to identify diabetes-complicated births and examine disparities by selected maternal demographics. It is important to better understand the usefulness and limitations of these data sources for surveillance of diabetes during pregnancy.

The 1989 US Standard Certificate of Live Birth included a single check box for diabetes as a maternal medical risk factor of pregnancy [10]. In 1989, birth certificates in 10 reporting jurisdictions differed from the national standard in that they distinguished between PDM and GDM. These jurisdictions were: Alaska, Colorado, Maine, Missouri, Nebraska, New York City, North Dakota, Oregon, Washington State, Wisconsin (Kowaleski, Personal Communication, November 28, 2005). With the 2003 revision, two checkboxes distinguishing PDM and GDM were added to the standard form. As of 2006, only 20 states had implemented the new birth certificate, and less than half of all births were being reported on the revised form [11]. Prior to implementing the 2003 revision, several more jurisdictions began distinguishing PDM from GDM. These included California, Connecticut, Minnesota, North Dakota, Utah and Washington (Edelman, Personal Communication, January 10, 2007).

Results from birth certificate quality studies vary widely, but information on maternal medical risk factors is generally considered poor relative to other birth certificate variables [12, 13]. There is wide variation in the personnel and procedures involved with birth certificate completion, and this may affect birth certificate quality [14–16]. Improvements were anticipated in the quality of birth certificate data with the advent of electronic birth certificate reporting, which became common in the mid-1990s [17]. But there is some question whether this is occurring [18]. Birth certificate design (specifically check boxes vs. open-ended responses) has been shown to influence birth certificate data accuracy [19].

Hospital discharge data are collected primarily for billing purposes but routinely used for state and national disease surveillance and research, including studies of maternal medical conditions and pregnancy complications [20]. Diabetes is identified using diagnosis codes, deliveries using procedure codes. Hospital discharge data have been extensively validated for the purpose of identifying diabetes [21]. However, few validation studies of hospital discharge data have been specific to obstetric events [22].

We conducted a comprehensive literature review to summarize the existing information on validity of hospital discharge data and birth certificate data for identifying diabetes-complicated births. Our review is modeled on Saydah et al. [21] who recently reviewed death certificates, administrative and survey data for identifying diabetes cases, but did not include birth certificates or address diabetes during pregnancy. Relative to standard diabetes surveillance, the challenges of surveillance for diabetes during pregnancy include: the need to distinguish between PDM and GDM; the lack of agreement on screening and diagnostic criteria for GDM; and the lack of an accepted gold standard.

Our objective is to thoroughly document what is currently known about the performance and usefulness of birth certificate and hospital data for identifying cases of diabetes-complicated pregnancy. We also hope to shed light on the strengths and limitations of existing validation studies to improve future research on the quality of these data sources.

Methods

We searched Medline and the Internet for articles and abstracts published between 1989 and 2007 describing studies linking US birth certificate and/or hospital discharge data with another source to assess their validity.

On Medline, we used “birth certificates” as a medical subject heading (MeSH) term, along with one of the following additional MeSH terms: comparative study, validation studies, reproducibility of results or sensitivity and specificity. We also looked for “birth certificates” as a MeSH term, along with any of the following terms in any part of the Medline record: accuracy, bias, completeness, credibility, evaluation, quality, reliability, underreporting or validity. We conducted the same searches as above using “hospital records” as a MeSH term, combined with “pregnancy” as a MeSH term. As we identified relevant articles, we used Medline’s “Related Articles” function to identify additional articles.

We reviewed references in relevant papers and contacted our colleagues in diabetes, maternal and child health and vital statistics to identify additional references. Finally, using the same terms as above, we used the Google search engine and the New York Academy of Medicine’s grey literature reports to locate additional unpublished studies.

We reviewed all studies meeting the search criteria above for mention of diabetes. We selected articles that: (1) reported their methods in sufficient detail to permit replication; and (2) reported a measure of validity, such as sensitivity, specificity or positive predictive value, or provided enough information to calculate such measures for diabetes. Where necessary, we contacted study authors for additional information and to ensure the accuracy of our calculations.

Results

Between 1989 and 2007, we found reports for 12 studies that compared US birth certificate and/or hospital discharge data with another source of data to assess their validity and included information on diabetes [13, 20, 23–31]. We omitted one study because of insufficient detail in reported methods and results [26]. We omitted another because measures of agreement could not be computed [13]. Because the latter study examined the 2003 birth certificate revision, we have included it in our Discussion.

Table 1 summarizes our results. In Table 1, italics indicate statistics that were not originally reported, but which we calculated for purposes of this review. Study designs, sample sizes and standards of comparison varied widely among the articles; none were optimally designed for our purposes.

Table 1 Validity of birth certificate and hospital discharge data for identifying diabetes-complicated births
Full size table

Specificities were consistently above 98% for both data sources, due in part to the relative rareness of diabetes-complicated births. Sensitivities for birth certificate data identifying PDM ranged from 47% to 52%, median 50% (kappas: min = 0.210, med = 0.497, max = 0.523). Sensitivities for birth certificate data identifying GDM ranged from 46% to 83%, median 65% (kappas: min = 0.545, med = 0.667, max = 0.828). Sensitivities for hospital discharge data (two studies) identifying PDM were 78% and 95% (kappas: 0.839 and 0.964) and for GDM were 71% and 81% (kappas: 0.584 and 0.840).

Four birth certificate studies were not able to distinguish between PDM and GDM [25, 28, 29, 31]. Sensitivities for these studies ranged from 33% to 74%. Where GDM and PDM could be distinguished, 83–97% of cases were GDM [20, 22, 30].

In a particularly large and well-designed study, Lydon-Rochelle et al. [20] compared the accuracy of birth certificate and hospital discharge data, alone and linked, using hospital delivery records as the “gold standard”. They found the combination of hospital discharge and birth certificate data to be more accurate than either alone. Linked birth certificate and hospital discharge data produced sensitivities of 97 and 93% for PDM and GDM, respectively.

Discussion

We identified one validation of the 2003 birth certificate. Foley [13] compared Pennsylvania birth certificate data gathered using the 1989 and 2003 revisions with ICD-9 codes from maternal and newborn hospital records. The study included 57,859 live births occurring at 110 Pennsylvania hospitals during 2002 (old birth certificate) and between January and June 2003 (new birth certificate). Aggregate prevalence of diabetes (PDM and GDM combined) differed significantly in birth certificate vs. hospital data in 2002 (3.9% vs. 4.4%, respectively) but not in 2003 (4.8% and 4.4%, respectively). Results from this comparison thus suggest that diabetes documentation improved with the 2003 revision. This study was not included in our review, however, and the following comments pertain to data from the 1989 birth certificate.

Birth certificate data and hospital discharge data underreported the number diabetes complicated births, whether looking at PDM, GDM or the two combined. Overall, hospital discharge data performed better than birth certificates, marginally so for identifying GDM but substantially so for identifying PDM—particularly when hospital medical records were treated as the comparison standard. The performance of birth certificate data for identifying PDM was consistently poor; birth certificate data typically identified about half of PDM-complicated births, regardless of the standard.

As previously observed, our results suggest that birth certificate data quality is lower for high risk births [32]. Reichman and Hade [29] found the lowest sensitivity of all those we identified among a group of high risk, Medicaid-eligible women. Piper et al. [28] examined births with adverse pregnancy outcomes along with matched controls and found that birth certificate data had higher sensitivity for identifying diabetes among the controls. Dobie et al. [25] also found relatively low sensitivity among births specifically selected to be low risk, but this could be attributable to using hospital and prenatal care records rather than hospital records alone as the comparison standard.

The results of our review suggest that policy analysts currently using birth certificate or hospital discharge data alone should: 1) use birth certificates only to identify GDM, not PDM; 2) focus on assessing trends and disparities, not on measuring the burden in absolute terms; and 3) include a caveat that results under represent the problem. Whenever possible, the two sources should be linked to improve accuracy.

A majority of the studies we reviewed were based on secondary analysis of datasets created for purposes other than validation; only 4 out of 10 were originally designed to validate the specified data source [20, 27, 30, 31]. Only one presented a rationale for their chosen standard of comparison [25]. Standards of comparison for the birth certificate validation studies that we reviewed varied widely, including (alone or combination): hospital discharge data; maternal delivery medical records; infant delivery records; prenatal records; postpartum outpatient records; and maternal interviews.

For hospital discharge data, medical records were the standard in both the studies we identified. Sensitivity was lower in the study that included prenatal care records [22] than the one that included only records of hospital care [20]. The mother’s prenatal care record is likely to be a more accurate comparison than the hospital medical record for pre-existing maternal medical risk factors such as diabetes [25, 29, 33] The presumed gold standard is particularly relevant for comparing the quality of birth certificate and hospital discharge data, because high agreement should be expected between hospital discharge claims and the hospital delivery record from which they derive [25]. Three of the studies we reviewed addressed prenatal care records: Yasmeen [22] included prenatal care records when available as part of the hospital record. Roohan [30] included the hospital record of prenatal care but excluded outpatient prenatal records even when available to ensure comparability across records. Dobie [25] included all hospital and outpatient prenatal and records.

Northam and Knapp recently reviewed studies of birth certificate reliability and validity and identified several challenges to summarizing results [34]. We experienced similar challenges in attempting to identify relevant literature, and we concur with their call for standard terminology. Our review was primarily concerned with validity, meaning in this case the ability of the data source to reflect accurately whether a mother had PDM or GDM during her pregnancy. Reliability, in this case, means consistency between birth certificates and hospital discharge data or between hospital discharge data and hospital medical records. Reliability is important but insufficient for effective surveillance of diabetes during pregnancy. This is one reason we emphasize including outpatient prenatal care records, which we believe are currently the best source of accurate information regarding maternal diabetes status, short of active surveillance.

In future validations of birth certificate and hospital discharge data for collecting information on maternal medical risk factors such as diabetes, we recommend that: 1) the comparison standard should include maternal and infant hospital delivery records as well as maternal prenatal outpatient records; 2) samples should be sufficiently large and study designs stratified to adequately represent diversity in facility location and size, providers delivering and documenting care, and in women giving birth, 3) studies should examine (alone and in combination) multiple data sources such as birth certificates, hospital discharge data, and electronic medical records, and 4) if possible, a standardized protocol should be developed to enable better comparisons across studies, states, and time.

We strongly concur with previous authors in recommending data linkages to improve accuracy [20, 35]. Surveillance of diabetes and other maternal and perinatal health issues would be markedly improved by making such linkages standard public health practice in all the states. We urge that the 2003 birth certificate revision be promptly implemented and rigorously evaluated in as many states as possible. Finally, it is crucial that future studies be designed to assess whether data quality may be changing over time amid rising GDM prevalence, potentially shifting screening and diagnosis practices and with implementation of the 2003 birth certificate revision.

Our review included 10 studies from 8 different states. Despite variation in study designs, sample populations and comparison standards, it is clear that improvement in PDM and GDM reporting are urgently needed, along with implementation and assessment of the revised 2003 birth certificate. Both PDM and GDM are increasing, and their adverse impact on mothers and their children are significant [8, 36, 37]. High quality data are essential to providing timely and appropriate care before, during, and after pregnancy to ensure the long-term health of mothers and infants. Women with a history of GDM are an ideal population on which to focus lifestyle or pharmacological interventions to prevent type 2 diabetes [5]. However, post-partum glucose screening rates remain low among women with GDM [38–40]. Accurate identification of PDM and GDM is required for public health surveillance to guide appropriate population-based strategies and policies supporting healthy pregnancies and follow-up care.