Introduction

Birth registration systems have collected information on the health of women and infants for more than a century, serving the dual purpose of establishing the legal identity of each child and providing data for public health purposes [1]. Birth data play a critical role in improving maternal and child health (MCH) and have been used to establish the relationship between smoking and adverse pregnancy outcomes, determine caesarean delivery rates, monitor teen pregnancy rates, determine the risk of low birth weight, and measure racial disparities [2]. In 2003, the National Center for Health Statistics (NCHS) [Centers for Disease Control and Prevention (CDC), Hyattsville, Maryland] coordinated efforts to revise the U.S. Standard Birth Certificate, updating the content and format of the certificate, creating standardized worksheets and guidelines for data collection, and writing specifications for electronic birth registration systems, which as of October 2013 were in use in 81 % (46/57) of vital event jurisdictions [3, 4]. These changes were designed to improve the quality of data used for MCH programs and research, including overall trends in live births and surveillance for specific risk factors and delivery outcomes [5]. However, few evaluations have been conducted to assess the impact of these changes on local jurisdictions and their data users.

New York City (NYC) has one of the largest birth registration systems in the United States, registering approximately 125,000 births annually [6]. In 2008, the NYC Department of Health and Mental Hygiene (DOHMH) revised the birth certificate to incorporate the 2003 national guidelines and modified its electronic birth reporting system to an Internet-based platform. Although birth data are widely used by MCH programs at DOHMH, knowledge is limited regarding how well the birth registration system serves these programmatic purposes. We performed an evaluation to characterize the attributes of the birth registration system in NYC, including its usefulness for MCH programs, simplicity, quality, timeliness, and representativeness, to identify system strengths and weaknesses, and to recommend improvements. Although other users of NYC birth data exist, this focused approached on local MCH programs allows for a more detailed evaluation of system attributes and specifically informs the actions of DOHMH.

Methods

System Description

Figure 1 illustrates the NYC birth registration system, which collects information on all live births within the five boroughs. A live birth is defined by local health code as “the complete expulsion or extraction from its mother of a product of conception, regardless of the duration of pregnancy, which after expulsion or extraction shows evidence of life, such as breathing, beating of the heart, pulsation of umbilical cord or definite movement of voluntary muscles” [7]. Approximately 99 % of births occur at one of 42 birthing facilities in the city, including hospitals (40) and dedicated birthing centers (2) [8]. Appointed administrative or clinical staff at these facilities, known locally as birth registrars, use modified US Standard worksheets to collect information from medical records, clinicians, and mothers for >250 pregnancy-related items. This information includes demographic information for the infant and parents, and medical items (e.g., information on prenatal history and care, insurance coverage, maternal morbidity, characteristics of labor and delivery, and infant outcomes). Birth registrars then enter information into an Internet-based records system, the Electronic Vital Event Registration System (EVERS) [9]. Local health code requires birth attendants at facilities and home births to report births within five business days; facilities with >100 births annually are required to report births electronically [7]. For <1 % of births, paper certificates are mailed to DOHMH and entered into EVERS by staff in the Bureau of Vital Statistics (BVS) [8].

Fig. 1
figure 1

Flow chart of the New York City birth registration system, 2008–2011. EVERS Electronic Vital Event Registration Systems, NCHS National Center for Health Statistics, SSA Social Security Administration, CIR Citywide Immunization Registry, MCH Maternal and Child Health. †Percentages are based on observed reporting methods in 2011 [8]. ††Days are based on reporting requirements from the NYC Health Code [7]

Full size image

BVS staff process legal birth certificates and corrections, conduct data cleaning and analysis, and disseminate data to local, regional, and national partners. BVS ensures system security and confidentiality through rigorous procedures for accessing, editing, storing and requesting data. A formal data use office maintains user agreements governing the timing of dissemination and provides careful oversight of unique identifiers, small cell counts and identification risks, and data uses in accordance with local health code. BVS funds birth registration system activities through its operating budget. System costs include software purchase and maintenance, travel, and personnel (i.e., primarily those personnel who issue certificates and process corrections). The sale of paper certificates generates approximately $4.7 million in revenue annually; however, this money is deposited in the NYC general fund, and a credit is provided to DOHMH through the city’s budgeting process (K. Koshar, personal communication, May 22, 2013). BVS receives limited funds from NCHS and the U.S. Social Security Administration for use of specific variables; DOHMH programs do not contribute funds for the system.

System Attributes

We evaluated the NYC birth registration system using CDC guidelines for evaluating public health surveillance systems [10]. These guidelines were first developed by the CDC in 1988 to assess the efficiency and effectiveness of public health surveillance systems, including medical records, registries, and vital registration systems. In order to assess the NYC birth registration system, we focused on five attributes described in the guidelines that were the most relevant to the system, including usefulness, simplicity, data quality, timeliness and representativeness (excluding flexibility, acceptability, and stability). Our evaluation was exempt from human subject review because Health Department has the legal authority to review birth records for surveillance and evaluation.

Usefulness

To assess the usefulness of the birth registration system, we first identified birth registration data users at DOHMH by reviewing data-sharing agreements for 2008–2011. During November–December 2012, we conducted in-person interviews with one representative from each program and asked about the number, type, and use of birth items.

Simplicity

We evaluated simplicity, or the structure and ease of operating the system, from the perspective of birth registrars, BVS staff, and MCH program representatives. We analyzed responses from a 2009 survey of birth registrars (n = 39), conducted as part of a joint NYC-CDC Data Quality Improvement Project [11], which asked about problematic items and training needs. At least one registrar and/or hospital staff member per facility completed the survey, with the exception of one hospital that did not respond and two birthing centers, which were not included in the project. Program representatives were asked during interviews about their experiences with creating and maintaining data-sharing agreements and accessing and using birth registration data, and about their knowledge of birth registration system operations.

Data Quality

To assess data quality, we calculated the percentage of missing and unknown observations for variables from 2008 to 2011 birth records (n = 502,274). We present items that represent the range of data quality and those items that were most frequently used by programs. We reviewed the results of the Data Quality Improvement Project, which calculated the sensitivity, specificity, and positive predictive value (PPV) of birth records compared with medical records [11]. We also examined EVERS elements and protocols for quality improvement. MCH program representatives were asked about their perceptions of data quality.

Timeliness

We evaluated timeliness by calculating the minimum, median, and maximum number of weekdays between birth and registration, by borough of birth and facility type, and between registration and distribution to programs, using filing dates and data sharing agreements, respectively. We used the Wilcoxon Rank Sum statistic to test for significant differences in the median reporting interval. We also gathered perceptions of timeliness from programs.

Representativeness

To assess representativeness, or the accuracy with which the system describes births across the whole population, we relied on indirect measures that influence or reflect reporting practices, including the legal requirements for birth reporting, administrative needs for birth certificates, and the number of births registered ≥1 year after delivery. Finally, we gathered perceptions from MCH program representatives about the significance of having access to data on all live births in NYC.

Results

System Attributes

Usefulness

Table 1 summarizes the MCH programs at the DOHMH that routinely use birth registration data. The most common use of data was for demographic information to identify women and newborns eligible for services or programs. This includes educational outreach for women who experienced gestational diabetes, referrals for early intervention for low birth weight infants, follow up for newborns who may have been exposed to hepatitis B at birth, and lead abatement services for new parents in apartment buildings with reported lead exposure. Additionally, birth registration data were used to identify the need for population-level interventions. For example, adolescent health programs (e.g., Bronx Teen Connection) used maternal birthdate and ZIP code to target primary prevention efforts in neighborhoods with high teen pregnancy rates.

Table 1 Maternal and child health programs at the New York City Department of Health and Mental Hygiene that routinely use the birth registration system, 2008–2011
Full size table

Programs also used birth registration data to evaluate the impact of programs and policies. For example, Latch On NYC, a hospital-based initiative to support breastfeeding, used birth data collected on infant feeding at the time of delivery to evaluate the impact of breastfeeding-friendly policies on rates of exclusive breastfeeding [12]. Infant feeding data have been disseminated widely to apprise internal and external stakeholders of the program’s progress. Birth registration data also have been used to develop and track agencywide performance measures. For example, teen pregnancy rates, which are derived from birth, abortion and stillbirth data, constitute a key child health indicator in Take Care New York, a DOHMH strategic policy agenda for the City of New York [13].

Data from the birth registration system form the basis for MCH surveillance activities. Birth records provide case information for infant and maternal mortality surveillance and constitute the population from which a sample is drawn for the Pregnancy Risk Assessment Monitoring System (PRAMS), a population-based survey of women who recently gave birth to a live infant [14, 15]. Birth records were also linked with external data sets to provide surveillance around specific events. For example, linked data from the World Trade Center Health Registry were used to study birth outcomes among women exposed to the World Trade Center disaster [16].

Finally, we determined that the majority of programs (11/13) used <40 of the 250 items collected in the system and these items were primarily demographic. Only two programs, PRAMS and the maternal mortality review, used >40 items, including some of the more detailed medical information (e.g., previous preterm birth, induction of labor, infertility treatments). Across all MCH programs, the most frequently used medical variables included infant feeding, previous live birth, insurance coverage, delivery method, gestational age, and birth weight.

Simplicity

On the basis of the Data Quality Improvement Project survey, 77.0 % of registrars (30/49) reported one or more problematic items in the system and cited dissatisfaction with the length, advanced literacy level, or intimate nature of questions for the mother; 61.4 % (25/39) requested additional resources and training on clinical terminology, complex variables (e.g., race/ethnicity and ancestry), and interacting with mothers. DOHMH provides written guidelines on definitions and data collection procedures [17]; no standard training exists across facilities.

MCH program representatives access birth registration data and corresponding documentation through shared files or secure Internet sites. Although all users reported that data files were easy to access and use, only two representatives reported feeling confident in their knowledge about the birth registration process, data quality, or changes in collection and reporting.

Data Quality

Table 2 presents the proportion of 2008–2011 birth records with missing and unknown information on selected demographic and medical variables. Infant demographics (e.g., name, sex, and date and place of birth) and maternal demographics (e.g., last name, age, education, and birthplace) were virtually 100 % complete. Birth weight was also 100 % complete. Items with >3.0 % missing or unknown values over the 4 years included date of first and last prenatal care visit, maternal weight, and pregnancy intention. A limited, but overall positive increase in completeness occurred from 2008 to 2011. For example, the proportion of records with complete information on infant feeding increased 6.3 % (from 93.7 to 99.7 %). The Data Quality Improvement Project results revealed that the birth registration system on average had high specificity (94.5 %) for 18 maternal and infant characteristics, compared with medical records, meaning it correctly classified women and newborns without a given outcome or characteristic [11]. Items also had relatively high PPV (83.6 %), meaning a high proportion of reported cases had the outcome reported. In contrast, the sensitivity, or the proportion of women correctly identified with an outcome, averaged 71.6 % and varied from 30 (e.g., gestational hypertension) to 100 % (e.g., delivery method). Higher sensitivity was identified among frequently used medical items, such as gestational age and birth weight.

Table 2 Percentage of frequently used variables with missing or unknown values, by item, for 2008–2011 birth records (N = 502,274)
Full size table

The electronic reporting system contains multiple data quality specifications, including alerts for out-of-range, missing, or conflicting data. Dedicated BVS quality-improvement staff clean selected variables, including prenatal care, infant feeding, and maternal weight variables; contact facilities to clarify problematic birth records; and provide written data entry guidelines and educational newsletters to birth registrars. Program representatives reported few quality concerns in using the birth records as a source of newborn and parent identifying information.

Timeliness

For 2008–2011, the median interval between delivery and birth registration was 5 business days (range 0–260 days). This interval decreased significantly from 2008 to 2011, from 6 to 5 days (p < 0.0001). Table 3 lists the median interval between delivery and registration by borough of birth, place of birth, type of facility, and median number of births. Overall, the median interval differed significantly by borough of birth; the interval for births occurring in Queens and Brooklyn was significantly longer than Manhattan. The median interval also was significantly higher for births that occurred at home versus facility (10 vs. 5 days, p < 0.0001), at city-operated facilities versus other (6 vs. 5 days, p < 0.0001), and at facilities with less than the median number of births for the 4-year period (<8,420 births) compared with facilities with >8,420 births (9 vs. 7 days, p < 0.0001).

Table 3 Median number of business days between delivery and registration, by borough and place of birth, hospital type, and number of births in New York City, 2008–2011
Full size table

The timing of dissemination to program users was in line with the timeline set forth in data use agreements; however, operational difficulties or competing priorities delayed the process on multiple occasions. On average, BVS released weekly and monthly data 4–6 weeks after registration and preliminary year-end data files 6–7 months after year’s end. Final year-end files and annual summaries presenting key trends in pregnancy outcomes were, on average, published the following December, representing an 11-month delay. The majority of programs (11/13) considered the timing of data dissemination acceptable for their purposes; two representatives preferred to receive data more frequently, especially when program services are time-sensitive.

Representativeness

Persons need birth certificates to prove citizenship, obtain social security numbers, enter schools, and join local sports leagues in NYC, motivating parents to ensure births are registered. Aggressive outreach efforts by BVS staff to facilities and legal requirements for facilities or birth attendants also promote complete registration of births. During 2008–2011, approximately 135 births were registered >1 year after delivery (33 in 2008, 59 in 2009, 11 in 2010, and 32 in 2011). The ability to capture all births in NYC was important to program representatives because data represented the whole city population, forming the denominator of total births for analyses of rates.

Discussion

Using CDC guidelines, we evaluated the attributes and performance of the NYC birth registration system in relationship to local MCH programs. The birth registration system provides DOHMH’s MCH programs valuable data that are timely, complete, and representative of the population. Programs use data to identify persons and populations in need of services, evaluate programs and policies, and conduct surveillance and research. The representativeness of the birth registration system is unique, compared with other systems, and allows for robust population-level surveillance of different demographic and medical variables. Maternal and infant demographics were used most often by programs; additional medical variables (e.g., infant feeding, birth weight, gestational age, delivery method, and insurance coverage) were also critical for evaluation, research, and surveillance. Major system weaknesses were the burden and complexity of collecting information for birth registrars and the inadequate data quality and usefulness of certain medical items.

On the basis of the findings of this evaluation, we developed a number of recommendations for improving the NYC birth registration system. First, training facility staff on the purpose and process of birth registration might improve data quality and simplicity. Despite the availability of written guidelines, birth registrars in NYC reported difficulty collecting and correctly entering information into the system. Moreover, the majority of registrars were interested in more training and resources. On the basis of these findings, BVS is developing an online training program to address registrars’ knowledge gaps. Similar online trainings have effectively improved cause-of-death reporting in NYC [18, 19]. Analogous efforts are under way at the national level; recently, a Birth Data Quality Workgroup at NCHS was charged with developing a national model of training for birth registrars [20].

Second, we recommended continued quality improvement efforts to study and improve items with poor data quality. The Data Quality Improvement Project results, some of which are presented in this paper, gave BVS the opportunity to identify items on the birth record having low validity and thus to identify opportunities for quality improvement initiatives. For example, the results of the Data Quality Improvement Project showed that the sensitivity of any and gestational hypertension on the birth certificate was 38.7 and 33.4 % respectively [11]. By investigating these items, BVS discovered that the EVERS edit rules did not allows registrars to select both gestational and chronic hypertension as risk factors, reducing the quality of these variables. Abstraction of the medical record, as part of a second data quality grant, will allow BVS to see if changes in the edit rule have improved the quality of hypertension information reported on the birth record. This continuous process of quality assessment and improvement will help BVS to identify problematic items, target quality improvement resources, and measure the impact on birth data quality.

Third, further collaboration between BVS and program users is needed to expand the use of birth registration data. Although MCH programs are one of the major users of birth data, representatives were unfamiliar with the range of available variables and registration process. Therefore, greater communication about the birth registration system might increase the number and frequency of birth items used. This process might also help raise awareness and interest among other programs that are not current users. For example, when the Latch On NYC program started in 2011, timely and accurate data related to breastfeeding by facility were needed. Program representatives worked with BVS to establish a data sharing agreement and data have been used broadly for evaluation, media, and funding opportunities. Simultaneously, BVS was able to focus resources on this item, including data cleaning and outreach to facilities, and as a result the number of unknown responses to the infant feeding question decreased substantially over 4 years. MCH program and vital event staff share many of the same goals and by sharing knowledge and resources together they can advocate for the value of birth registration and expand the usefulness of the data.

Finally, future revisions of the birth registration system should seek to remove items that are consistently poor quality and unused. This process would both simplify data collection for birth registrars and allow limited resources to be focused on improving the quality of a smaller number of items. Our findings indicate that that while the NYC birth certificate contains >250 items, the majority of MCH programs use <40 items, or <20 % of the data available. Significant concerns with the quality of some medical items also exist in NYC and in other states [11, 21]. Furthermore, the majority of birth registrars reported that the certificate was too lengthy and had problematic items. While these findings support the need to remove items, any revision of the birth certificate should take into account the other local, state, and national users of NYC birth data, who may utilize data in different ways than DOHMH MCH programs. For example, medical items that capture rare events may seem irrelevant for local programs, but as part of a national data set provide critical surveillance that would not be possible otherwise. Though the adaption of the 2003 revision is still underway in many states, NCHS is currently developing new criteria for removing items which may help to guide the revision process. These recommendations, together with local assessment of the training, quality improvement efforts, and collaboration with wide-ranging data users, can help BVS to identify items for revision or removal.

This study has certain limitations that should be considered. We included only those attributes we considered relevant to this system. We did not directly interview mothers, health care providers, or registrars, but rather relied on previous surveys of birth registration staff. Additionally, the timing of the birth registrar survey, shortly after the new system was put in place, could negatively skew responses; however, based on more recent anecdotal evidence we believe the responses are still representative of the types of barriers staff face. We also did not conduct our own validity study; therefore, the results presented on sensitivity, specificity, and PPV are subject to the same limitations as the Data Quality Improvement Project [11]. In particular, this study relied on the medical record as a gold standard, which is not always complete or accurate. In our resource calculation, we did not include costs incurred by facilities for collection and entry of birth records. Finally, our list of data users does not include every MCH program at the Health department or those outside DOHMH (e.g., community-based organizations, foundations, or academic institutions).

Conclusion

Recent revisions to birth registration systems have focused on improving the quality of MCH data. Although data are used broadly for public health purposes, studies rarely have evaluated the structure and performance of these systems at the local level. We present a comprehensive evaluation of the NYC birth registration system, documenting system strengths and weaknesses and providing recommendations for further improvement. We determined that the NYC birth registration system provides timely, representative, and useful information for local MCH programs; however, medical data quality and usefulness varied. We recommend reducing the number of items collected, training birth registrars, continuing research and targeted efforts to improve data quality, and increasing collaboration with MCH programs and other data users. Birth registration systems have become increasingly complex in recent years and are relied on for establishing the legal identity of a child, surveillance, and programmatic purposes; therefore, ongoing evaluation and improvement are essential.