Improving Data Quality in New York City's Immunization Registry
- Annual birth cohort: 124,252 (1998)
- Geographic area covered: Five boroughs of New York City
- Legislation: 1994 health code amendment authorizes registry and mandates reporting of all immunizations for children under age 8 to the registry effective January 1, 1997
- Percentage of immunizations given in private sector: 65 percent
- Other sources of funding: Federal, state, city
In October 1997, when CIR applied for the All Kids Count grant, 60 percent of private, office-based providers were sending immunization reports to the registry in response to a mandate that went into effect in January that year. By the end of the All Kids Count program, that total had increased to 91.5 percent, a gain of 31.5 percentage points. Although the mandate undoubtedly played a large role in motivating providers to participate in the registry, the registry's marketing program including regular contact with providers, provider research (interviews, surveys, focus groups), and the availability of multiple reporting methods, helped to ensure submission of immunization data to the registry.
Data submission by a high percentage of providers in and of itself, however, is not enough to ensure a registry's success. Incomplete, irregular, or inaccurate reporting can seriously undermine the quality of the data in the registry and result in a loss of confidence by those who count on the registry to provide accurate immunization data, including providers, parents, and health plans.
In 1998, it became clear that CIR data was not high quality. While some record fragmentation and duplication was expected to occur in a database containing records on 1.7 million children provided by some 1,200 different sources (104 public providers, 1,083 private providers, and 30 managed care organizations), duplication approached three records for every two children. In addition to the sheer volume of data from such a large number of diverse sources, the registry lacked the ability to match records based on a combination of fields, including name, date of birth, gender, parent information, and address, all of which are frequently reported incorrectly. The registry tried to manually cull duplications from the database, but realized such as task would involve some 1,700 hours reviewing 260,000 record pairs. An automated process was needed.
CIR addressed the problem through quality assurance and technical initiatives. A quality assurance protocol, piloted in 10 sites then implemented citywide, involved conducting a chart review of a representative sample of children in a provider's office. The data are then compared to data reported to the registry, as well as to the data already in the registry. Registry staff give the provider a written feedback report and meet with him/her to develop a plan for improving the completeness and accuracy of data reported to the registry. This helps to eliminate data quality problems on the "front end."
To address data quality on the "back end," two sophisticated computer programs were developed with expert consultants. MEDD (Maximum Entropy De-Duplication) and Smart Search identify and merge vast quantities of fragmented or duplicate records with up to 99 percent accuracy. These powerful automated de-duplication systems decreased the manpower required to review records by more than 98 percent, while significantly improving the quality of the data. As of February 1999, approximately 35 percent of the duplicate records had been eliminated. Ongoing use of quality assurance protocols and technical de-duplication solutions will create a constant process of data quality improvement that will help to ensure the usefulness of CIR data to parents, providers, and health plans. The data quality lessons learned by CIR and communicated widely by registry staff through publications and meeting presentations have considerable value for other registries, especially those in large urban areas with large immigrant populations.
Major CIR initiatives for the future include development of new ways for providers to report to the registry, thus making it less burdensome and, most likely, more accurate. CIR also is pursuing additional ways to give providers useful information back, e.g., reports that can be generated on demand, thus reinforcing accurate reporting.
In the longer-term, CIR and the Lead Quest program within the New York Department of Health are developing an integrated system that will leverage resources of the immunization and lead programs. Both target the same population, yet the separate systems currently are unable to relate data for the same child. Built on the CIR infrastructure and based on a Master Child Index, the integrated information system would allow both health program staff and providers access to immunization and lead status on-line. Other systems, such as asthma, could be added to the system as modules.
GRANT DETAILS & CONTACT INFORMATION
New York City: Citywide Immunization Registry (CIR)
City of New York Department of Health (New York, NY)
Dates: February 1998 to May 2000
Program Director: David A. Ross, Sc.D.
Former Program Director: William H. Foege, M.D., M.P.H. (December 1991 through July 1997)
Former Program Director: William C. Watson, Jr., M.P.A. (August 1997 through July 2001)