Data Quality

Completeness of public health information is essential for the accurate assessment of community health progress and disease surveillance. Yet challenges persist with respect to the level of completeness that public health agencies receive in reports submitted by health care providers. Missing and incomplete data can jeopardize information reliability and quality resulting in inaccurate disease evaluation and management (1). Additionally, incomplete data can prolong the time required for disease investigators to complete their work on a reported case. Thus, it is important to determine where the scarcity of information is coming from to recognize the characteristics of provider reporting.

Objective

To examine the completeness of data elements required for notifiable disease surveillance from official, provider-based reports submitted to a local health department.

Security threats and the recent emergence of avian influenza in Europe have heightened the profile of and need for a good surveillance strategy during such events. The two main rationales for enhanced infectious disease surveillance at mass events include a perceived increased risk of infectious disease events and a need to detect and respond to events more quickly. Moreover, the requirements of the International Health Regulations (IHR) issued by the World Health Organization (WHO), which take effect in mid-2007, define the need for timely reporting of infectious diseases during international mass events [1]. Therefore, an enhanced surveillance, based on Germany’s pre-existing system of mandatory notifications was conducted in the12 World Cup cities.

Objective

In this abstract, we describe the major findings of an evaluation of our enhanced infectious disease surveillance activities during the FIFA Soccer World Cup 2006 in Germany.

Meaningful Use has increased interest in submission of ELR to public health agencies, prompting these agencies to analyze their reporting process. Tennessee’s reporting regulations require anyone with knowledge of or suspecting a reportable disease or event report to the local health department. Although it is understood that laboratories are more diligent and routine reporters, focus in listing of these events is from the healthcare provider perspective. Public health agencies must acknowledge the differences in provider case reporting and laboratory result reporting. Despite Tennessee Department of Health's (TDH) required use of standardized vocabulary for ELR such as Logical Observation Identifiers Names and Codes (LOINC) to identify the test performed and Systematized Nomenclature of Medicine-Clinical Terms (SNOMED-CT) to identify organism names, ordinal results, and specimen type, internally inconsistent information in messages has been identified. For example, a performed test with LOINC value 13950-1 encodes for a hepatitis A virus IgM antibody test from serum or plasma using an enzyme immunoassay (EIA) and calls for an ordinal result. However the sender describes a Hepatitis C Antibody (Anti HCV) test and provides a numeric result. In order to achieve semantic understanding of the actionable content of ELR messages, a systematic means to document and validate vocabulary is needed.

Objective

To develop a means for validating standardized vocabulary used to report laboratory events via prescribed electronic laboratory reporting (ELR) standards and implementation guides in order to limit internally inconsistent information within ELR messages intended for public health action.

The utility of specific sources of data for surveillance, and the quality of those data, are an ingoing issue in public health(1). Syndromic surveillance is typically conducted as a secondary use of data collected as part of routine clinical practice, and as such the data can be of high quality for the clinical use but of lower quality for the purpose of surveillance. A major data quality issue with surveillance data is that of timeliness. Data used in surveillance typically arrive as a periodic process, inherently creating a delay in the availability of the data for surveillance purposes. Surveillance data are often collected from multiple sources, each with their own processes and delays, creating a situation where the data available for surveillance are accrued piecemeal.

Objective

This abstract discusses the quality issues identified in using Distribute. From 2006 to 2012, the ISDS ran Distribute (2), a surveillance system for monitoring influenza like illness (ILI) and gastroenteritis (GI) ED visits on a nationwide basis. This system collected counts for ILI, GI and total ED visits, aggregated to the level of jurisdiction. The primary data quality issue faced with the Distribute system was that of timeliness due to accrual lag; variable delays in the receipt of surveillance data from sources by jurisdictions together with variable delays in the reporting of aggregate data from jurisdictions to Distribute resulted in data which accrued over time(3).

Abbreviation, misspellings, and site specific terminology may misclassify chief complaints syndromes. The Emergency Medical Text Processor (EMT-P) is system that cleans emergency department chief complaints and returns standard terms. However, little information is available on the implementation of EMT-P in a syndromic surveillance system.

Objective

To describe the implementation and baseline evaluation of EMT-P developed by the University of North Carolina.

In a classical surveillance system one looks for disturbances in the number of cases, but in a spatio-temporal system, not only the number of cases observed but also where they are located is reported. What location is reported, and to which degree of accuracy it is reported are important. At one extreme les near-perfect information about each case, as with contact tracing; at the other extreme we have no information about location; viz. just that the patient exisits, or a temporal system. From maximum spatial precision to no spatial precision, one gains in speed of reporting and privacy; but one loses power to detect outbreaks. For example, in Ozonoff et al. we see that more than one address is better than just a single one. This general point is intuitively appealing, and can be demonstrated. 

Objective

This paper quantifies the effect of not providing full information about the location of patients when dealing with spatio-temporal systems in syndromic surveillance. The study investigates the loss of power to detect clusters when aggregation takes place. 

In September 2004, Kingston, Frontenac and Lennox and Addington Public Health began a 2-year pilot project to develop and evaluate an Emergency Department Chief Complaint Syndromic Surveillance System in collaboration with the Ontario Ministry of Health and Long Term Care – Public Health Branch, Queen’s University, Public Health Agency of Canada, Kingston General Hospital and Hotel Dieu Hospital. At this time, the University of Pittsburgh’s Real-time Outbreak and Disease Surveillance (RODS, Version 3.0) was chosen as the surveillance tool best suited for the project and modifications were made to meet Canadian syndromic surveillance requirements. To evaluate the design and implementation of the system, a multi-sectored approach to evaluation was taken. Individual evaluations of the process, technical aspects and of cost/benefit were conducted to demonstrate proof of concept and the associated costs. An overall outcome or effectiveness evaluation will take place in spring 2006.

Objective

This paper outlines the approach used to evaluate an emergency department syndromic surveillance system on the following areas: process and outcome, cost/benefit and technical.

The North Carolina Disease Event Tracking and Epidemiologic Collection Tool (NC DETECT) receives a designated set of data elements electronically available from 110 emergency departments (EDs) (98%) on at least a daily basis via a third party data aggregator. While automated processes monitor for data quality problems such as improper file formats or missing required elements, data corruption can occur at several stages before receipt, and if undetected, data can appear reliable. Hospitals might map to standard codes incorrectly, data aggregators might manipulate text improperly, or updates might be confused with original records. These inaccuracies cause delays and oversights in identifying events of public health importance.

Objective

This study evaluates the validity of a subset of ED data collected in NC DETECT, as well as measures the effectiveness of the data quality processes in place for this surveillance system.

Data quality for syndromic surveillance extends beyond validating and evaluating syndrome results. Data aggregators and data providers can take additional steps to monitor and ensure the accuracy of the data. In North Carolina, hospitals are mandated to transmit electronic emergency department data to the North Carolina Disease Event Tracking and Epidemiologic Tool (NC DETECT) system at least every 24 hours. Protocols have been established to ensure the highest level of data quality possible. These protocols involve multiple levels of data validity and reliability checks by NC DETECT staff as well as feedback from end-users concerning data quality. Hospitals also participate in the data quality processes by providing metadata including historical trends at each facility.

Objective

The purpose of this project is to describe the initiatives used by the NC DETECT to ensure the quality of ED data for surveillance.