Evaluation of Syndromic Surveillance

Los Alamos National Laboratory has been funded by the Defense Threat Reduction Agency to determine the relevance of data streams for an integrated global biosurveillance system. We used a novel method of evaluating the effectiveness of data streams called the “surveillance window”. The concept of the surveillance window is defined as the brief period of time when information gathered can be used to assist decision makers in effectively responding to an impending outbreak. We used a stepwise approach to defining disease specific surveillance windows;

1. Timeline generation through historical perspectives and epidemiological simulations.
2. Identifying the surveillance windows between changes in “epidemiological state” of an outbreak.
3. Data streams that are used or could have been used due to their availability during the generated timeline are identified. If these data streams fall within a surveillance window, and provide both actionable and non-actionable information, they are deemed to have utility.

Objective

The goal of this project is the evaluation of data stream utility in integrated, global disease surveillance. This effort is part of a larger project with the goal of developing tools to provide decision-makers with timely information to predict, prepare for, and mitigate the spread of disease.

Data is collected daily by the DOHMH from 49 of the 52 NYC EDs, representing approximately 95% of all ED visits in NYC. Variability in data fields between and within EDs has been noticed for some time. Differences in chief complaint (CC) characteristics and inconsistent availability of data elements, such as disposition and diagnosis, suggest that procedures, coding practices and health information systems (HIS) are not standardized across all NYC EDs, and may change within EDs. These differences may have an unapparent effect on the DOHMH’s ability to consistently categorize ED visits into syndrome groupings, which may alter how syndromic trends are analyzed. Prior to this project, the DOHMH had no method in place to regularly capture, evaluate or utilize this level of ED-specific information.

Objective

To describe the development, implementation, and analysis of a hospital based emergency department (ED) survey and site visit project conducted by the New York City (NYC) Department of Health and Mental Hygiene (DOHMH).

The U.S. Environmental Protection Agency (EPA) designed a program to pilot multi-component contamination warning systems (CWSs), known as the “Water Security initiative (WSi).” The Cincinnati pilot has been fully operational since January 2008, and an additional four pilot utilities will have their own, custom CWSs by the end of 2012. A workshop amongst the pilot cities was conducted in May 2012 to discuss lessons learned from the design, implementation, operation, maintenance, and evaluation of each city’s PHS component.

Objective

This paper describes the lessons learned from operation and maintenance of the public health surveillance (PHS) component of five pilot city drinking water contamination warning systems (CWS) including: Cincinnati, New York, San Francisco, Philadelphia, and Dallas.

Cold weather exposure-related injuries range from hypothermia to less severe conditions such as frost bite, trench foot, and chilblains, which are all preventable causes of mortality and morbidity. In recent years, NYC has successfully used syndromic surveillance of heat-related ED visits to inform emergency response during heat waves. Similar timely surveillance of cold-exposure related injuries could also inform public health protection measures during severe winter weather or cold season power outages. We conducted a retrospective analysis to compare hypothermia and cold-injury patient case characteristics, as well as temporal and meteorological correlates, between syndromic surveillance data and hospital discharge data.

Objective:

1) Develop cold exposure-related injury syndromic case definitions

2) use historical data to compare trends among cases identified in syndromic surveillance and cases identified in NY Statewide Planning and Research Cooperative System (SPARCS) hospital discharge data to evaluate representativeness and

3) develop regression models to examine relationships with cold weather conditions, and compare relationships across case definitions and data sources.

 

Maine has been conducting syndromic surveillance since 2007 using the Early Aberration Reporting System (EARS). An evaluation of the syndromic surveillance system was conducted to determine if system objectives are being met, assess the system’s usefulness, and identify areas for improvement. According to CDC’s Guidelines for Evaluating Public Health Surveillance Systems, a surveillance system is useful if it contributes to the timely prevention and control of adverse health events. Acceptability includes the willingness of participants to report surveillance data; participation or reporting rate; and completeness of data.

Objective:

To assess the usefulness and acceptability of Maine’s syndromic surveillance system among hospitals who currently participate.

 

Syndromic surveillance data has predominantly been used for surveillance of infectious disease and for broad symptom types that could be associated with bioterrorism. There has been a growing interest to expand the uses of syndromic data beyond infectious disease. Because many of these conditions are specific and can be swiftly diagnosed (as opposed to infectious agents that require a lab test for confirmation) there could be added value in using the ICD9 ED discharge diagnosis field collected by SS. However, SS discharge diagnosis data is not complete or as timely as chief complaint data. Therefore, for the time being SS chief complaint data is relied on for non-infectious disease surveillance. SPARCS data are based on clinical diagnoses and include information on final diagnosis, providing a means for comparing the chief complaint (from SS) to a diagnosis code (from SPARCS), for evaluating how well the syndrome is captured by SS and for assessing if it would be advantageous to get SS ED diagnosis codes in a more timely and complete manner.

Objective:

To evaluate several non-infectious disease related syndromes that are based on chief complaint (cc) emergency department (ED) syndromic surveillance (SS) data by comparing these with the New York Statewide Planning and Research Cooperative System (SPARCS) clinical diagnosis data. In particular, this work compares SS and SPARCS data for total ED visits and visits associated with three noninfectious disease syndromes, namely asthma, oral health and hypothermia.

 

The NYC Department of Health and Mental Hygiene (DOHMH) ED syndromic surveillance system receives data from 95% of all ED visits in NYC totaling 4 million visits each year. The data include residential ZIP code as reported by the patient. ZIP code information has been used by the DOHMH to separate visits into NYC and nonNYC for analysis; and, a closer examination of non-NYC visits may further inform disease surveillance.

Objective

To classify visits to NYC emergency departments (ED) into NYC residential, NYC PO Box or commercial building, commuters to NYC, and out-of-town visitors. To describe patterns in each group, to evaluate how they differ, and to consider how the differences can affect syndromic surveillance analyses and results.

Understanding your data is a fundamental pillar of disease surveillance success. With the increase in automated, electronic surveillance tools many public health users have begun to rely on those tools to produce reports that contain processed results to perform their daily jobs. These tools can focus on the algorithm or visualizations needed to produce the report, and can easily overlook the quality of the incoming data. The phrase “garbage in, garbage out” is often used to describe the value of reports when the incoming data is not of high quality. There is a need then, for systems and tools that help users determine the quality of incoming data.

Objective

The objective of this project was to develop visualizations and tools for public health users to determine the quality of their surveillance data. Users should be able to determine or be warned when significant changes have occurred to their data streams, such as a hospital converting from a free-text chief complaint to a pick list. Other data quality factors, such as individual variable completeness and consistency in how values are mapped to standard system selections should be available to users. Once built, these new visualizations should also be evaluated to determine their usefulness in a production disease surveillance system.

A decade ago, the primary objective of syndromic surveillance was bioterrorism and outbreak early event detection (EED. Syndromic systems for EED focused on rapid, automated data collection, processing and statistical anomaly detection of indicators of potential bioterrorism or outbreak events. The paradigm presented a clear and testable surveillance objective: the early detection of outbreaks or events of public health concern. Limited success in practice and limited rigorous evaluation, however, led to the conclusion that syndromic surveillance could not reliably or accurately achieve EED objectives. At the federal level, the primary rationale for syndromic surveillance shifted away from bioterrorism EED, and towards allhazards biosurveillance and SA. The shift from EED to SA occurred without a clear evaluation of EED objectives, and without a clear definition of the scope or meaning of SA in practice. Since public health SA has not been clearly defined in terms of operational surveillance objectives, statistical or epidemiological methods, or measurable outcomes and metrics, the use of syndromic surveillance to achieve SA cannot be evaluated.

Objective

Review concept of situation awareness (SA) as it relates to public health surveillance, epidemiology and preparedness. Outline hierarchical levels and organizational criteria for SA. Initiate consensus building process aimed at developing a working definition and measurable outcomes and metrics for SA as they relate to syndromic surveillance practice and evaluation.

Objective:

To describe disease and illness surveillance utilized during the 2012 Republican National Convention (RNC) held August 26-30, 2012 in Tampa, FL.

Introduction:

While the Tampa Bay Area has previously hosted other high profile events that required heightened disease surveillance (e.g., two Super Bowls), the 2012 RNC marked the first national special security event (NSSE) held in Florida. The Hillsborough County Health Department (HCHD), in conjunction with the Pinellas County Health Department (PinCHD) coordinated disease surveillance activities during this time frame. This presentation will focus of the disease surveillance efforts of the Hillsborough County Health Department during the 2012 RNC. In addition to the surveillance systems that are used routinely, the HCHD Epidemiology Program implemented additional systems designed to rapidly detect individual cases and outbreaks of public health importance. The short duration of RNC, coupled with the large number of visitors to our area, provided additional surveillance challenges. Tropical Storm Isaac, which threatened Tampa in the days leading up to RNC, and an overwhelming law enforcement presence likely dissuaded many protestors from coming to Tampa. As a result, a tiny fraction of the number of protestors that were expected actually showed up.