Evaluation of Syndromic Surveillance

Evaluation is a major topic in order to enhance syndromic surveillance. In May 2004, a CDC working group developed a framework for evaluating public health surveillance systems for early detection of outbreaks. This framework has been used to evaluate some civilian and also some military syndromic surveillance systems, as the French system 2SE FAG (Surveillance spatiale des épidémies au sein des forces armées en Guyane) and the UK system RMS (Real time Medical Surveillance). Those systems have been set up since the 2002 Prague summit. But because the objectives and the functioning of those systems have some military specificities, the current CDC framework was not totally adapted for their evaluation. This study presented a proposal of a new framework for evaluating military syndromic surveillance systems.

Objective

The objective of this study was to propose a new framework for evaluating military surveillance systems for early detection of outbreaks. This one was based on the French and UK military real time surveillance systems.

Los Angeles County Department of Health Services is currently testing SaTScan’s space-time permutation model to assist in identifying aberrant illness activity in the community and determine it’s ability to detect outbreaks through analyzing real-time syndromic data. SaTScan could be useful especially due to its ability to provide geographic locations of outbreaks in the community.

Objective

To determine the usefulness of SaTScan as an outbreak and illness cluster detection tool in syndromic surveillance and to compare to a purely temporal CUSUM algorithm.

Aerial transmission and direct contact are important factors for flu. Consequently, close contact with large groups of people, such as during mass transit, present opportunities for transmission. One protective method that decreases the probabilities of becoming ill is vaccination. The potential health impact of  erminating subway service during a flu epidemic depends on both the potential for transmission and vaccination rates among riders. Mass transit, a major method of transit in NYC, exhibits a non-random distribution of riders based on demographics and socio-economic status. There is also a trend in vaccination rates by demography and socio-economic status. This analysis uses individual-based data on vaccination and ridership to separately predict vaccination and ridership for inclusion in agent-based models that can be used to assess impact of public health interventions.

Objective

Agent-based models (ABMs) have been developed to simulate epidemics including smallpox and pandemic flu. The ABM approach is an effective method to assess the potential impact of interventions on disease spread. Integrating the ABM approach with syndromic surveillance data provides potential benefits such  as permitting a realistic specification of some critical model contact parameters, and permitting synthetic outbreaks to be generated with extremely fine resolution (e.g., age, gender, and address). This would provide the ability to test various clustering detection algorithms – a key component of syndromic surveillance methods. RTI International (the Models of Infectious Disease Agent Study (MIDAS) informatics group) and NYC DOHMH (a premier syndromic surveillance research center) collaborated to create a NYC-ABM of flu transmission. This poster describes implementation of several features required for accurate model specification, including assigning immunization rates and subway ridership. Incorporating subway ridership is of great interest, because a large subway system, like the NYC system, has never been investigated as a contributor of disease spread.

School absenteeism data could be used as an early indicator for disease outbreaks. The increase in absences, however, may be driven by non-sickness related factors. Reason for absence combined with syndrome-specific information might make absenteeism data more useful for early outbreak detection.

Objective

This is a pilot evaluation to determine the usefulness of syndrome-specific school absenteeism data for public health surveillance systems.

The 2003 heat wave in France (15,000 extra deaths in 10 days) led the French institute for public health surveillance to modify its public health surveillance system. One of the major objectives of this program was a real time surveillance based on emergency departments (EDs). Trials experiments started in 2004 with a daily automatic data collection from 20 hospitals in the Paris area. The objectives of this new system were: 1) to detect early all threats for public health; and 2) to measure the impact of an identified phenomena.

In 2006 France was concerned by a new heat wave. It was the opportunity for recording health data during a hot period through this real time system.

Objective

This paper describes the performances of a syndromic surveillance system based on EDs during a heat wave.

Ideal anomaly detection algorithms should detect both sudden and gradual changes, while keeping the background false positive alert rate at a tolerable level. Further, the algorithm needs to perform well when the need is to detect small outbreaks in low-incidence diseases. For example, when surveillance is done based on the specific ICD9 diagnosis of flu rather than a larger syndromic grouping, the baseline counts will generally be low, in the range of 0 or 1 per day even in a large sample of EDs. 

Objective

Our goal was to determine the sensitivity of detection of various inserted outbreak sizes and shapes using a modified Holt-Winters detection algorithm applied to daily flu count data before the flu season and after its peak. We compare our results to C3 of EARS.

On December 14th, 2006, a severe windstorm in western Washington caused hundreds of thousands of residents to lose power. On December 15, 2006, there was a surge in emergency department (ED) visits for patients presenting with signs of acute carbon monoxide (CO) poisoning. A Public Health investigation was initiated following the storm to determine the extent of CO poisoning due to the windstorm. A retrospective analysis was later undertaken to evaluate how well our syndromic surveillance system was able to identify patients who presented to area EDs with carbon monoxide poisoning.

Objective

We evaluated the performance of our ED syndromic data for detecting visits associated with CO poisoning.

In September 2004, Kingston, Frontenac and Lennox and Addington Public Health began a 2-year pilot project to develop and evaluate an Emergency Department Chief Complaint Syndromic Surveillance System in collaboration with the Ontario Ministry of Health and Long Term Care – Public Health Branch, Queen’s University, Public Health Agency of Canada, Kingston General Hospital and Hotel Dieu Hospital. At this time, the University of Pittsburgh’s Real-time Outbreak and Disease Surveillance (RODS, Version 3.0) was chosen as the surveillance tool best suited for the project and modifications were made to meet Canadian syndromic surveillance requirements. To evaluate the design and implementation of the system, a multi-sectored approach to evaluation was taken. Individual evaluations of the process, technical aspects and of cost/benefit were conducted to demonstrate proof of concept and the associated costs. An overall outcome or effectiveness evaluation will take place in spring 2006.

Objective

This paper outlines the approach used to evaluate an emergency department syndromic surveillance system on the following areas: process and outcome, cost/benefit and technical.

Following the heat wave that scorched France in August 2003 a national daily gathering of mortality data was decided in link with the National Institute for Statistics and Economic Studies (Insee). Such gathering is based on Public Records Office equipped with the appropriate software in order to transmit their data to Insee. Then data received daily are transmitted automatically to the National Institut for Health Surveillance. Data are encrypted and transmitted 7 days per week through direct FTP in a pretermined format. For each death certificate, the following information are recorded: zip code, age, sex, date of death.

A pilot test started in June, 2004 with 147 cities for one year. The good evaluation of the system pushed to enlarge it to all eligible cities in France. The enlarged system started on November 1, 2005 and concerned 1,152 Public Records Office which represents around 75% of the daily French mortality.

Reunion Island (population 770,000) is being affected by the most important outbreak of chikungunya disease ever described in the medical literature. Between March 1, 2005 and May 30, 2006, an estimated 255,000 cases have been reported in this French territory located in the Indian Ocean. The vast majority of the cases have been occurring from mid-December, with a peak of 45000 cases week 5, 2006.

The disease is a self-limiting febrile viral disease characterised by arthralgia or arthritis. The symptoms may last for several months but recovery was, until now, considered universal.

Objective

This paper describes the on going surveillance of mortality during the largest outbreak of chikungunya ever known. It is based on a new automatic gathering of mortality data and it is also the first opportunity to test this system in real condition.

A U.S. Department of Defense program is underway to assess health surveillance in resource-poor settings and to evaluate the Early Warning Outbreak Reporting System. This program has included several information-gathering trips, including a trip to Lao PDR in September, 2006.

Objective

This modeling effort will provide guidance for policy and planning decisions in developing countries in the event of an acute respiratory illness epidemic, particularly an outbreak with pandemic potential.