Evaluation

In November of 2011 BioSense 2.0 went live to provide tools for public health departments to process, store, and analyze meaningful use syndromic surveillance data. In February of 2012 ESSENCE was adapted to support meaningful use syndromic surveillance data and was installed on the Amazon GovCloud. Tarrant County Public Health Department agreed to pilot the ESSENCE system and evaluate its performance compared to a local version ESSENCE they currently used. The project determined the technical feasibility of utilizing the Internet cloud to perform detailed public health analysis, necessary changes needed to support meaningful use syndromic surveillance data, and any public health benefits that could be gained from the technology or data.

Objective:

This project represents collaboration among CDC’s BioSense Program, Tarrant County Public Health and the ESSENCE Team at the Johns Hopkins University APL. For over six months the Tarrant County Public Health Department has been sending data through the BioSense 2.0 application to a pilot version of ESSENCE on the Amazon GovCloud. This project has demonstrated the ability for local hospitals to send meaningful use syndromic surveillance data to the Internet cloud and provide public health officials tools to analyze the data both using BioSense 2.0 and ESSENCE. The presentation will describe the tools and techniques used to accomplish this, an evaluation of how the system has performed, and lessons learned for future health departments attempting similar projects.

 

During summer 2012, Washington State Department of Health (WA DOH) surveyed ILINet providers and found that more than half either utilize their electronic medical record system (EMRS) to gather and report weekly ILINet data, or intend to implement queries to do so in the future. There are a variety of EMRS being used state-wide, and providers that currently utilize these systems to report ILINet data apply a wide range of methods to query their data. There exists great interest in the evaluation of ambulatory care data within the context of Meaningful Use and little research is published in this area. WA DOH sought to evaluate electronic data from WA outpatient clinic networks in order to determine if a syndromic ILI definition previously validated for emergency department (ED) data accurately identified ILI visits in electronic ambulatory care data.

Objective:

To determine if a syndromic influenza-like illness (ILI) definition previously validated for emergency department (ED) data accurately identified ILI visits in electronic ambulatory care data.

Medical claims and EHR data sources offer the potential to ascertain disease and health risk behavior prevalence and incidence, evaluate the use of clinical services, and monitor changes related to public health interventions. Passage of the HITECH Act of 2009 supports the availability of standardized EHR data for use by public health officials to obtain actionable information. While full adoption of EHRs is still years away, there are presently publicly- and commerciallyavailable EHR and medical claims data sets that could enhance public health surveillance at a national, regional and state level. The purposes of this evaluation were to i.) demonstrate the feasibility of gaining access to such data, ii.) evaluate their ability to augment current surveillance activities by developing measures for twenty separate healthcare indicators (e.g., HIV screening), iii.) evaluate each data source across a set of criteria needed for an effective surveillance system, and iv.) assess the ability of the data sources to evaluate changes in healthcare utilization and preventive services that may be a result of the 2009 Health Reform legislation.

Objective:

To assess the utility of inpatient and ambulatory clinical data compiled by public and commercial sources to enhance the Centers for Disease Control and Prevention’s surveillance activities.

 

Cholera causes frequent outbreaks in Nigeria, resulting in mortality. In 2010 and 2011, 41,936 cases (case fatality rate [CFR]-4.1%) and 23,366 cases (CFR-3.2%) were reported (1). Reported cases in Nigeria by week 26, 2012 was 309 (CFR-1.29%) involving 20 Local Government Areas in 6 States. In Nigeria, there are currently eleven (11) States including Niger state at high risk for cholera/bloodless diarrhea outbreaks. In 2011, Niger state had 2472 cholera cases (CFR-2%) and 45,111 other diarrhea diseases cases, recorded in more than half of state Purpose of surveillance system is to ensure early detection of cholera and other diarrheal cases and to monitor trends towards evidencebased decision for management, prevention and control.

Objective:

To determine how the cholera and other diarrheal disease surveillance system in Niger state is meeting its surveillance objectives, to evaluate its performance and attributes and to describe its operation to make recommendations for improvement.

 

The evaluation of biosurveillance system components is a complex, multi-objective decision that requires consideration of a variety of factors. Multi-Criteria Decision Analysis provides a methodology to assist in the objective analysis of these types of evaluation by creating a mathematical model that can simulate decisions. This model can utilize many types of data, both quantitative and qualitative, that can accurately describe components. The decision-maker can use this model to determine which of the system components best accomplish the goals being evaluated. Before MCDA can be utilized effectively, an evaluation framework needs to be developed. We built a robust framework that identified unique metrics, surveillance goals, and priorities for metrics. Using this framework, we were able to use MCDA to assist in the evaluation of data streams and to determine which types would be of most use within a global biosurveillance system.

Objective:

The use of Multi-Criteria Decision Analysis (MCDA) has traditionally been limited to the field of operations research, however many of the tools and methods developed for MCDA can also be applied to biosurveillance. Our project demonstrates the utility of MCDA for this purpose by applying it to the evaluation of data streams for use in an integrated, global biosurveillance system.

 

Telephone triage is a relatively new data source available to biosurveillance systems.1-2Because early detection and warning is a high priority, many biosurveillance systems have begun to collect and analyze data from non-traditional sources [absenteeism records, overthe-counter drug sales, electronic laboratory reporting, internet searches (e.g. Google Flu Trends) and TT]. These sources may provide disease activity alerts earlier than conventional sources. Little is known about whether VA telephone program influenza data correlates with established influenza biosurveillance.

Objective:

To evaluate the utility and timeliness of telephone triage (TT) for influenza surveillance in the Department of Veterans Affairs (VA).

A decade ago, the primary objective of syndromic surveillance was bioterrorism and outbreak early event detection (EED. Syndromic systems for EED focused on rapid, automated data collection, processing and statistical anomaly detection of indicators of potential bioterrorism or outbreak events. The paradigm presented a clear and testable surveillance objective: the early detection of outbreaks or events of public health concern. Limited success in practice and limited rigorous evaluation, however, led to the conclusion that syndromic surveillance could not reliably or accurately achieve EED objectives. At the federal level, the primary rationale for syndromic surveillance shifted away from bioterrorism EED, and towards allhazards biosurveillance and SA. The shift from EED to SA occurred without a clear evaluation of EED objectives, and without a clear definition of the scope or meaning of SA in practice. Since public health SA has not been clearly defined in terms of operational surveillance objectives, statistical or epidemiological methods, or measurable outcomes and metrics, the use of syndromic surveillance to achieve SA cannot be evaluated.

Objective

Review concept of situation awareness (SA) as it relates to public health surveillance, epidemiology and preparedness. Outline hierarchical levels and organizational criteria for SA. Initiate consensus building process aimed at developing a working definition and measurable outcomes and metrics for SA as they relate to syndromic surveillance practice and evaluation.