Cochrane Dennis

A number of different methods are currently used to classify patients into syndromic groups based on the patient’s chief complaint (CC). We previously reported results using an “Ngram” text processing program for building classifiers (adapted from business research technology at AT&T Labs). The method applies the ICD9 classifier to a training set of ED visits for which both the CC and ICD9 code are known. A computerized method is used to automatically generate a collection of CC substrings (or Ngrams), with associated probabilities, from the training data. We then generate a CC classifier from the collection of Ngrams and use it to find a classification probability for each patient. Previously, we presented data showing good correlation between daily volumes as measured by the Ngram and ICD9 classifiers.

Objective

Our objective was to determine the optimized values for the sensitivity and specificity of the Ngram CC classifier for individual visits using a ROC curve analysis. Points on the ROC curve correspond to different classification probability cutoffs.

One limitation of syndromic surveillance systems based on emergency department (ED) data is the time and expense to investigate peak signals, especially when that involves phone calls or visits to the hospital. Many EDs use electronic medical records (EMRs) which are available remotely in real time. This may facilitate the investigation of peak signals.

Syndromic surveillance of emergency department(ED) visit data is often based on computerized classifiers which assign patient chief complaints (CC) tosyndromes. These classifiers may need to be updatedperiodically to account for changes over time in the way the CC is recorded or because of the addition of new data sources. Little information is available as to whether more frequent updates would actually improve classifier performance significantly. It can be burdensome to update classifiers which are developed and maintained manually. We had available to us an automated method for creating classifiers thatallowed us to address this question more easily. The “Ngram” method, described previously, creates a CC classifier automatically based on a training set of patient visits for which both the CC and ICD9 are available. This method measures the associations of text fragments within the CC (e.g. 3 characters for a “3-gram”) with a syndromic group of ICD9 codes. It then automatically creates a new CC classifier based on these associations. The CC classifier thus created can then be deployed for daily syndromic surveillance.

Objective

Our objective was to determine if performance of the Ngram classifier for the GI syndrome was improved significantly by updating the classifier more frequently.

The existing New York State Department of Health emergency department syndromic surveillance system has used patient’s chief complaint (CC) for assigning to six syndrome categories (Respiratory, Fever, Gastrointestinal, Neurological, Rash, Asthma). The sensitivity and specificity of the CC computer algorithms that assign CC to syndrome categories are determined by using chart review as the criterion standard. These analyses are used to refine the algorithm and to evaluate the effect of changes in the syndrome definitions. However, the chart review (CR) method is labor intensive and expensive. Using an automated ICD9 code-based assignment as a surrogate for chart review could offer a significant cost reduction in this process and allow us to survey a much larger sample of visits.

Patient’s chief complaint (CC) is often used for syndromic surveillance for bioterrorism and outbreak detection, but little is known about the inter-hospital variability in the sensitivity of this method. Objective: Our objective was to characterize the variability of a gastrointestinal (GI) CC text-matching algorithm.

Ideal anomaly detection algorithms shoulddetect both sudden and gradual changes, while keeping the background false positive alert rate at a tolerable level. The algorithms should also be easy to use. Our objective was to develop an anomaly detection algorithm that adapts to the time series being analyzed and reduces false positive signals.

Objective

Several authors have described ways to introduce artificial outbreaks into time series for the purpose of developing, testing, and evaluating the effectiveness and timeliness of anomaly detection algorithms, and more generally, early event detection systems. While the statistical anomaly detection methods take into account baseline characteristics of the time series, these simulated outbreaks are introduced on an ad hoc basis and do not take into account those baseline characteristics. Our objective was to develop statistical-based procedures to introduce artificial anomalies into time series, which thus would have wide applicability for evaluation of anomaly detection algorithms against widely different data streams.

To evaluate four algorithms with varying baseline periods and adjustment for day of week for anomaly detection in syndromic surveillance data.

Effective anomaly detection depends on the timely, asynchronous generation of anomalies from multiple data streams using multiple algorithms. Our objective is to describe the use of a case manager tool for combining anomalies into cases, and for collaborative investigation and disposition of cases, including data visualization.