Cochrane Dennis

The CDC recently developed sub-syndromes for classifying disease to enhance syndromic surveillance of natural outbreaks and bioterrorism. They have developed ICD9 classifiers for six GI Illness subsyndromes: Abdominal Pain, Nausea and Vomiting, Diarrhea, Anorexia, Intestinal infections, and Food poisoning. If the number of visits for sub-syndromes varies significantly by age it may impact the design of outbreak detection methods.

Objective

We hypothesized that the percentage of visits for the GI sub-syndromes varied significantly with age.

In order to detect influenza outbreaks, the New York State Department of Health emergency department (ED) syndromic surveillance system uses patients’ chief complaint (CC) to assign visits to respiratory and fever syndromes. Recently, the CDC developed a more specific set of “sub-syndromes” including one that included only patients with a CC of flu or having a final ICD9 diagnosis of flu. Our own experience was that although flu may be a common presentation in the ED during the flu season, it is not commonly diagnosed as such. Emergency physicians usually use a symptomatic diagnosis in preference, probably because rapid testing is generally unavailable or may not change treatment. The flu subsyndrome is based on a specific ICD9 code for influenza. It is unknown whether patient visits that meet these restrictive criteria are sufficiently common to be of use, or whether patients who identify themselves as having the flu are correct.

Objective

Our objective was to examine the CC and ICD9 classifiers for the influenza sub-syndrome to assess the frequency of visits and the agreement between the CC, ICD9 code and chart review for these patient visits.

The Centers for Disease Control and Prevention BioSense has developed chief complaint (CC) and ICD9 sub syndrome classifiers for the major syndromes for early event detection and situational awareness. The prevalence of these sub-syndromes in the emergency department population and the performance of these CC classifiers have been little studied. Chart reviews have been used in the past to study this type of question but because of the large number of cases to review, the labor involved would be prohibitive. Therefore, we used an ICD9 code classifier for a syndrome as a surrogate by chart reviews to estimate the performance of a CC classifier.

Objective

To determine the prevalence of the sub-syndromes based on the ICD9 classifiers, and to determine the sensitivity, specificity, positive predictive value and negative predictive value of CC classifiers for the sub-syndromes associated with the respiratory and gastrointestinal syndromes using the ICD9 classifier as the criterion standard.

Syndromic surveillance of emergency department (ED) visit data is often based on computer algorithms which assign patient chief complaints (CC) and ICD code data to syndromes. The triage nurse note (NN) has also been used for surveillance. Previously we developed an “NGram” classifier for syndromic surveillance of ED CC in Italian for detection of natural outbreaks and bioterrorism. The classifier is developed from a set of ED visits for which both the ICD diagnosis code and CC are available by measuring the associations of text fragments within the CC (e.g. 3 characters for a “3-gram”) with a syndromic group of ICD codes. We found good correlation between daily volumes by the ICD10 classifier and estimated by NGrams. However, because the CC was limited to 23 options based on the pick list, it might be possible to obtain results as good as the NGram method or better using a simpler probabilistic approach. Also, in addition to the CC, the Italian data included a free-text NN note. We might be able achieve improved performance by applying the n-gram method to the NN or the CC supplemented by the NN.

Objective

Our objective was to compare the performance of the NGram CC classifier to two discrete classifiers based on probabilistic associations with the CC pick list items. Also, we wished to determine the performance of the NGram method applied to CC alone, NN alone, and CC plus NN.

Syndromic surveillance of emergency department (ED) visit data is often based on computer algorithms which assign patient chief complaints (CC) to syndromes. ICD9 code data may also be used to develop visit classifiers for syndromic surveillance but the ICD9 code is generally not available immediately, thus limiting its utility. However, ICD9 has the advantages that ICD9 classifiers may be created rapidly and precisely as a subset of existing ICD9 codes and that the ICD9 codes are independent of the spoken language. If a classifier based on ICD9 codes could be used to automatically create the code for a chief-complaint assignment algorithm then CC algorithms could be created and updated more rapidly and with less labor. They could also be created in multiple spoken languages. We had developed a method for doing this based on an “ngram” text processing program adapted from business research technology (AT&T Labs). The method applies the ICD9 classifier to a training set of ED visits for which both the CC and ICD9 code are known. A computerized method is used to automatically generate a collection of CC substrings with associated probabilities, and then generate a CC classifier program. The method includes specialized selection techniques and model pruning to automatically create a compact and efficient classifier.

Objective

Our objective was to determine how closely the performance of an ngram CC classifier for the gastrointestinal syndrome matched the performance of the ICD9 classifier.

Previously we used an “N-Gram” classifier for syndromic surveillance of emergency department (ED) chief complaints (CC) in English for bioterrorism. The classifier is trained on a set of ED visits for which both the ICD diagnosis code and CC are available by measuring the associations of text fragments within the CC (e.g. 3 characters for a “3-gram”) with a syndromic group of ICD codes. Because the ICD system is language independent, the technique has the potential advantage of rapid automated deployment in multiple languages. Our objective was to apply the N-Gram method to a training set of Turkish ED data to create a Turkish CC classifier for the respiratory syndrome (RESP) and determine its performance in a test set.

Objective

To determine how closely the performance of an ngram CC classifier for the RESP syndrome matched the performance of the ICD9 classifier.

Previously we developed an “Ngram” classifier for syndromic surveillance of emergency department (ED) chief complaints (CC) in Turkish for bioterrorism. The classifier is developed from a set of ED visits for which both the ICD diagnosis code and CC are available. A computer program calculates the associations of text fragments within the CC (e.g. 3 characters for a “3-gram”) with a syndromic group of ICD codes. The program then generates an algorithm which can be deployed to evaluate chief complaint data in real-time. However, the N-gram method differs from most other classifiers in that it assigns a probability that each visit falls within the syndrome rather than ruling the visit “in” or “out” of the syndrome. It is possible to dichotomize visits “in” or “out” using N-grams by choosing a cut-off sensitivity for the n-grams used, but this affects the specificity of the method. The effect of this trade-off is best measured by a receiveroperator curve.

Objective

Our objective was to determine the sensitivity and specificity of the Ngram CC classifier for individual ED visits. We also wish to compare these results to those obtained when we substituted anglicized characters for 6 problematic Turkish characters.

The Centers for Disease Control and Prevention BioSense project has developed chief complaint (CC) and ICD9 sub-syndrome classifiers for the major syndromes for early event detection and situational awareness. This has the potential to expand the usefulness of syndromic surveillance, but little data exists evaluating this approach. The overall performance of classifiers can differ significantly among syndromes, and presumably among subsyndromes as well. Also, we had previously found that the seasonal pattern of diarrhea was different for patients < 60 months of age (younger) and for patients > 60 months of age (older).

Objective

Using chart review as the criterion standard to estimate the sensitivity, specificity, positive predictive value and negative predictive value of New York State hospital emergency department CC classifiers for patients < 60 months of age and > 60 months of age for the gastrointestinal (GI) syndrome and the following GI sub-syndromes: “abdominal pain”, “nausea-vomiting” and “diarrhea”.

Ideal anomaly detection algorithms should detect both sudden and gradual changes, while keeping the background false positive alert rate at a tolerable level. Further, the algorithm needs to perform well when the need is to detect small outbreaks in low-incidence diseases. For example, when surveillance is done based on the specific ICD9 diagnosis of flu rather than a larger syndromic grouping, the baseline counts will generally be low, in the range of 0 or 1 per day even in a large sample of EDs. 

Objective

Our goal was to determine the sensitivity of detection of various inserted outbreak sizes and shapes using a modified Holt-Winters detection algorithm applied to daily flu count data before the flu season and after its peak. We compare our results to C3 of EARS.