PEDIATRICS Vol. 102 No. 4
October 1998,
p. e48
ELECTRONIC ARTICLE:
Computer-assisted Diagnosis of Pediatric Rheumatic Diseases
Balu H. Athreya*, 
,
May L. Cheh§, and
Lawrence C. Kingsland III§
From the * duPont Hospital for Children, Wilmington, Delaware;
the Thomas Jefferson University, Jefferson Medical College,
Philadelphia, Pennsylvania; and the § Computer Science Branch, National
Library of Medicine, National Institutes of Health, Bethesda, Maryland.
 |
ABSTRACT |
Objective. AI/RHEUM is a multimedia
expert system developed originally to assist in the diagnosis of
rheumatic diseases in adults. In the present study we evaluated the
usefulness of a modified version of this diagnostic decision support
system in diagnosing childhood rheumatic diseases.
Methodology. AI/RHEUM was modified by the addition of 5 new diseases to the knowledge base of the system. Criteria tables for
each of the diseases included in the knowledge base were modified to
suit the needs of children. The modified system was tested on 94 consecutive children seen in a pediatric rheumatology clinic.
Results. AI/RHEUM made the correct diagnosis in 92% of
the cases when the diagnosis was available in the knowledge base of the
system. It was also shown to be effective in the education of pediatric trainees through its multimedia features.
Conclusions. AI/RHEUM is an expert system that may be
helpful to the nonspecialist as a diagnostic decision support system
and as an educational tool.
Key words:
computer-assisted diagnosis,
multimedia,
rheumatic diseases,
expert system.
Computers have come to play a major part in patient care
activities, including literature searches, patient monitoring, and drug
prescriptions. Imaging techniques such as magnetic resonance imaging
and nuclear medicine would be impossible without computer technology.
All of these areas are grouped under computer-based clinical decision
support systems by Habbema,1 who recognized four
levels of medical decision-making: 1) collecting background information
on patients, both general and specific; 2) assessing diagnostic and
therapeutic methods that include algorithms; 3) developing a global
clinical policy, such as protocols and guidelines, which are becoming
increasingly important in this era of cost-containment; and 4)
developing an individualized clinical strategy that includes medical
expert systems.
The term expert system refers to a computer program that
simulates the diagnostic reasoning of an experienced clinician in specific areas of medicine such as rheumatology. Expert systems belong
to the general area of artificial intelligence in which symbolic
reasoning methods are used. An expert system has two components
a
knowledge base and an inference mechanism. The knowledge base can be in
any field of knowledge. In medicine, the knowledge base can include
information about specific diseases and an inference mechanism designed
to work with the knowledge base and with patient data to reach a
conclusion such as a diagnosis or treatment plan.
There are complex software packages that can build general expert
systems, called expert system shells. CTX, written at the National
Library of Medicine (Bethesda, MD), is one such shell.2 AI/RHEUM, a knowledge-based medical consultant system built using the
CTX shell, is a multimedia expert system originally developed by
Kingsland et al3 in the 1980s to assist general physicians with the diagnosis of rheumatic diseases in adults. We now report on a
modification of this system to address childhood rheumatic diseases.
| |
MATERIALS AND METHODS |
AI/RHEUM runs on IBM-compatible personal computers and UNIX
systems. The knowledge base includes diagnostic criteria tables for 54 common rheumatic diseases (Table 1). The criteria tables are designed using the most important clinical and laboratory features of each of the diseases. These tables were originally developed at the University of Missouri, Columbia, MO, and reviewed by
a panel of rheumatologists. More recently, H. James Williams, MD, at
the University of Utah has modified some of these tables.
One of the authors (B.H.A.) modified the criteria tables to suit the
needs of children, prepared a set of screening criteria to enter
patient data into the system, and also added 5 more diseases to the
knowledge base.
AI/RHEUM's reasoning uses the criteria table paradigm with 1 criteria
table for each of the diseases covered. Criteria tables (Fig
1) consist of the following elements: major
criteria, minor criteria, required criteria, and exclusions. The
decision elements may be symptoms, signs, or laboratory data from the
patient data file. The decision element may also be an intermediate
hypothesis derived from a combination of findings. An example of an
intermediate hypothesis is when the data entry includes pain in the
joints and limitation of range of movement and the program generates arthritis as the decision element.
The expert system matches the individual patient findings with the
criteria tables and reaches diagnostic conclusions at three different
levels of certainty: definite, probable, and possible. An example of a
patient with definite diagnosis of Wegener's granulomatosis is given
in Fig 2. When a patient has vasculitis of
small vessels, midline granuloma, and antineutrophil cytoplasmic
antibody the diagnosis is definite. If the same patient has small
vessel vasculitis and antineutrophil cytoplasmic antibody, the
diagnosis of Wegener's granulomatosis will be probable. Whenever the
diagnosis is suggested at the possible level, the system recommends
that you keep the diagnosis in the differential list and look for
further clues to exclude or include it.
Patient findings are entered on data forms online for each patient. The
data form consists of multiple screens to enter clinical findings and
laboratory data. The system can capture >500 patient findings but does
not require entry for each finding.
During data entry, the physician can use the multimedia capability of
the system to obtain expert consultation on each of the patient
findings. For example, if the physician wants to know what synovial
biopsy is, he or she can choose one of the three options associated
with that finding, choosing from: tell-me-more, show-me-more, or
search-for-more (Fig 3).
The tell-me-more section consists of texts that define the clinical
terms under four categories: what (is the observation), why (is it
important), how (is it obtained), and a reference list. Figure
4 is an example of a tell-me-more screen for
enthesitis.
The show-me-more option connects the user directly to a video disc
containing >6000 slides and several video sequences. These include
clinical photographs, microscopic slides, and roentgenograms. Figure
5 is an image from the collection on
Wegener's granulomatosis.
By clicking on the search-for-more option, the physician is connected
to Medline through the Grateful-Med or Internet-Grateful-Med program.
All of these options are accessible during data entry without having to
move in and out of the data entry mode.
At the completion of the data entry, the physician requests a summary
of the patient data (Fig 2), followed by a request for the results (Fig
6). The results show the diagnosis at
different levels of certainty, and the output is available in a
fraction of a second. The output also lists findings that support the
diagnosis, findings present in the patient that do not fit the
diagnosis, and findings which, if present, will strengthen the
diagnosis (Fig 7).
If the physician wants to know why the diagnosis was or was not
triggered, he or she can request information on that particular diagnosis in relation to the findings present in that specific patient.
The system will give the findings that triggered that diagnosis. Figure
8 shows the output from the program on the
patient with Wegener's granulomatosis when the program was asked for
the reasons why Henoch-Schonlein purpura was also triggered. Similar
explanations can be obtained for the inclusion of juvenile rheumatoid
arthritis in the list of differential diagnoses.
Evaluation
We conducted a prospective study to test the ability of this
system to offer consultation on patients with pediatric rheumatic diseases. Case histories of consecutive patients seen by one of the
authors (B.H.A.) at either the Children's Hospital of Philadelphia or
at the Alfred I. duPont Hospital for Children were entered into the
AI/RHEUM case entry screens after the initial visit. Most of the
patients had a few laboratory data elements available from the
referring physicians. Some had none. Six of the patients were from
Brazil and were entered by a pediatric rheumatologist from that
country. All patients except the 6 from Brazil had a follow-up period
of 2 months to 1 year to confirm or reject the diagnosis.
| |
RESULTS |
The results were analyzed to determine whether the initial
diagnosis entertained by the physician was suggested by the AI/RHEUM expert system at least at the possible level (Table
2). There were a total of 97 diagnoses in 94 patients
(3 patients had a dual diagnosis). Overall, 78 (80%) of the 97 diagnoses were correctly made by AI/RHEUM. Eighteen of the 97 diagnoses
did not have criteria tables as part of the knowledge base included in the AI/RHEUM program. Of the 79 conditions for which criteria tables
were available in the program, the system made the correct diagnosis in
73 (92%).
In 5 patients, the program arrived at an incorrect conclusion although
the correct diagnosis was available in the knowledge base. One was a
patient with Behcet's disease who did not have oral ulcers. According
to strict criteria, the expert system was correct in rejecting the
diagnosis of Behcet's disease by suggesting nonspecific vasculitis as
a possibility. In a second patient, with monarticular arthritis and
positive antinuclear antibody, the expert system suggested the
diagnosis of pauciarticular type I or rheumatoid factor negative
polyarticular arthritis. After 1 year of follow-up, the arthritis has
cleared and there is no final diagnosis in this patient. In 2 patients
with Lyme disease and in 1 with acute rheumatic fever, the expert
system did not arrive at the correct conclusion because of the
requirements of the criteria tables. In 1 patient, the program
incorrectly refused to make the diagnosis of Lyme disease although the
diagnosis was available in the system. This was also because of the
stringent requirements of the criteria table.
Of the 18 patients with diagnoses not included in the knowledge base,
the expert system correctly recognized that criteria tables were not
available for 5 diseases and refused to come to a conclusion. In the
other 13 patients, the system offered some other diagnosis. Seven of
them were appropriate. For example, panniculitis and retroperitoneal
fibrosis are not included in the system; however, the system suggested
nonspecific vasculitis as a possibility. In 6 other patients, the
alternatives suggested by the system were not appropriate. Three of
these children had leukemia, 1 had reflex sympathetic dystrophy
syndrome, 1 had lymphangioma of the synovium, and 1 had lymphedema.
| |
DISCUSSION |
Computer-assisted decision making is currently confined to
specific areas of patient management such as routine follow-up and drug
monitoring. It has not played a major part in clinical diagnosis
despite the availability of some excellent programs. The CTX expert
system can be adapted to many areas of medicine for developing a
consultation and an educational program for nonspecialists in that
field. It is ideal for performing triage functions for primary care
physicians and emergency room physicians. The multimedia capacity of
the program allows for interactive learning and immediate feedback.
AI/RHEUM developed originally to assist in the diagnosis of rheumatic
diseases in adults has been tested extensively. In the initial study,
the system was tested in 384 carefully selected adult cases with an
accuracy of 94%.4 Another study, from Keio University
in Japan, involved 59 patients with various connective tissue diseases.
The diagnosis made by AI/RHEUM was in full or partial agreement with
those of the Japanese rheumatologists in 54 cases (92%). The criteria
table for mixed connective tissue disease was found to have a
sensitivity of 90% and specificity of 96%.5
Moens and Van der Korst6 tested the performance of AI/RHEUM
in 570 different cases from a Dutch outpatient rheumatology clinic and
compared it with predictions of diagnostic outcome made by
rheumatologists. In this study, physicians made 839 diagnoses. Of
these, 694 were definite and 145 were possible. Of these, the original
model ranked 735 (88%) and the modified model 727 (87%) among the 5 most common diagnoses. Sensitivity was 62% but the specificity was
97% to 98%.
The modified AI/RHEUM system when tested in children was shown to make
the diagnosis correctly in 92% of patients when the diagnosis was
available in the knowledge base. Obviously the system cannot make those
diagnoses for which there are no criteria tables (Table
3) in the system and should refuse to make a diagnosis for these conditions. This was what we found for 5 conditions. In 13 patients, the program offered a different diagnosis because some of the
signs and symptoms are common to many rheumatic diseases.
Two diseases account for 6 of the 18 incorrect conclusionsLyme
disease accounting for 2 of 5 among the group of diseases for which
criteria tables are available and leukemia and malignancy accounting
for 5 of the 13 among the diseases without criteria tables. We propose
to review the criteria table for Lyme disease and add a criteria table
for leukemia. We also plan to perform sensitivity and specificity
analysis on all the criteria tables.
More than the consultative role, this expert system provides a strong
educational component. All through the case-entry mode, are prompts
requesting written and visual information pertaining to the clinical
features of the disease in question. This aspect was not tested in
formal way. However, trainees in pediatrics who used the system found
it to be educational. Some of them said that AI/RHEUM helped them
consider a diagnosis they would not have considered otherwise.
The system is simple to use and capable of performing some functions of
an expert in pediatric rheumatology, but it needs to be tested at other
pediatric centers. The educational component needs formal testing. The
sensitivity of the system in diagnosing conditions listed in the
knowledge base needs to be tested.
Finally, the legal ramifications of an expert system for use by
generalists is an uncharted territory. Although an expert system such
AI/RHEUM can be made available through the Internet, the legal and
financial issues have to be resolved before such expert systems are
available more widely.
| |
FOOTNOTES |
Received for publication Jan 8, 1998; accepted May 19, 1998.
Reprint requests to (B.H.A.) duPont Hospital for Children, 1600 Rockland Rd, Wilmington, DE 19899.
| |
ACKNOWLEDGMENT |
We thank Joseph Santoroski for providing programming and
technical assistance.
| |
REFERENCES |
-
Habbema JDF
Decision support technics for clinical practice.
Ned Tijdschr Geneskd.
1988;
132:343-349
-
Cheh ML, Kingsland LC III. Criteria table representation in the medical domain. In: Kahn M, Smith J Jr, Buchanan B, Musen M, Szolovitz P, eds. Working Notes of the AAAI Spring Symposium Series 199. Artificial Intelligence in Medicine. Menlo Park, CA: American Association for Artificial Intelligence; 1992:11-24
-
Kingsland LC, Lindberg DB, Sharp GC. Anatomy of a knowledge-based consultant system: AI/RHEUM. MD Comput. 1986:3:18-26
-
Kingsland LC III. The evaluation of medical expert systems: experience with the AI/RHEUM knowledge-based consultant system in rheumatology. In: Proceedings of the Ninth Annual Symposium on Computer Applications in Medical Care. Washington, DC: IEEE Computer Society Press; 1985:292-295
-
Porter JF,
Kingsland LC,
Lindberg DA,
The AI/RHEUM knowledge-based computer consultant system in rheumatology: performance in the diagnosis of 59 connective tissue disease patients from Japan.
Arthritis Rheum
1988;
31:219-226 [Medline]
-
Moens HJ
Van der Korst JK. Development and validation of a computer program using Bayes's theorem to support diagnosis or rheumatic disorders.
Ann Rheum Dis.
1992;
51:266-271 [Abstract/Free Full Text]
Pediatrics (ISSN 0031 4005). Copyright ©1998 by the American Academy of Pediatrics
Top
Abstract
Materials & Methods
