INT J TUBERC LUNG DIS 6(9):814817

2002 IUATLD

Observer variation in detecting lymphadenopathy

on chest radiography
G. Du Toit, G. Swingler, K. Iloni
School of Child and Adolescent Health, Red Cross Childrens Hospital and University of Cape Town,
Cape Town, South Africa
SUMMARY
OBJECTIVE:

To assess inter- and intra-observer agreement in the detection of lymphadenopathy on chest radiography in children at risk for tuberculosis.
P A T I E N T S A N D M E T H O D S : Retrospective examination
of the antero-posterior and lateral chest radiographs of
children aged 1 month to 11 years discharged from the
short-stay ward of the Red Cross Childrens Hospital,
Cape Town, with a diagnosis of tuberculosis or pneumonia. Four paediatric pulmonologists viewed the
radiographs independently. The main outcome measures were inter- and intra-observer agreement on the
presence or absence of lymphadenopathy, reported as
present, absent or equivocal, and expressed as weighted
kappa statistics.
R E S U L T S : Weighted kappa for the six pairs of observers

ranged from 0.14 (95%CI 0.020.30) to 0.52 (95%CI

0.350.69). After a 3-month interval, intra-observer
agreement ranged from 0.44 (95%CI 0.250.62) to 0.71
(95%CI 0.560.87). The average weighted kappa for
inter-observer agreement was 0.33, and the average
intra-observer kappa was 0.55.
C O N C L U S I O N S : There was fair inter- and moderate
intra-observer agreement among paediatric pulmonologists in detecting lymphadenopathy on chest radiography in children. Caution is necessary when basing clinical decisions on the presence of lymphadenopathy on
chest radiography.
K E Y W O R D S : observer variation; radiography; lymphatic diseases; lymphadenopathy; tuberculosis

TUBERCULOSIS is difficult to diagnose in children

because of the non-specific symptoms and infrequent
isolation of organisms. Strong reliance is thus often
placed on chest radiography,1 with mediastinal lymphadenopathy regarded as the radiological hallmark of
primary tuberculosis.2,3 The World Health Organizations proposed diagnostic criteria for pulmonary TB
include pulmonary lymph nodes as one of five criteria
for probable tuberculosis.4 Other diagnostic scoring
systems for childhood tuberculosis also include radiographic lymphadenopathy.57 The validity and reliability of the detection of radiographic lymphadenopathy
is thus of central importance to the diagnosis of childhood tuberculosis. A search of the Medline, HealthStar, LILACS and AIDSLINE databases and the African Health Anthology (October 2000) failed to
identify any studies of either validity or observer variation in the detection of lymphadenopathy on chest
radiography in children aged less than 18 years.
This study was performed to measure inter- and
intra-observer agreement in the detection of lymphadenopathy on chest radiography in children at risk
for tuberculosis.

METHODS
The study was performed at the Red Cross Childrens
Hospital (RCCH) in the Western Cape Province of
South Africa. This province had a tuberculosis incidence of 468 cases per 100 000 population in 1998,
which is one of the highest in the world.8
Radiographs of children with pulmonary tuberculosis have many features in common with those of children with non-tuberculous pneumonia, with few recognised distinguishing features of tuberculosis other than
lymphadenopathy. Therefore radiographs of children
with a diagnosis of either pulmonary tuberculosis or
pneumonia were studied. The aim was not to recruit
children with known tuberculosis and controls without
tuberculosis, but rather to assemble a representative
spectrum of radiographs from a clinical setting in which
tuberculosis was part of the differential diagnosis.
Children discharged or transferred from the shortstay ward at RCCH from 1 January 1996 to 31 December 1998 with a diagnosis of pulmonary tuberculosis or
pneumonia were identified retrospectively from the
hospitals electronic database. In order to assemble a

Correspondence to: Dr George Du Toit, School of Child & Adolescent Health, Institute of Child Health Building, Red Cross
Hospital, Rondebosch 7700, South Africa. Tel: (27) 21-6585318. Fax: (27) 21-6891287. e-mail: gdutoit@ich.uct.ac.za
Article submitted 23 October 2001. Final version accepted 29 May 2002.

Observer variation in detecting lymphadenopathy on chest X-ray

sample with a statistically efficient prevalence of lymphadenopathy, we selected 60 patients with a diagnosis of
tuberculosis and 40 with a diagnosis of pneumonia.
Selection was random, using a computer generated list
of random numbers and sampling frames of all
patients discharged during the study period with a
diagnosis of tuberculosis or pneumonia. Radiographs
were included if both a lateral and antero-posterior
radiograph were available from that admission.
Four paediatric pulmonologists attached to RCCH
interpreted the radiographs. Each observer independently viewed all of the 100 films in random order.
Radiographs were labelled from one to 100, and the
viewing sequence was assigned using a computer
generated list of random numbers. No clinical information was provided other than that the radiographs
were from a random sample of patients who had
been admitted to the overnight ward with chest disease. The observers categorised lymphadenopathy as
present, absent or equivocal. The same observers
viewed the films again in different random order after
a 3-month interval. Viewing occurred in batches of 50
to avoid viewer fatigue. No time constraints were
applied, and no criteria for the presence of lymphadenopathy were prescribed.
Agreement between individual observers and within
each observer were expressed as weighted kappa statistics.9 Kappa is a measure of the degree of inter-rater
agreement, over and above that expected by chance. It
has a maximum value of 1, indicating complete agreement. A kappa of zero means that agreement is no better than if it were due solely to chance; a positive value
means that the observed agreement is greater than
would be expected by chance alone. The minimum
kappa value is 1, indicating complete disagreement.
Weighted kappa takes into account the degree of
disagreement. Because the categories yes, equivocal
and no are ordered, all disagreements are not equally
severe; for example, a yes-no disagreement is more
severe than an equivocal-yes disagreement, and has
different clinical implications.

RESULTS
Of the 100 children, 56 were male and 44 female.
Their median age was 22 months with a range of 1
month to 11 years. Forty-nine were discharged home
from the overnight ward and the other 51 were
referred for further hospital care.
At the first viewing lymphadenopathy was reported
as present in 18% of cases, absent in 62% and equivocal in 20%.
Inter- and intra-observer agreement is shown in the
Table. Weighted kappa for the six pairs of observers
ranged from 0.14 (95%CI 0.020.30) to 0.52
(95%CI 0.350.69) and for intra-observer agreement
from 0.44 (95%CI 0.250.62) to 0.71 (95%CI 0.56
0.87). The average weighted kappa for inter-observer

Table

815

Inter- and intra-observer agreement. Weighted kappas

Inter-observer agreement
Observer
pairs
Kappa
AB
AC
AD
BC
BD
CD

0.19
0.52
0.37
0.14
0.39
0.36

Intra-observer agreement
95%CI
0.040.35
0.350.69
0.220.52
0.020.30
0.230.54
0.20.51

Observer Kappa
A
B
C
D

0.58
0.71
0.44
0.47

95%CI
0.410.74
0.560.87
0.250.62
0.320.62

agreement was 0.33, and the average intra-observer

kappa was 0.55.
Kappa scores can also be computed from the perspective of the response.10 These are measures for the
particular response categories, and are not weighted.
Kappa was 0.40 (95%CI 0.320.48) for the presence
of mediastinal lymphadenopathy, 0.28 (95%CI 0.20
0.36) for its absence and 0.03 (95%CI 0.050.11)
for equivocal readings.

DISCUSSION
Observer agreement has been categorised as slight if
kappa is 0.20, fair if kappa is 0.210.40, moderate from 0.41 to 0.60, substantial from 0.61 to 0.80
and almost perfect from 0.81 to 1.0.11 Inter-observer
agreement in this study was thus fair (on average
0.33). The use of weighted kappa resulted in higher
measures of agreement than the use of unweighted
kappa. This level of inter-observer agreement is nevertheless lower than that reported for other chest radiolographic features of respiratory infection in children:
for example, unweighted kappas of 0.43 to 0.55 for
bronchial wall thickening, 0.46 to 0.79 for consolidation or pneumonia and 0.78 to 0.83 for hyperinflation.12 Agreement in our study may also be higher
than in the circumstances in which tuberculous lymphadenopathy is usually assessed, because the observers
were all very experienced at assessing chest radiographs, had similar backgrounds and training, and
worked in the same institution in a province with one
of the highest incidences of childhood TB in the
world. Our sample included a spectrum of radiographs from a setting in which tuberculosis was part
of the differential diagnosis. This is likely to have provided a more meaningful estimate of agreement than if
we had used only films with lymphadenopathy known
to be present or absent, because the latter sample
would have excluded many equivocal films.
There was greater agreement on the presence of
lymphadenopathy (0.40) than its absence (0.28). As
expected, intra-observer agreement was higher than
inter-observer agreement, but was mostly moderate,
ranging from 0.44 to 0.71.
Our findings suggest that caution is necessary when

816

The International Journal of Tuberculosis and Lung Disease

basing clinical decisions on the presence or absence of

lymphadenopathy on chest radiography. Although
many factors in addition to lymphadenopathy are
involved in the diagnosis of childhood TB, these findings have direct relevance to the diagnosis of TB in
children. The findings may also be applicable to the
radiological detection of lymphadenopathy in children with other conditions.
This investigation addressed the repeatability of
the observations and not the validity, for which comparison with a credible reference standard would be
necessary. The low inter-observer agreement, however, suggests that the validity of detection by some or
all of the observers was low. No criteria for the diagnosis of lymphadenopathy were prescribed, and the
wide variation in intra-observer agreement suggests
that different observers used different (formal or
informal) criteria in their assessment of lymphadenopathy. When questioned after the completion of
the study, the observers varied in the criteria they had
used in detecting lymphadenopathy. This suggests
that the use of explicit criteria for lymphadenopathy
could increase the repeatability of detection.
Computerised tomography (CT) with contrast
injection is regarded as more accurate than chest radiography in detecting lymphadenopathy.3,13 In settings
where CT is readily available, attempts should nevertheless be made to define and improve the accuracy of
chest radiography, given the higher radiation dose,
the higher cost and the need for contrast injection
associated with CT. In settings where CT is not available, and where the vast majority of children with
tuberculosis live, the need for further research into
the accuracy of chest radiography is clear.
We conclude that lymphadenopathy in children is
detected on chest radiography with fair inter- and
moderate intra-observer agreement. Further research is
needed using a credible reference standard to test the

validity of detection. Such research could also enable

the development and evaluation of criteria to maximise the validity and repeatability of such detection.
References
1 Coulter J B S. Tuberculosis. In: Campbell A G M, McIntosh N,
eds. Forfar and Arneils Textbook of Paediatrics. 4th ed. Edinburgh, UK: Churchill Livingstone, 1992: pp 13901403.
2 Lamont A C, Cremin B J, Pelteret R M. Radiological patterns of
pulmonary tuberculosis in the paediatric age group. In: Paediatric
Radiology. 1st ed. London, UK: Springer, 1986: pp 27.
3 Cremin B J, Jamieson D H. Childhood tuberculosis: modern
imaging and clinical concepts. 1st ed. London, UK: Springer,
1995: pp 1931.
4 World Health Organization. Provisional guidelines for the diagnosis and classification of the EPI target diseases for primary
health care, surveillance and special studies. EPI/GEN/83/4.
Geneva: WHO, 1983.
5 Migliori G B, Borghesi A, Rossanigo P, et al. Proposal of an improved score method for the diagnosis of pulmonary tuberculosis in childhood in developing countries. Tubercle Lung
Dis 1992; 73: 145149.
6 Osborn C M. The challenge of diagnosing childhood tuberculosis in a developing country. Arch Dis Child 1995; 72: 369374.
7 Donald P R. Diagnostic considerations in management and epidemiology. In: Coovadia H M, Benatar S R, eds. A century of
tuberculosis: South African perspectives. 1st ed. Cape Town,
South Africa: Oxford University Press, 1991.
8 Department of Health, Republic of South Africa. Epidemiological
comments Vol.1 No.2. Notifiable Medical Conditions. Western
Cape Province, South Africa: Department of Health, June 1999.
9 Fleiss J L. Statistical methods for rates and proportions, 2nd
ed. New York: Wiley and Sons, 1981: pp 212236.
10 Altman D G. Practical statistics for medical research. 2nd ed.
London, UK: Chapman & Hall, 1991: pp 403409.
11 Sackett D L, Haynes R B, Guyatt G H, Tugwell P. Clinical epidemiology. A basic science for clinical medicine. 2nd ed. Boston, MA; Little Brown and Company, 1991: p 30.
12 Swingler G H. Observer variation in chest radiography of acute
lower respiratory infections in children: a systematic review.
BMC Medical Imaging, 2001; 1:1 http://www.biomedcentral.
com/14712342/1/1/
13 Kuhn J P, Slovis T L, Silverman F N, Kuns L R. Mediastinum.
In: Silverman F N, Kuhn J P, Caffey J, eds. Paediatric X-ray diagnosis. 9th ed. St Louis, MO: Mosby, 1993.

RSUM
O B J E C T I F : Dterminer les concordances entre observateurs et chez le mme observateur dans la dtection des
adnopathies sur les clichs thoraciques denfants risque
de tuberculose.
P A T I E N T S E T M T H O D E S : Dans une tude rtrospective, les clichs thoraciques comportant un film antropostrieur et de profil des enfants gs un mois 11 ans
et sortant dune salle bref sjour de lHpital Red
Cross Children, Cape Town et avec un diagnostic de
tuberculose ou de pneumonie, ont t examins indpendamment par quatre pneumologues pdiatriques. Les
mesures principales de rsultats taient la concordance
entre divers observateurs et chez le mme observateur
sur la prsence ou labsence dadnopathies, dcrites
comme prsentes, absentes ou douteuses et exprimes
sous forme de statistiques kappa pondres.

Le kappa pondr pour les six paires

dobservateurs se situe entre 0,14 (IC95% 0,020,30) et
0,52 (IC95% 0,350,69). Aprs un intervalle de 3 mois,
la concordance par le mme observateur se situe entre
0,44 (IC95% 0,250,62) et 0,71 (IC95% 0,560,87). Le
kappa moyen pondr pour une concordance entre
observateurs est de 0,33. Le kappa pour le mme observateur est de 0,55.
C O N C L U S I O N S : Parmi les pneumologues pdiatriques,
on observe une concordance satisfaisante entre observateurs et modre pour le mme observateur, pour ce
qui concerne la dtection dadnopathies dans les clichs
thoraciques denfants. Il est ds lors ncessaire dtre
prudent lorsque lon base des dcisions cliniques sur la
prsence dadnopathies au clich thoracique.
RSULTATS :

Observer variation in detecting lymphadenopathy on chest X-ray

817

RESUMEN

Evaluar la concordancia inter e intra-observador en la deteccin de las adenopatas en las radiografas

de trax en nios con riesgo de tuberculosis.
M A R C O D E R E F E R E N C I A Y P A C I E N T E S : Estudio retrospectivo. La radiografa de trax ntero-posterior y lateral de nios de un mes a 11 aos de edad, dados de alta
de una sala de corta estada del Hospital de Nios de la
Cruz Roja, Ciudad del Cabo, con diagnstico de tuberculosis o neumona, fueron examinados independientemente por cuatro neumlogos peditricos. Las principales
mediciones de los resultados feuron la concordancia intere intra-observador sobre la presencia o ausencia de linfoadenopatas, descritas como presentes, ausentes o dudosas
y expresadas en estadsticas kappa ponderadas.
R E S U L T A D O S : El kappa ponderado para los seis pares
OBJETIVO :

de observadores se sita entre 0,14 (IC95% 0,020,30)

y 0,52 (IC95% 0,350,69). Despus de un intervalo de 3
meses, la concordancia intra-observador se situaba entre
0,44 (IC95% 0,250,62) y 0,71 (IC95% 0,560,87). El
kappa ponderado promedio de la concordancia interobservador fue de 0,33. El kappa promedio intra-observador fue de 0,55.
C O N C L U S I N : Los neumlogos peditricos se observ
una concordancia inter-observador satisfactoria e intraobservador moderada , con respecto a la deteccin de
linfoadenopatas en las radiografas de trax de nios.
Es necesario ser prudente cuando las decisiones clnicas
se basan en la presencia de linfoadenopatas en la
radiografa de trax.