Published online May 1, 2008
PEDIATRICS Vol. 121 No. 5 May 2008, pp. e1229-e1239 (doi:10.1542/peds.2007-0871)
This Article
Right arrow Abstract Freely available
Right arrow Full Text (PDF)
Right arrow P3Rs: Submit a response
Right arrow Alert me when this article is cited
Right arrow Alert me when P3Rs are posted
Right arrow Alert me if a correction is posted
Right arrow Citation Map
Services
Right arrow E-mail this article to a friend
Right arrow Similar articles in this journal
Right arrow Similar articles in PubMed
Right arrow Alert me to new issues of the journal
Right arrow Add to My File Cabinet
Right arrow Download to citation manager
Right arrow reprints & permissions
Citing Articles
Right arrow Citing Articles via CrossRef
Google Scholar
Right arrow Articles by Kapogiannis, B. G.
Right arrow Articles by Bulterys, M.
PubMed
Right arrow PubMed Citation
Right arrow Articles by Kapogiannis, B. G.
Right arrow Articles by Bulterys, M.
Related Collections
Right arrow Infectious Disease & Immunity

ARTICLE

Trends in Bacteremia in the Pre- and Post-Highly Active Antiretroviral Therapy Era Among HIV-Infected Children in the US Perinatal AIDS Collaborative Transmission Study (1986–2004)

Bill G. Kapogiannis, MDa,b, Minn M. Soe, MD, MPHc, Steven R. Nesheim, MDa, Kevin M. Sullivan, PhD, MPHc, Elaine Abrams, MDd, John Farley, MD, MPHe, Paul Palumbo, MDf, Linda J. Koenig, PhDg and Marc Bulterys, MD, PhDg

Division of Infectious Diseases, Departments of a Pediatrics
b Medicine, Emory University School of Medicine
c Department of Epidemiology, Rollins School of Public Health, Emory University, Atlanta, Georgia
d Department of Pediatrics, Harlem Hospital Center, New York, New York
e Department of Pediatrics, University of Maryland, Baltimore, Maryland
f Department of Pediatrics, University of Medicine and Dentistry, Newark, New Jersey
g Division of HIV/AIDS Prevention, National Center for HIV, STD, and TB Prevention, Centers for Disease Control and Prevention, Atlanta, Georgia


   ABSTRACT
TOP
 ABSTRACT
METHODS
 RESULTS
 DISCUSSION
 CONCLUSIONS
 REFERENCES
 
OBJECTIVE. HIV-infected children are at high risk for bacteremia. Highly active antiretroviral therapy has reduced rates of opportunistic infections; less is known about its effect on pediatric bacteremia rates. Thus, we sought to determine its impact on bacteremia incidence in HIV-infected children.

METHODS. Children born during 1986–1998 were followed until 2004 in the Perinatal AIDS Collaborative Transmission Study. We determined the pre–and post–highly active antiretroviral therapy (before and after January 1, 1997) incidence of bacteremia among HIV-infected children and characterized the CD4% temporal declines and mortality among patients with and those without incident bacteremias.

RESULTS. Among 364 children, 68 had 118 documented bacteremias, 97 before and 21 after January 1, 1997. Streptococcus pneumoniae constituted 56 (58%) pre–and 13 (62%) post–highly active antiretroviral therapy cases. The incidence rate ratio of bacteremias comparing post–versus pre–highly active antiretroviral therapy was 0.3 overall and 0.2, 0.2, and 0.4 among children aged 0 to 24, 25 to 48, and 49 to 72 months, respectively. Kaplan-Meier analysis for time to first bacteremia in children born during the pre–highly active antiretroviral therapy compared with the post–highly active antiretroviral therapy era revealed that 69% and 94%, respectively, remained bacteremia free at a median follow-up of 6 years. The Cox proportional hazards model also showed a significant reduction of bacteremias in the post–highly active antiretroviral therapy era, even after controlling for gender and race. Among children <6 years of age, those who experienced bacteremia had faster temporal CD4% decline than those who never had bacteremia. Survival analysis revealed that HIV-infected children with bacteremia experienced higher overall mortality when controlling for gender, race, and clinic site.

CONCLUSIONS. A significant decrease in bacteremia incidence and a prolongation in the time to first bacteremia incident were seen in the post–highly active antiretroviral therapy era. Children with a steeper decline of CD4 T cells were more likely to develop bacteremia. Children who experienced bacteremia had an associated higher mortality than their bacteremia-free counterparts.

Key Words: pediatric HIV/AIDS • bacteremia incidence • HAART

Abbreviations: OI—opportunistic infection • IVIG—intravenous immunoglobulin • PACTS—Perinatal AIDS Collaborative Transmission Study • CDC—Centers for Disease Control and Prevention • HOPE—HIV Follow-up of Perinatally Exposed Children • TMP-SMX—trimethoprim-sulfamethoxazole • CI—confidence interval

Bacteremia is an important cause of morbidity and mortality among HIV-infected individuals. In contrast to HIV-infected adults in whom other opportunistic infections (OIs) predominate, HIV-infected children are at high risk for serious bacterial infections with encapsulated bacteria,1–4 particularly during the early years of life. Enhanced susceptibility to these organisms arises in the first 2 years as the immune response to capsular polysaccharides matures5 and is further manifested by poor immune responses to various polysaccharide vaccine antigens.6–9 In HIV-infected children, this risk is magnified by the direct effects of HIV-related T- and B-cell dysfunction,10–15 leading to the production of polyclonal, nonspecific immunoglobulin16–18 and, uncommonly, to hypogammaglobulinemia.19

The incidence of bacteremia in HIV-infected children in the United States during the first few years of life is estimated to range from 3.3 to 12.2 per 100 person-years20–24 and is ~100-fold that of children of the same age without HIV infection.25–27 In children not infected with HIV, the peak incidence of bacteremia occurs around 24 months of age and then declines, whereas such a decline is not seen among HIV-infected children. The most common blood isolate among bacteremic HIV-infected children is Streptococcus pneumoniae.2–4,21,28 Two randomized, placebo-controlled trials demonstrated that intravenous immunoglobulin (IVIG) can reduce the rate of serious bacterial infections among these children.29–31

In wealthier industrialized countries, the introduction of highly active antiretroviral therapy (HAART) during 1996 revolutionized the care of individuals with HIV infection.32–35 Since then, the incidence of AIDS-related OIs in adults has declined dramatically, and patients living with HIV are now more than ever being managed as outpatients.32–35 Before the HAART era, HIV-infected adults experienced OIs at rates of 15.1 to 50.0 per 100 person-years32,33,35 and bacteremia events at rates of 2.4 to 11.8 per 100 person-years.36–38 After HAART implementation in adults, the rates of AIDS-related OIs declined to a range of 2.2 to 13.3 per 100 person-years,32,33,35 and rates of bacteremias declined to a range of 0.8 to 6.3 per 100 person-years.36,37 Similar trends were recently published on the impact of HAART on the incidence of such AIDS-related morbidities in children, although these cohorts were not followed since birth.39,40 In studies of bacteremia, birth cohorts have clear advantages over cohorts with later ages of enrollment. In HIV-uninfected children, rates of bacteremia with classic encapsulated bacteria are highest in the first 2 years of life.25–27 Likewise, HIV-infected birth cohorts can capture very high bacteremia rates,20–24 as well as other morbidity and mortality causes during this same period.41–44 We present the analysis of trends in bacteremia over an extended period among a large prospective birth cohort enrolled in the Perinatal AIDS Collaborative Transmission Study (PACTS).


   METHODS
TOP
 ABSTRACT
 METHODS
RESULTS
 DISCUSSION
 CONCLUSIONS
 REFERENCES
 
Subjects and Study Design
PACTS was a Centers for Disease Control and Prevention (CDC)-sponsored multicenter, prospective cohort study of HIV-infected pregnant women and their newborns conducted in 4 US cities to monitor the incidence of mother-to-child HIV transmission and to describe the natural course of pediatric HIV disease progression. This study was conducted with approval of the institutional review boards at the CDC and at the respective centers. Signed parental informed consent was obtained for each participant. Race was reported to identify and analyze any pertinent covariates. The classification scheme was determined by study investigators at the conception of PACTS and was used cooperatively by site study coordinators and participants for classification. Sites began enrollment as follows: New York City, 1986; Baltimore, 1989; Atlanta, 1990; and Newark, 1990. Mother-infant pairs were followed continuously until September 30, 1999, 1 year after enrollment terminated. This cohort has been described previously in more detail.45 Clinical data on PACTS-HIV Follow-up of Perinatally Exposed Children (HOPE) enrollees were collected for the period between the 2 studies (October 1, 1999 to March 1, 2000) and subsequently every 6 months.42,46 Data collection ceased in April 2004. Hereafter, both PACTS and PACTS-HOPE enrollees are collectively referred to as the study cohort. An analysis was undertaken to determine the incidence and prevalence of HIV-related bacteremias over time, their relationship to disease progression, and the impact of HAART on these events.

Clinical and Laboratory Data Collection
Clinical charts were reviewed at each study visit, and interim bloodstream infections were identified by organism name. Only pathogenic organisms causing bacteremia were abstracted for analysis, the criteria for selection of which are described below. Microbiologic isolates were compared between the pre- and the post-HAART eras for patients who developed bacteremia.

Medications and Rationale for Definition of "HAART Era"
Data were collected at each study visit regarding any antiretroviral medications used since the previous visit. "HAART" was defined as the receipt of combination antiretroviral therapy that consisted of ≥3 antiretroviral medications that included 2 nucleoside reverse-transcriptase inhibitors combined with either a protease inhibitor or a nonnucleoside reverse-transcriptase inhibitor; a small minority of children received 3 nucleoside reverse-transcriptase inhibitors. Events that occurred during the pre- and post-HAART eras were defined as those that occurred before and after January 1, 1997, respectively. The reasons for this choice of date as the cutoff are twofold. First, 1996 marked the initial availability of protease inhibitors47; however, uptake by pediatric clinics was infrequent and did not begin increasing until 1997, the year when many US centers were initiating HAART in a substantial proportion of HIV-infected children.48 Second, because the exact date of HAART initiation in US HIV-infected children varies by site and caregiver, thereby necessitating an arbitrary estimate, we determined the most relevant date for our entire HIV-infected cohort using Kaplan-Meier analysis for time to initiation of HAART starting from several potential dates. A survival curve was generated for each potential date, and all of the curves were superimposed for comparison. We selected from these survival analyses the curve of which the corresponding date allowed the minimum misclassification of patients. The date was further confirmed by analyzing the frequency distribution of HAART initiation by calendar year among children born before and those born after January 1, 1997, showing that 1997 was the year in which most children (42%) started HAART. Of the remainder, 25% started HAART in 1996 or before and 33% in 1998 or thereafter. The alternative analytic approach using individual HAART initiation dates was not selected because of its potential for introducing bias toward the inclusion of data from sicker children who may have started HAART earlier.48

Data Analysis and Statistical Methods
Our primary end point was any bacteremia event that occurred during the study period (see "Definitions"). These events were used to calculate the incidence rate of bacteremia during the pre- and post-HAART eras (see "Incidence Rate Calculation"). The time to occurrence of a first bacteremia event in children born in the pre-HAART era and those born in the post-HAART era was also evaluated (see "Time to First Bacteremia Event").

Definitions
A bacteremia event was defined as a bloodstream infection documented by ≥1 blood culture growing pathogenic bacteria. Bacteria defined as nonpathogens were Bacillus species in children without indwelling catheters, as well as all nonaureus staphylococci and viridans streptococci; nonpathogens were excluded from analysis. Organisms reported as "others," "unknown," and missing data were also excluded. All of the remaining isolates were considered pathogens. A bacteremia could occur more than once in a child and be tabulated as a separate event, provided that ≥14 days elapsed between the 2 episodes. Because the presence of indwelling venous catheters may have introduced confounding, analyses for the incidence rate of bacteremia and for time to first bacteremia event were first performed for those children whose bacteremia occurred only in the absence of an indwelling catheter. The results reported herein include those of children with indwelling catheters, because their exclusion did not alter the findings.

The following variables were analyzed for children with bacteremia events in the pre- and post-HAART era: patient demographic characteristics, antiretroviral treatment, immunologic status (absolute CD4 cell count and percentage), HIV RNA quantification, presence or absence of indwelling venous catheters, receipt of the 23-valent pneumococcal polysaccharide vaccine, and use of trimethoprim-sulfamethoxazole (TMP-SMX) prophylaxis. Except where noted otherwise, a child could contribute data more than once and to both time periods.

Incidence Rate Calculation
The incidence rate of bacteremia was first calculated for the pre- and post-HAART eras using a person-time approach and was then further stratified by age, gender, and race to yield stratum-specific incidence rates with rate ratios and Mantel-Haenszel adjusted incidence rate ratios with 95% confidence intervals (CIs). If there was a 0 cell because of the lack of a bacteremia event, the CI with exact Mid-P method was used to obtain an upper confidence limit (Computer Programs for Epidemiologic Analysis, PEPI version 2).49 The follow-up time for the pre- and post-HAART eras was the total number of months that all of the children lived before and subsequent to January 1, 1997, respectively.

Time to First Bacteremia Event
Survival analysis was used to compare the time until the development of the first bacteremia event among HIV-infected children born before and after January 1, 1997, up to 6 years of age. Children born in the pre-HAART era who were followed beyond January 1, 1997, had their follow-up times censored thereafter. Kaplan-Meier and adjusted survival plots controlling for gender and race were constructed for comparison between the above 2 groups. To determine the effect of the post-HAART era on the time to development of a first bacteremia, Cox proportional hazard analysis was used to yield the final model after testing interaction terms and assessing confounding by other covariates. While constructing the models, multicollinearity and regression diagnostics were applied to evaluate model fit.

Mediating Variables
Possible causes of the change in bacteremia incidence between the 2 eras include improved immunologic status, virologic status, uptake of TMP-SMX prophylaxis, and uptake of the 23-valent pneumococcal polysaccharide vaccine. The pneumococcal conjugate vaccine became commercially available in February 2000; however, its receipt was not ascertained in this study. IVIG receipt was infrequent and not analyzed.

Immunologic status was assessed by CD4% and virologic status by HIV RNA quantification (HIV viral load). To determine the influence of these parameters, their distribution by age was estimated by linear regression analysis with 95% CIs as described previously by Denny et al.50 Data chosen for this analysis were limited to children aged 0 through 72 months; this interval was selected because it included nearly all of the bacteremia events (113 of 118 [96%]). Data from the same child at different ages were identified, and a single result was randomly selected for analysis; this approach was used to avoid overrepresentation of individual data, within which there may be high correlation. The comparison of the slope of the 2 regression lines between patients with and without bacteremia events was analyzed by means of assessing an interaction term in the linear regression model. Finally, the cumulative proportion of total follow-up time spent by the cohort on TMP-SMX prophylaxis was compared between pre- and post-HAART eras, as was the proportion of children who had received the 23-valent pneumococcal vaccine by 3 years of age.

Mortality Analysis
Mortality of HIV-infected children was analyzed by survival analysis to compare the time until death between children with ≥1 bacteremia event with those who never developed bacteremia. Children born in the pre-HAART era who were followed beyond January 1, 1997, had their follow-up times censored thereafter. Cox proportional hazard analysis was also performed, controlling for gender, race, and clinical sites of care as covariates.

Statistics
Demographic data were analyzed by SAS 9.0 (SAS Institute, Inc, Cary, NC) and Epi Info 3.3.2 (CDC, Atlanta, GA). Estimates of incidence density rate and rate ratio were computed using OpenEpi (www.OpenEpi.com, Atlanta, GA).51 Survival analyses, including Cox proportional hazard models, were conducted by SAS 9.0. All of the significance tests were 2-tailed, and a P value ≤ .05 was considered statistically significant. Confounding by a variable was defined as a ≥5% difference between a crude and an adjusted estimate.


   RESULTS
TOP
 ABSTRACT
 METHODS
 RESULTS
DISCUSSION
 CONCLUSIONS
 REFERENCES
 
Study Cohort
There were 364 HIV-infected children enrolled in this study cohort (Table 1). Individual outcomes may have been influenced by many dynamic exposure variables over the 18-year course of the study, including age, birth date, and length of exposure to defined time periods (pre-HAART, pre-TMP-SMX, and pre-IVIG) during high-risk ages (Fig 1). The majority of children were black or Hispanic (336 of 364 [92%]). Three fourths (274 of 364) of enrollees were born before 1995, before widespread institution of zidovudine prophylaxis for prevention of mother-to-child transmission of HIV52–54 (Fig 1). An additional 52 (14%) children were born in 1995–1996, yielding a total of 326 children (90%) born during the pre-HAART era (Fig 1). The median follow-up duration for the entire cohort was 5.9 years (interquartile range: 2.2–9.5 years).


View this table:
[in this window]
[in a new window]

  		TABLE 1 Characteristics of HIV-Infected Study Population (PACTS-HOPE Study)

 

Figure 1
View larger version (76K):
[in this window]
[in a new window]

  		FIGURE 1 Relationship of births and bacteremia event distribution to potential effect modifying exposures experienced by HIV-infected children enrolled in PACTS/PACTS-HOPE who developed bacteremia during the pre-HAART (before January 1, 1997) and post-HAART eras. Top, Distribution of births (first row) and bacteremia events (second row) among the study cohort by calendar year. Bottom, HIV disease management milestones and their availability by calendar year.

 
Characteristics of Children With Bacteremia
There were 118 documented bacteremia events in 68 children, of whom 29 (43%) had >1 event (Table 1). Approximately 60% (71 of 118) of bacteremias occurred before 1995, before widespread institution of zidovudine prophylaxis,52–54 when vertical HIV infection would have been more common (Fig 1). This peak in bacteremias trailed the births by ~2 years. An additional 26 (22%) occurred between 1995 and 1997, yielding a total of 82% (97 of 118) of the events during the pre-HAART era (Fig 1). No bacteremias occurred during 2002–2004.

Pre- and Post-HAART
There were 97 and 21 bacteremia events in the pre- and post-HAART eras, respectively. Bacteremia events were not uniformly distributed among gender, calendar year of birth, or age at diagnosis but had similar distribution among race in the pre- and post-HAART eras (Table 2). Most of the events occurred in children born between 1989 and 1994 (102 of 118 [86%]) and in those who were <6 years of age (113 of 118 [96%]). Subsequent analyses of bacteremia incidence are, therefore, focused to this age group.


View this table:
[in this window]
[in a new window]

  		TABLE 2 Characteristics of Patients Who Developed Pathogenic Bacteremia During the Pre-HAART (Before January 1, 1997) and Post-HAART (PACTS-HOPE Study) Eras

 
Pathogenic Bacteria
The distribution of pathogenic organisms was similar between the pre- and post-HAART eras (Table 3). The majority of bacteremia events, 56 (58%) of 97 in the pre-HAART era and 13 (62%) of 21 in the post-HAART era, respectively, were caused by Streptococcus pneumoniae.


View this table:
[in this window]
[in a new window]

  		TABLE 3 Pathogenic Isolates Recovered From Patients Who Developed Bacteremia During the Pre-HAART (Before January 1, 1997) and Post-HAART (PACTS-HOPE Study) Eras

 
Overall Incidence of Bacteremia Among HIV-Infected Children
There were 97 and 16 bacteremia events among children <6 years of age from 364 HIV-infected children in the pre- and post-HAART eras, respectively (Table 4). Recurring events in the same child were included in this analysis. The overall incidence of bacteremia was 9.8 and 3.0 events per 100 person-years in the pre- and post-HAART era, respectively. This yielded an overall incidence rate ratio of 0.3 (95% CI: 0.2 to 0.5) between the post- and pre-HAART eras corresponding with a rate reduction of 70%.


View this table:
[in this window]
[in a new window]

  		TABLE 4 Incidence of Bacteremia Events Among HIV-Positive Children in the First 6 Years of Life During the Pre- and Post-HAART Eras (n = 364)

 
Stratified Incidence of Bacteremia Among HIV-Infected Children
When stratified by the age at bacteremia diagnosis, the incidences of bacteremia events were similar and declined similarly in all 3 of the age groups between the pre- and post-HAART eras (Table 4). Tests for trend were not significant for incidence rates within increasing age strata in either the pre- or post-HAART era (P > .05, {chi}2). The incidence rate ratio of bacteremia events (post- versus pre-HAART) for children aged 0 to 24, 25 to 48, and 49 to 72 months was 0.2 (95% CI: 0.05 to 0.8), 0.2 (95% CI: 0.1 to 0.6), and 0.4 (95% CI: 0.2 to 0.9), respectively.

On stratification by gender, the reduction in bacteremia incidence was similar between boys and girls (Table 4). The incidence rate ratio of bacteremia events for boys and girls was 0.3 (95% CI: 0.1 to 0.7) and 0.3 (95% CI: 0.2 to 0.6), respectively.

Stratification by race was accomplished by analyzing children as either black non-Hispanic or "other groups combined," because there were predominantly black non-Hispanic children (~70%) in this cohort. The reduction in bacteremia incidence was less pronounced for black non-Hispanic children compared with those children classified as "other groups combined" (Table 4). The incidence rate ratio of bacteremia events for black non-Hispanic children was 0.3 (95% CI: 0.20 to 0.60) and for "other groups combined" was 0.1 (95% CI: 0.02 to 0.90). The overall and stratified incidence rate analyses were repeated after excluding those children with indwelling intravascular catheters and yielded similar results (data not shown).

HAART and Time to Development of First Bacteremia
Kaplan-Meier survival curves are presented for time to first bacteremia event for the pre- and post-HAART eras. Among children born in the post-HAART era there were fewer (2 vs 59) bacteremia events, and there was a significant prolongation in the time to development of their first bacteremia event when compared with their pre-HAART counterparts (P = .02, log rank test; Fig 2). A Cox proportional hazards model of time to first bacteremia event was used to control for gender and race yielding consistent results with the Kaplan-Meier analysis by demonstrating a decrease in bacteremia events during the post-HAART era. The crude and adjusted analyses showed a hazard ratio of 0.21 (95% CI: 0.05 to 0.85; P = .03) and 0.21 (95% CI: 0.05 to 0.87; P = .03), respectively.


Figure 2
View larger version (11K):
[in this window]
[in a new window]

  		FIGURE 2 Kaplan-Meier analysis depicts time to development of the first bacteremia in children ≤6 years of age, comparing children born before January 1, 1997 (pre-HAART) with those born afterward (post-HAART).

 
Mediating Variables
Differences were analyzed in the temporal trends of CD4% and HIV RNA viral load among children who developed bacteremia events compared with those who never had bacteremia. CD4% was used instead of the absolute CD4 cell count for analysis, because the former is generally accepted as the more stable parameter, particularly during the early years of life, when large fluctuations in the normal absolute CD4 cell count ranges can occur.50 Among <6-year-olds, children who never developed a bacteremia event experienced a decline in their CD4% (r = –0.21; 95% CI: –0.32 to –0.09; P < .001, Pearson's test), as did those who had ≥1 bacteremia in their lifetime (r = –0.46; 95% CI: –0.64 to –0.23; P < .001, Pearson's test); however, the slope of the CD4% regression line was less steep, and, therefore, the rate of temporal CD4% decline was slower among those who never developed a bacteremia event (–0.14; 95% CI: –0.22 to –0.06) compared with those with ≥1 event in their lifetime (–0.40; 95% CI: –0.60 to –0.20; P = .01; Fig 3). Analogous analysis using the log of CD4% resulted in a similarly slower rate of temporal decline of CD4% among children who never developed bacteremia compared with those with ≥1 event (P = .01). Similar analysis of the HIV RNA viral loads did not reveal any such differences. Analysis of the cumulative proportion of total follow-up time spent by the cohort on TMP-SMX prophylaxis revealed that TMP-SMX prophylaxis was not statistically different among children living in the post-HAART era (495 of 2649 total person-years [18.7%]) compared with those in the pre-HAART era (543 of 2836 total person-years [19%]). Similarly, the prevalence of 23-valent pneumococcal polysaccharide vaccine receipt by 3 years of age during the post-HAART era (49%) was also not increased compared with that during the pre-HAART era (49%).


Figure 3
View larger version (23K):
[in this window]
[in a new window]

  		FIGURE 3 Correlation of CD4% T-cell trends among HIV-infected children <6 years of age, comparing those children who had (BSI+) with those who never developed bacteremia (BSI–).

 
Mortality
The Kaplan-Meier analysis for time to death between HIV-infected children with bacteremia compared with those who never developed an event showed a significantly higher overall mortality (P = .013, log rank test; Fig 4). This finding was also consistent with the result from a crude Cox proportional hazard analysis (unadjusted hazard ratio: 1.72; 95% CI: 1.12 to 2.66; P = .01) and an analysis controlling for gender, race, and clinical site of care (adjusted hazard ratio: 1.78; 95% CI: 1.13 to 2.82; P = .01).


Figure 4
View larger version (16K):
[in this window]
[in a new window]

  		FIGURE 4 Kaplan-Meier analysis depicts the mortality of HIV-infected children, comparing those children who had with those who never experienced bacteremia.

 

   DISCUSSION
TOP
 ABSTRACT
 METHODS
 RESULTS
 DISCUSSION
CONCLUSIONS
 REFERENCES
 
This 18-year, multicenter, prospective perinatal cohort study of HIV-infected children clearly demonstrates an association between the implementation of HAART and a significant decrease in the incidence of bacteremia events, as well as the prolongation of the time to development of a first bacteremia when compared with the pre-HAART era. To our knowledge, this study of 364 US HIV-infected children, 68 of whom developed 118 bacteremia events, is the largest of its kind to evaluate bacteremia incidence in this population and is the first to determine a protective association with the receipt of HAART among a birth cohort.

Our results are consistent with previous studies of bacteremia incidence in HIV-infected children before the availability of HAART in which incidence ranged from 6.7 to 12.2 events per 100 person-years,20,22–24 but the results contrast with a larger study showing an incidence of 3.3 events per 100 person-years.21 They also contrast with a recent follow-up of this cohort reporting an incidence of 0.35 events per 100 person-years associated with HAART.39 Although the latter Pediatric AIDS Clinical Trials Group study involved a larger cohort of subjects, they were not a birth cohort and were subjects participating in 13 different protocols with differing selection criteria and treatments. In addition, the former studies20,22–24 describe only children <6 years of age, whereas in the latter study by the Pediatric AIDS Clinical Trials Group, 25% of children originally21 and 79% in the recent follow-up report39 were enrolled at ages ≥6 years, well past the window of high risk for infections with encapsulated organisms. In addition, this study only analyzed the incidence of first bacteremia and not all events. All of these factors may have introduced significant selection biases into their analyses. Our finding of pneumococcal infection being the predominant cause of bacteremia in almost two thirds of the cases in either era is consistent with previous reports.2–4,21,28

Potential effect modifiers of bacteremia incidence between the 2 eras include the receipt of IVIG, TMP-SMX prophylaxis, and the 23-valent pneumococcal polysaccharide vaccine; however, because this study was not designed to detect the independent contributions of multiple variables, we analyzed immunologic and virologic parameters as surrogate markers of HAART efficacy. We used the method of Denny et al50 to compare temporal trends in the CD4% cells between children with bacteremias and those who never developed an event. Not surprisingly, the temporal CD4% cell trends of children who never had bacteremia declined more slowly as time progressed. Similar trends were not noted when HIV RNA was analyzed. It seems unlikely that receipt of TMP-SMX prophylaxis or pneumococcal vaccine biased our observations toward reducing bacteremia incidence in the post-HAART era given the absence of increased use among children living in this era.

The overall mortality of HIV-infected children with bacteremia was significantly higher than that of their bacteremia-free counterparts. Whether bacteremia hastens mortality or is simply a surrogate for or the result of the overall clinical decline is difficult to ascertain from this analysis. A more complete analysis of mortality among HIV-infected children, including known risk factors, is beyond the scope of the present study and has been published previously by others.48

Although this study was prospective in nature, there are notable limitations. Despite our attempts to minimize misclassification of patients as having spent time without HAART when they could have actually been on HAART during the pre-HAART era and vice versa during the post-HAART era, a small number of patients could actually have been misclassified in such a manner, thereby potentially introducing bias; however, our analytic approach defining the HAART era as the period after January 1, 1997, could have led both to underestimation and overestimation of the actual incidence of bacteremia events during either the pre- or the post-HAART era and, therefore, could potentially have resulted in little net bias in either given direction. For instance, the small number of children who could have potentially been misclassified as not having received HAART during the defined pre-HAART era when they actually were HAART recipients could have likely received such therapy earlier because they were sicker, thereby contributing to an overestimation of bacteremia events during the defined pre-HAART era. Simultaneously, one could also argue that the benefit of HAART could have reduced the number of observed events in these same children, underestimating the rate of bacteremia events during the pre-HAART era. The converse logic could also apply to misclassified patients during the defined post-HAART era, in that the few children not on HAART may have been more stable (underestimating incidence) but also concurrently may not have experienced the benefit of HAART on reducing the risk of bacteremia (overestimating incidence). This study was also not controlled and, therefore, not designed to identify factors that affected or may have caused the decrease in bacteremia incidence. Furthermore, during the course of this 18-year study, standards in pediatric HIV care have undergone numerous changes spanning the spectrum from prevention to diagnosis to management (prophylactic antibiotics, IVIG, immunizations, mother-to-child transmission prophylaxis, antiretroviral medications and HIV RNA viral load monitoring; Fig 1). The natural history of the disease and the changing exposure to therapies (dictated by birth date) present many challenges to data analysis (Fig 1). Different combinations of these factors likely influenced bacteremia incidence. For example, children born between 1986 and 1991 would contribute little person-time to the post-HAART era because of their expected high 10-year mortality and the lower age-related mortality of children who survived to that age.

Another potential limitation that deserves mention is the possibility for ascertainment bias introduced by data from subjects who may have received the pneumococcal conjugate vaccine, which could have occurred only after it became commercially available in February 2000; ascertainment for its use would not have been possible in the original design of this prospective study. However, >95% (347 of 364 children) of our cohort was born before 1998 and, therefore, missed the opportunity for routine receipt of the pneumococcal conjugate vaccine according to the recommended 4-dose schedule (2, 4, 6, and 12–15 months of age). Thus, <5% of the cohort could be affected by such a potential vaccine-related effect modification on incidence reduction of pneumococcal bacteremias during early childhood when they would have been at highest risk for such events.


   CONCLUSIONS
TOP
 ABSTRACT
 METHODS
 RESULTS
 DISCUSSION
 CONCLUSIONS
REFERENCES
 
We conclude that HAART was associated with a significant reduction in the incidence of bacteremia in this study population and that this decline cannot be explained by age, gender, race, pneumococcal vaccine coverage, or TMP-SMX use. This reduction is largely attributable to the widespread use of HAART but also to improvements in care, such as in the identification of at-risk infants through maternal screening. Paralleling these improvements in management has been the striking decline in mother-to-child HIV transmission. The unique features of this long-term birth cohort study allowed us to measure and contextualize these trends. These findings may have significant implications for current initiatives aimed at the introduction of HAART in resource-poor settings and in their potential positive impact on morbidity and mortality from bacterial infections among HIV-infected children in these areas.


   ACKNOWLEDGMENTS
 
The Perinatal AIDS Collaborative Transmission Study and Perinatal AIDS Collaborative Transmission Study-HIV Follow-up of Perinatally Exposed Children were funded by the CDC through cooperative agreements U64/CCU207228 (Medical and Health Research Association of New York City), U64/CCU202219 (University of Medicine and Dentistry, New Jersey-New Jersey Medical School), U64/CCU306825 (University of Maryland School of Medicine), and U64/CCU404456 (Emory University School of Medicine).

At Emory University, Vickie Grimes served as research nurse for the duration of the study, and her work is greatly appreciated. At the University of Maryland, Drs Peter Vink and Vicki Tepper served as site principal investigators, Sue Hines as the study coordinator, and Katie Peery as research assistant. At the University of Medicine and Dentistry of New Jersey, Linda Bettica, RN, served as research coordinator, and Jeffrey Swerdlow served as database manager. We also thank the participants in the New York City Perinatal AIDS Collaborative Transmission Study Group: Bronx Lebanon Hospital: Saroj Bakshi, Genevieve Lambert, Elizabeth Adams, and Delia Grant; Harlem Hospital Center: Susan Champion, Julia Floyd, Cynthia Freeland, Margaret Heagarty, Pamela Prince, Desiree Minnott, and Aretha Bellmore; Jacobi Hospital Center: Joanna Dobroszycki, Adell Harris, and Andrew Wiznia; Metropolitan Hospital Center: Mahrukh Bamji, Grace Canillas, Lynn Jackson, and Nancy Cruz; Medical and Health Research Association of New York City, Inc: Tina Alford, Rosalind Carter, Mary Ann Chiasson, Eileen Rillamas-Sun, Donald Thea, and Jeremy Weedon; Montefiore Medical Center: Ellie Schoenbaum and Marcelle Naccarato. Finally, at the Centers for Disease Control and Prevention, the following staff contributed substantially to Perinatal AIDS Collaborative Transmission Study: Martha Rogers, Nathan Shaffer, Rick Steketee, and RJ Simonds, and to Perinatal AIDS Collaborative Transmission Study-HIV Follow-up of Perinatally Exposed Children: Darcy Freedman, Jeff Wiener, Bob Yang, and April Bell.


   FOOTNOTES
 
Accepted Oct 11, 2007.

Address correspondence to Bill G. Kapogiannis, MD, National Institute of Child Health and Human Development, Centers for Research for Mothers and Children, Pediatric, Adolescent and Maternal AIDS Branch, 6100 Executive Blvd, Room 4B11J, Bethesda, MD 20892-7510. E-mail: kapogiannisb{at}mail.nih.gov

The authors have indicated they have no financial relationships relevant to this article to disclose.

The findings and conclusions in this report are those of the authors and do not necessarily represent the views of the Centers for Disease Control and Prevention.

Dr Kapogiannis's current affiliation is Pediatric, Adolescent, and Maternal AIDS Branch, National Institute of Child Health and Human Development, National Institutes of Health, Bethesda, MD.

Dr M. Bulterys' current affiliation is Centers for Disease Control and Prevention, Global AIDS Program, Lusaka, Zambia.


What's Known on This Subject

HIV-infected children have a very high incidence of bacteremia, particularly in the first few years of life. HAART-associated trends reported recently among a heterogeneous nonbirth cohort are more modest, probably because the period of highest bacteremia risk was not captured.

 

What This Study Adds

This is the first large prospective birth cohort study documenting dramatic HAART-associated improvement in the incidence of bacteremias in HIV-infected children by capturing the critical early window of high bacteremia risk that other designs would miss or suffer from significant ascertainment bias.

 


   REFERENCES
TOP
 ABSTRACT
 METHODS
 RESULTS
 DISCUSSION
 CONCLUSIONS
 REFERENCES
 

  1. Hatherill M. Sepsis predisposition in children with human immunodeficiency virus. Pediatr Crit Care Med. 2005;6 (3 suppl):S92 –S98[CrossRef][Medline]
  2. Krasinski K, Borkowsky W, Bonk S, Lawrence R, Chandwani S. Bacterial infections in human immunodeficiency virus-infected children. Pediatr Infect Dis J. 1988;7 (5):323 –328[ISI][Medline]
  3. Bernstein LJ, Krieger BZ, Novick B, Sicklick MJ, Rubinstein A. Bacterial infection in the acquired immunodeficiency syndrome of children. Pediatr Infect Dis. 1985;4 (5):472 –475[ISI][Medline]
  4. Roilides E, Marshall D, Venzon D, Butler K, Husson R, Pizzo PA. Bacterial infections in human immunodeficiency virus type 1-infected children: the impact of central venous catheters and antiretroviral agents. Pediatr Infect Dis J. 1991;10 (11):813 –819[ISI][Medline]
  5. Ward JI, Zangwill KM. Haemophilus influenzae. In: Feigin RD, Cherry JD, eds. Textbook of Pediatric Infectious Diseases. 4th ed. Philadelphia, PA: WB Saunders; 1998:1464 –1482
  6. Espin Rios I, Garcia-Fulgueiras A, Navarro Alonso JA, et al. Seroconversion and duration of immunity after vaccination against group C meningococcal infection in young children. Vaccine. 2000;18 (24);2656 –2660[CrossRef][ISI][Medline]
  7. Sell SH, Wright PF, Vaughn WK, Thompson J, Schiffman G. Clinical studies of pneumococcal vaccines in infants. I. Reactogenicity and immunogenicity of two polyvalent polysaccharide vaccines. Rev Infect Dis. 1981;3 (suppl):S97 –S107[ISI][Medline]
  8. Leinonen M, Sakkinen A, Kalliokoski R, Luotonen J, Timonen M, Makela PH. Antibody response to 14-valent pneumococcal capsular polysaccharide vaccine in pre-school age children. Pediatr Infect Dis. 1986;5 (1):39 –44[ISI][Medline]
  9. Douglas RM, Paton JC, Duncan SJ, Hansman DJ. Antibody response to pneumococcal vaccination in children younger than five years of age. J Infect Dis. 1983;148 (1):131 –137[ISI][Medline]
  10. Ammann AJ, Schiffman G, Abrams D, Volberding P, Ziegler J, Conant M. B-cell immunodeficiency in acquired immune deficiency syndrome. JAMA. 1984;251 (11):1447 –1449[Abstract]
  11. Clerici M, Stocks NI, Zajac RA, et al. Detection of three distinct patterns of T helper cell dysfunction in asymptomatic, human immunodeficiency virus-seropositive patients. Independence of CD4+ cell numbers and clinical staging. J Clin Invest. 1989;84 (6):1892 –1899[ISI][Medline]
  12. Roilides E, Clerici M, DePalma L, Rubin M, Pizzo PA, Shearer GM. Helper T-cell responses in children infected with human immunodeficiency virus type 1. J Pediatr. 1991;118 (5):724 –730[CrossRef][ISI][Medline]
  13. Moir S, Malaspina A, Ogwaro KM, et al. HIV-1 induces phenotypic and functional perturbations of B cells in chronically infected individuals. Proc Natl Acad Sci USA 2001;98 (18):10362 –10367[Abstract/Free Full Text]
  14. Lane HC, Fauci AS. Immunologic abnormalities in the acquired immunodeficiency syndrome. Annu Rev Immunol. 1985;3 :477 –500[CrossRef][ISI][Medline]
  15. Moir S, Ogwaro KM, Malaspina A, et al. Perturbations in B cell responsiveness to CD4+ T cell help in HIV-infected individuals. Proc Natl Acad Sci USA 2003;100 (10):6057 –6062[Abstract/Free Full Text]
  16. Lane HC, Masur H, Edgar LC, Whalen G, Rook AH, Fauci AS. Abnormalities of B-cell activation and immunoregulation in patients with the acquired immunodeficiency syndrome. N Engl J Med. 1983;309 (8):453 –458[Abstract]
  17. Bernstein LJ, Ochs HD, Wedgwood RJ, Rubinstein A. Defective humoral immunity in pediatric acquired immune deficiency syndrome. J Pediatr. 1985;107 (3):352 –357[CrossRef][ISI][Medline]
  18. Schnittman SM, Lane HC, Higgins SE, Folks T, Fauci AS. Direct polyclonal activation of human B lymphocytes by the acquired immune deficiency syndrome virus. Science. 1986;233 (4768):1084 –1086[Abstract/Free Full Text]
  19. Simpson BJ, Andiman WA. Difficulties in assigning human immunodeficiency virus-1 infection and seroreversion status in a cohort of HIV-exposed in children using serologic criteria established by the Centers for Disease Control and Prevention. Pediatrics. 1994;93 (5):840 –842[Abstract/Free Full Text]
  20. Peters VB, Hyatt AC, Schechter C, Diamant EP, Hodes DS. Evaluation of prophylaxis against invasive pneumococcal infections in human immunodeficiency virus-infected children. Pediatr Infect Dis J. 1994;13 (7):667 –669[ISI][Medline]
  21. Dankner WM, Lindsey JC, Levin MJ. Correlates of opportunistic infections in children infected with the human immunodeficiency virus managed before highly active antiretroviral therapy. Pediatr Infect Dis J. 2001;20 (1):40 –48[CrossRef][ISI][Medline]
  22. Andiman WA, Mezger J, Shapiro E. Invasive bacterial infections in children born to women infected with human immunodeficiency virus type 1. J Pediatr. 1994;124 (6):846 –852[CrossRef][ISI][Medline]
  23. Lichenstein R, King JC Jr, Farley JJ, Su P, Nair P, Vink PE. Bacteremia in febrile human immunodeficiency virus-infected children presenting to ambulatory care settings. Pediatr Infect Dis J. 1998;17 (5):381 –385[CrossRef][ISI][Medline]
  24. Farley JJ, King JC Jr, Nair P, Hines SE, Tressler RL, Vink PE. Invasive pneumococcal disease among infected and uninfected children of mothers with human immunodeficiency virus infection. J Pediatr. 1994;124 (6):853 –858[CrossRef][ISI][Medline]
  25. Zangwill KM, Vadheim CM, Vannier AM, Hemenway LS, Greenberg DP, Ward JI. Epidemiology of invasive pneumococcal disease in southern California: implications for the design and conduct of a pneumococcal conjugate vaccine efficacy trial. J Infect Dis. 1996;174 (4):752 –759[ISI][Medline]
  26. Dagan R, Engelhard D, Piccard E, Englehard D. Epidemiology of invasive childhood pneumococcal infections in Israel. The Israeli Pediatric Bacteremia and Meningitis Group. JAMA. 1992;268 (23):3328 –3332[Abstract]
  27. Eskola J, Takala AK, Kela E, Pekkanen E, Kalliokoski R, Leinonen M. Epidemiology of invasive pneumococcal infections in children in Finland. JAMA. 1992;268 (23):3323 –3327[Abstract]
  28. National Institute of Child Health and Human Development. Intravenous immune globulin for the prevention of bacterial infections in children with symptomatic human immunodeficiency virus infection. The National Institute of Child Health and Human Development Intravenous Immunoglobulin Study Group. N Engl J Med. 1991;325 (2):73 –80[Abstract]
  29. Mofenson LM, Moye J Jr, Bethel J, Hirschhorn R, Jordan C, Nugent R. Prophylactic intravenous immunoglobulin in HIV-infected children with CD4+ counts of 0.20 x 10(9)/L or more. Effect on viral, opportunistic, and bacterial infections. The National Institute of Child Health and Human Development Intravenous Immunoglobulin Clinical Trial Study Group. JAMA. 1992;268 (4):483 –488[Abstract]
  30. Mofenson LM, Moye J Jr, Korelitz J, Bethel J, Hirschhorn R, Nugent R. Crossover of placebo patients to intravenous immunoglobulin confirms efficacy for prophylaxis of bacterial infections and reduction of hospitalizations in human immunodeficiency virus-infected children. The National Institute of Child Health and Human Development Intravenous Immunoglobulin Clinical Trial Study Group. Pediatr Infect Dis J. 1994;13 (6):477 –4784[ISI][Medline]
  31. Spector SA, Gelber RD, McGrath N, et al. A controlled trial of intravenous immune globulin for the prevention of serious bacterial infections in children receiving zidovudine for advanced human immunodeficiency virus infection. N Engl J Med. 1994;331 (18):1181 –1187[Abstract/Free Full Text]
  32. Ledergerber B, Egger M, Erard V, et al. AIDS-related opportunistic illnesses occurring after initiation of potent antiretroviral therapy: the Swiss HIV Cohort Study. JAMA. 1999;282 (23):2220 –2226[Abstract/Free Full Text]
  33. Mocroft A, Katlama C, Johnson AM, et al. AIDS across Europe, 1994–98: the EuroSIDA study. Lancet. 2000;356 (9226):291 –296[CrossRef][ISI][Medline]
  34. Ledergerber B, Mocroft A, Reiss P, et al. Discontinuation of secondary prophylaxis against Pneumocystis carinii pneumonia in patients with HIV infection who have a response to antiretroviral therapy. Eight European Study Groups. N Engl J Med. 2001;344 (3):168 –174[Abstract/Free Full Text]
  35. Palella FJ Jr, Delaney KM, Moorman AC, et al. Declining morbidity and mortality among patients with advanced human immunodeficiency virus infection. HIV Outpatient Study Investigators. N Engl J Med. 1998;338 (13):853 –860[Abstract/Free Full Text]
  36. Grau I, Pallares R, Tubau F, et al. Epidemiologic changes in bacteremic pneumococcal disease in patients with human immunodeficiency virus in the era of highly active antiretroviral therapy. Arch Intern Med. 2005;165 (13):1533 –1540[Abstract/Free Full Text]
  37. Tumbarello M, Tacconelli E, Donati KG, et al. HIV-associated bacteremia: how it has changed in the highly active antiretroviral therapy (HAART) era. J Acquir Immune Defic Syndr. 2000;23 (2):145 –151[ISI][Medline]
  38. Redd SC, Rutherford GW 3rd, Sande MA, et al. The role of human immunodeficiency virus infection in pneumococcal bacteremia in San Francisco residents. J Infect Dis. 1990;162 (5):1012 –1017[ISI][Medline]
  39. Gona P, Van Dyke RB, Williams PL, et al. Incidence of opportunistic and other infections in HIV-infected children in the HAART era. JAMA. 2006;296 (3):292 –300[Abstract/Free Full Text]
  40. Resino S, Resino R, Maria Bellon J, et al. Clinical outcomes improve with highly active antiretroviral therapy in vertically HIV type-1-infected children. Clin Infect Dis. 2006;43 (2):243 –252[CrossRef][ISI][Medline]
  41. Gray L, Newell ML, Thorne C, Peckham C, Levy J; European Collaborative Study. Fluctuations in symptoms in human immunodeficiency virus-infected children: the first 10 years of life. Pediatrics. 2001;108 (1):116 –122[Abstract/Free Full Text]
  42. Abrams EJ, Wiener J, Carter R, et al. Maternal health factors and early pediatric antiretroviral therapy influence the rate of perinatal HIV-1 disease progression in children. AIDS. 2003;17 (6):867 –877[CrossRef][ISI][Medline]
  43. Berk DR, Falkovitz-Halpern MS, Hill DW, et al. Temporal trends in early clinical manifestations of perinatal HIV infection in a population-based cohort. JAMA. 2005;293 (18):2221 –2231[Abstract/Free Full Text]
  44. Scott GB, Hutto C, Makuch RW, et al. Survival in children with perinatally acquired human immunodeficiency virus type 1 infection. N Engl J Med. 1989;321 (26):1791 –1796[Abstract]
  45. Freedman D, Koenig L, Wiener J, et al. Challenges to re-enrolling perinatally HIV-infected and HIV-exposed but uninfected children into a prospective cohort study: strategies for locating and recruiting hard-to-reach families. Paedr Perinat Epidemiol. 2006;20 (4):338 –347[CrossRef]
  46. Bulterys M, Nesheim S, Abrams EJ, et al. Lack of evidence of mitochondrial dysfunction in the offspring of HIV-infected women. Retrospective review of perinatal exposure to antiretroviral drugs in the Perinatal AIDS Collaborative Transmission Study. Ann NY Acad Sci. 2000;918 :212 –221[Abstract/Free Full Text]
  47. US Food and Drug Administration. Electronic orange book: approved drug products with therapeutic equivalence evaluations. Available at: www.accessdata.fda.gov/scripts/cder/ob/docs/obdetail.cfm?Appl_No=020659&TABLE1=OB_Rx. Accessed September 3, 2005
  48. Gortmaker SL, Hughes M, Cervia J, et al. Effect of combination therapy including protease inhibitors on mortality among children and adolescents infected with HIV-1. N Engl J Med. 2001;345 (21):1522 –1528[Abstract/Free Full Text]
  49. Gahlinger PM, Abramson JH. Computer Programs for Epidemiologic Analysis [computer program]. PEPI Version 2. Stone Mountain, GA: USD, Inc; 1995
  50. Denny T, Yogev R, Gelman R, et al. Lymphocyte subsets in healthy children during the first 5 years of life. JAMA. 1992;267 (11):1484 –1488[Abstract]
  51. Rothman K, Greenland S. Modern Epidemiology. 2nd ed. Philadelphia, PA: Lippincott-Raven; 1998
  52. U.S. Public Health Service recommendations for human immunodeficiency virus counseling and voluntary testing for pregnant women. MMWR Recomm Rep. 1995;44 (RR-7):1 –15[Medline]
  53. Recommendations of the U.S. Public Health Service Task Force on the use of zidovudine to reduce perinatal transmission of human immunodeficiency virus. MMWR Recomm Rep. 1994;43 (RR-11):1 –20[Medline]
  54. Connor EM, Sperling RS, Gelber R, et al. Reduction of maternal-infant transmission of human immunodeficiency virus type 1 with zidovudine treatment. Pediatric AIDS Clinical Trials Group Protocol 076 Study Group. N Engl J Med. 1994;331 (18):1173 –1180[Abstract/Free Full Text]
PEDIATRICS (ISSN 1098-4275). ©2008 by the American Academy of Pediatrics




This Article
Right arrow Abstract Freely available
Right arrow Full Text (PDF)
Right arrow P3Rs: Submit a response
Right arrow Alert me when this article is cited
Right arrow Alert me when P3Rs are posted
Right arrow Alert me if a correction is posted
Right arrow Citation Map
Services
Right arrow E-mail this article to a friend
Right arrow Similar articles in this journal
Right arrow Similar articles in PubMed
Right arrow Alert me to new issues of the journal
Right arrow Add to My File Cabinet
Right arrow Download to citation manager
Right arrow reprints & permissions
Citing Articles
Right arrow Citing Articles via CrossRef
Google Scholar
Right arrow Articles by Kapogiannis, B. G.
Right arrow Articles by Bulterys, M.
PubMed
Right arrow PubMed Citation
Right arrow Articles by Kapogiannis, B. G.
Right arrow Articles by Bulterys, M.
Related Collections
Right arrow Infectious Disease & Immunity