Accuracy of a Modified qSOFA Score for Predicting Critical Care Admission in Febrile Children

BACKGROUND AND OBJECTIVES: The identification of life-threatening infection in febrile children presenting to the emergency department (ED) remains difficult. The quick Sequential Organ Failure Assessment (qSOFA) was only derived for adult populations, implying an urgent need for pediatric scores. We developed and validated a novel, adapted qSOFA score (Liverpool quick Sequential Organ Failure Assessment [LqSOFA]) and compared its performance with qSOFA, Pediatric Early Warning Score (PEWS), and National Institute for Health and Care Excellence (NICE) high-risk criteria in predicting critical care (CC) admission in febrile children presenting to the ED. METHODS: The LqSOFA (range, 0–4) incorporates age-adjusted heart rate, respiratory rate, capillary refill, and consciousness level on the Alert, Voice, Pain, Unresponsive scale. The primary outcome was CC admission within 48 hours of ED presentation, and the secondary outcome was sepsisrelated mortality. LqSOFA, qSOFA, PEWS, and NICE high-risk criteria scores were calculated, and performance characteristics, including area under the receiver operating characteristic curve, were calculated for each score. RESULTS: In the initial (n = 1121) cohort, 47 CC admissions (4.2%) occurred, and in the validation (n = 12 241) cohort, 135 CC admissions (1.1%) occurred, and there were 5 sepsis-related deaths. In the validation cohort, LqSOFA predicted CC admission with an area under the receiver operating characteristic curve of 0.81 (95% confidence interval [CI], 0.76 to 0.86), versus qSOFA (0.66; 95% CI, 0.60 to 0.71), PEWS (0.93; 95% CI, 0.90 to 0.95), and NICE high-risk criteria (0.81; 95% CI, 0.78 to 0.85). For predicting CC admission, the LqSOFA outperformed the qSOFA, with a net reclassification index of 10.4% (95% CI, 1.0% to 19.9%). CONCLUSIONS: In this large study, we demonstrate improved performance of the LqSOFA over qSOFA in identifying febrile children at risk for CC admission and sepsis-related mortality. Further validation is required in other settings. WHAT’S KNOWN ON THIS SUBJECT: The quick Sequential Organ Failure Assessment has been shown to more accurately predict mortality or ICU transfer than systemic inflammatory response syndrome or the quick Pediatric Logistic Organ Dysfunction-2 in an emergency department population, but with only moderate prognostic accuracy. WHAT THIS STUDY ADDS: In this retrospective study of .12 000 febrile children, the Liverpool quick Sequential Organ Failure Assessment outperforms the quick Sequential Organ Failure Assessment in predicting critical care admission. Liverpool quick Sequential Organ Failure Assessment is a rapid bedside tool that should undergo implementation testing. To cite: Romaine ST, Potter J, Khanijau A, et al. Accuracy of a Modified qSOFA Score for Predicting Critical Care Admission in Febrile Children. Pediatrics. 2020;146(4): e20200782 Institute of Infection and Global Health and School of Medicine, University of Liverpool, Liverpool, United Kingdom; Department of Infectious Diseases, Informatics Department, and Emergency Department, Alder Hey Children’s National Health Service Foundation Trust, Liverpool, United Kingdom; Paediatric Critical Care Research Group, Child Health Research Centre, The University of Queensland and Paediatric ICU, Queensland Children’s Hospital, South Brisbane, Australia; Centre for Trials Research, College of Biomedical and Life Sciences, Cardiff University, Cardiff, United Kingdom; and Liverpool Health Partners, Liverpool, United Kingdom Prof Carrol conceptualized and designed the study, analyzed data, reviewed and revised the manuscript, and oversaw all aspects of the study; Dr Romaine drafted the initial manuscript, cleaned data, analyzed data, and reviewed and revised the manuscript; Drs Khanijau, Wright, and McGalliard, Mr Leigh, and Ms Potter collected data, cleaned data, conducted initial analyses, and reviewed and revised the manuscript; Mr Edwardson and Mr Johnston designed the data collection instruments, conducted data extraction from the electronic patient record, and reviewed and revised the manuscript; Dr Kerr supervised management and flow of patients in the emergency department and reviewed and revised the manuscript; (Continued) PEDIATRICS Volume 146, number 4, October 2020:e20200782 ARTICLE by guest on July 14, 2021 www.aappublications.org/news Downloaded from Acute febrile illness is one of the most common reasons for presentation to the pediatric emergency department (ED). Within this heterogeneous group of patients, only a small minority will have sepsis. However, for these children, mortality rates remain high, and delays in recognition and treatment of sepsis lead to an increased risk of mortality. To date, a sensitive and specific tool to identify patients with sepsis in the ED remains elusive. In adults, the quick Sequential Organ Failure Assessment (qSOFA) score is proposed as a screening tool for clinicians outside the ICU to promptly identify patients with infection who are more likely to have poorer outcomes. The presence of $2 of the 3 qSOFA components, altered mentation, raised respiratory rate (RR), and low systolic blood pressure (BP), was associated with an increased risk of mortality, but the derivation and validation of Sepsis-3 and the qSOFA did not involve pediatric data. The use of the qSOFA to identify pediatric sepsis has many limitations. First, researchers in several studies in both adult and pediatric populations have reported poor sensitivity, ranging from 37% to 70% in predicting in-hospital mortality and only 36% to 54% in predicting ICU admission. Second, researchers in many qSOFA studies have used mortality as their primary outcome, but mortality in pediatric ED settings is ,1%, therefore critical care (CC) admission is a more suitable outcome because this allows assessment of whether the score can identify those patients requiring additional support, regardless of survival. Third, hypotension is a late sign of pediatric septic shock. Furthermore, BP is infrequently measured in the pediatric ED. Schlapbach et al compared the prognostic performance of an ageadjusted qSOFA with systemic inflammatory response syndrome (SIRS) criteria in PICU patients with infection and demonstrated that the qSOFA was not substantially superior to SIRS criteria at predicting mortality and prolonged ICU stay. Researchers of a small pediatric ED study comparing the qSOFA, quick Pediatric Logistic Organ Dysfunction-2, and SIRS criteria for their ability to predict mortality or ICU admission found the qSOFA had the greatest predictive validity but achieved only moderate prognostic accuracy. To address these limitations, we amended the age-adjusted qSOFA to create a novel score known as the Liverpool quick Sequential Organ Failure Assessment (LqSOFA). Although different scoring systems vary in relation to the number of items considered, and the cutoffs applied, past literature indicates that respiratory, cardiovascular, and central nervous system alterations are the most consistent features of severe illness in children and adults. Accordingly, the qSOFA in adults is based on RR, systolic BP, and Glasgow Coma Scale (GCS). In contrast to adults, hypotension represents a late sign of pediatric septic shock. Therefore, a predictive score that incorporates earlier signs of shock, such as tachycardia and prolonged capillary refill time (CRT), could improve on the sensitivity demonstrated by the qSOFA. Researchers of a study assessing various vital signs for their ability to predict severity of infection in an ED population reported heart rate (HR) to be the most sensitive. Researchers in several studies in both adults and children have found a prolonged CRT acts as a marker for sepsis and is associated with a greater degree of organ dysfunction (as measured by Sequential Organ Failure Assessment score) and risk of mortality. The Alert, Voice, Pain, Unresponsive (AVPU) scale is the standard assessment of neurologic state in pediatric EDs in the United Kingdom. It is quick and simple to measure and has been shown to correlate well with GCS. For these reasons, we a priori selected RR, HR, CRT, and AVPU for the LqSOFA. A recent secondary analysis of the Fluid Expansion as Supportive Therapy study demonstrated the relevance of respiratory, cardiovascular, and central nervous system dysfunction in identifying patients at a greater risk of mortality, further supporting this approach. The 2005 Pediatric Sepsis Consensus thresholds for abnormal HR and RR, used in both SIRS and the age-adjusted qSOFA, were based on values that were likely determined through clinical consensus. In contrast, evidencebased sets of reference ranges have since been developed, on ED and hospitalized patients, which may be more discriminative in the febrile ED population. Therefore, we compared these 2 sets of reference ranges to identify the most suitable for use in the LqSOFA. Our aim with the current study was to develop and validate a novel, rapid, bedside scoring system for use in febrile children attending the ED to predict CC admission within 48 hours. We compared the performance of our score with the age-adjusted qSOFA, the Pediatric Early Warning Score (PEWS) in routine use at our hospital, and National Institute for Health and Care Excellence (NICE) high-risk criteria.

WHAT'S KNOWN ON THIS SUBJECT: The quick Sequential Organ Failure Assessment has been shown to more accurately predict mortality or ICU transfer than systemic inflammatory response syndrome or the quick Pediatric Logistic Organ Dysfunction-2 in an emergency department population, but with only moderate prognostic accuracy.
WHAT THIS STUDY ADDS: In this retrospective study of .12 000 febrile children, the Liverpool quick Sequential Organ Failure Assessment outperforms the quick Sequential Organ Failure Assessment in predicting critical care admission. Liverpool quick Sequential Organ Failure Assessment is a rapid bedside tool that should undergo implementation testing. Acute febrile illness is one of the most common reasons for presentation to the pediatric emergency department (ED). 1 Within this heterogeneous group of patients, only a small minority will have sepsis. However, for these children, mortality rates remain high, 2 and delays in recognition and treatment of sepsis lead to an increased risk of mortality. 3 To date, a sensitive and specific tool to identify patients with sepsis in the ED remains elusive. In adults, the quick Sequential Organ Failure Assessment (qSOFA) score is proposed as a screening tool for clinicians outside the ICU to promptly identify patients with infection who are more likely to have poorer outcomes. 4,5 The presence of $2 of the 3 qSOFA components, altered mentation, raised respiratory rate (RR), and low systolic blood pressure (BP), was associated with an increased risk of mortality, but the derivation and validation of Sepsis-3 and the qSOFA did not involve pediatric data.
The use of the qSOFA to identify pediatric sepsis has many limitations. First, researchers in several studies in both adult and pediatric populations have reported poor sensitivity, ranging from 37% to 70% in predicting in-hospital mortality [6][7][8] and only 36% to 54% in predicting ICU admission. [7][8][9][10] Second, researchers in many qSOFA studies have used mortality as their primary outcome, 4,6-8 but mortality in pediatric ED settings is ,1%, therefore critical care (CC) admission is a more suitable outcome because this allows assessment of whether the score can identify those patients requiring additional support, regardless of survival. 10 Third, hypotension is a late sign of pediatric septic shock. 3 Furthermore, BP is infrequently measured in the pediatric ED. 11 Schlapbach et al 6 compared the prognostic performance of an ageadjusted qSOFA with systemic inflammatory response syndrome (SIRS) criteria in PICU patients with infection and demonstrated that the qSOFA was not substantially superior to SIRS criteria at predicting mortality and prolonged ICU stay. Researchers of a small pediatric ED study comparing the qSOFA, quick Pediatric Logistic Organ Dysfunction-2, and SIRS criteria for their ability to predict mortality or ICU admission found the qSOFA had the greatest predictive validity but achieved only moderate prognostic accuracy. 9 To address these limitations, we amended the age-adjusted qSOFA to create a novel score known as the Liverpool quick Sequential Organ Failure Assessment (LqSOFA). Although different scoring systems vary in relation to the number of items considered, and the cutoffs applied, past literature indicates that respiratory, cardiovascular, and central nervous system alterations are the most consistent features of severe illness in children and adults. Accordingly, the qSOFA in adults is based on RR, systolic BP, and Glasgow Coma Scale (GCS). 4 In contrast to adults, hypotension represents a late sign of pediatric septic shock. 3 Therefore, a predictive score that incorporates earlier signs of shock, such as tachycardia and prolonged capillary refill time (CRT), 3,12 could improve on the sensitivity demonstrated by the qSOFA. Researchers of a study assessing various vital signs for their ability to predict severity of infection in an ED population reported heart rate (HR) to be the most sensitive. 13 Researchers in several studies in both adults and children have found a prolonged CRT acts as a marker for sepsis 12,13 and is associated with a greater degree of organ dysfunction (as measured by Sequential Organ Failure Assessment score) 14 and risk of mortality. 15,16 The Alert, Voice, Pain, Unresponsive (AVPU) scale is the standard assessment of neurologic state in pediatric EDs in the United Kingdom. It is quick and simple to measure and has been shown to correlate well with GCS. 17 For these reasons, we a priori selected RR, HR, CRT, and AVPU for the LqSOFA. A recent secondary analysis of the Fluid Expansion as Supportive Therapy study demonstrated the relevance of respiratory, cardiovascular, and central nervous system dysfunction in identifying patients at a greater risk of mortality, further supporting this approach. 18 The 2005 Pediatric Sepsis Consensus 19 thresholds for abnormal HR and RR, used in both SIRS and the age-adjusted qSOFA, were based on values that were likely determined through clinical consensus. 20 In contrast, evidencebased sets of reference ranges have since been developed, on ED 21 and hospitalized patients, 22 which may be more discriminative in the febrile ED population. Therefore, we compared these 2 sets of reference ranges to identify the most suitable for use in the LqSOFA.
Our aim with the current study was to develop and validate a novel, rapid, bedside scoring system for use in febrile children attending the ED to predict CC admission within 48 hours. We compared the performance of our score with the age-adjusted qSOFA, the Pediatric Early Warning Score (PEWS) in routine use at our hospital, and National Institute for Health and Care Excellence (NICE) high-risk criteria. 23

Study Population and Definitions
The study was conducted at a single center: Alder Hey Children's Hospital, Liverpool, United Kingdom. This is one of Europe's largest pediatric hospitals, managing ∼60 000 ED attendances annually. 24 The initial cohort used to amend the qSOFA into the LqSOFA (referred to as amendment cohort) was used in a previously published cohort from a prospective data set of febrile children (,16 years) attending the ED between November 2010 and April 2012. 24 Children were eligible if the treating clinician decided to perform blood tests. Of the 1872 patients recruited to this study, 670 were excluded because of a lack of capacity for study recruitment in the ED, 15 declined consent, and 4 were excluded because of a primary immunodeficiency. The remaining cohort of 1183 patients was used to develop the LqSOFA score by using age-adjusted percentile ranges and rapid-assessment adaptations of other components of the qSOFA score (Supplemental Information). For the validation cohort, children were identified retrospectively from the hospital electronic patient record if they presented to the ED between September 1, 2015, and August 31, 2017. Eligible patients were identified by reviewing ED and CC admissions during this period. Fever was defined as a temperature $38°C recorded in the ED or a history of fever reported in the previous 3 days. Exclusion criteria were missing observation data, no history of fever, and transfer from another hospital. Missing observation data were defined as missing $2 components required to calculate any of the scoring systems. If only 1 component was missing (ie, not measured), this was deemed to be normal. Ethical approval was granted for both cohorts (research ethics committee reference: 10/H1014/53 and 16/LO/1684).

Outcome Definitions
The primary outcome was CC admission within 48 hours of ED attendance. The secondary outcome was sepsis-related mortality. This was defined as CC admission within 48 hours of ED presentation with suspected sepsis, which led directly or indirectly to in-hospital death within 28 days of admission, as determined by a review of the clinical notes.

Score Calculations
The variables for LqSOFA were developed by using adapted qSOFA components (Supplemental Information). Two evidence-based sets of reference ranges for abnormal HR and RR (O'Leary et al 21 and Bonafide et al 22 ) were compared in the amendment cohort, with the highest performing reference range selected for use in the LqSOFA (Supplemental Information). Scores for LqSOFA, qSOFA, PEWS, and NICE high-risk criteria were calculated by using the worst observations recorded in the ED. Each scoring system differs in both the included vital signs and the age-thresholds used; these can be seen in full in the Supplemental Information. PEWS was in routine use clinically and was therefore calculated electronically in real-time, whereas scores for the other 3 systems were calculated retrospectively for study purposes only, by using individually documented vital sign information in the electronic patient record. As with the qSOFA, each component in the LqSOFA could be scored as 0 or 1, making a total possible score of 4 ( Table 1). If blood tests were performed, the worst lactate and worst C-reactive protein (CRP) recorded in the ED were collected.

Statistical Analysis
We assessed the prognostic ability of each scoring system using the area under the receiver operating characteristic curve (AUC). Sensitivity, specificity, positive and negative predictive value, positive and negative likelihood ratio, odds ratio, and accuracy were calculated for each score alongside an asymptotic 95% confidence interval (CI). LqSOFA and qSOFA were compared by using net reclassification improvement (NRI). 25 Statistical analyses were performed by using SPSS version 25 (IBM SPSS Statistics, IBM Corporation) and R version 3.6.1. 26 Graphics were generated by using ggplot2 27 and eulerr. 28

Study Population
A total of 62 (5.2%) patients were excluded from the amendment cohort because of missing observation data, leaving 1121 patients for the amendment cohort (Supplemental Information). Of 14 121 patients identified for the validation cohort, 1880 were excluded: 1249 (8.8%) because of missing observation data, 454 (3.2%) because of no history of fever, 96 (0.7%) because of duplicates, 77 (0.5%) because patients were $16 years of age, and 4 (0.03%) because of transfer from another hospital (Fig 1). Importantly, 848 (75.6%) and 8354 (68.2%) patients in the amendment and validation cohorts, respectively, did not have any BP measurements recorded while in the ED. Demographic characteristics, clinical outcomes, and scores of the amendment and validation cohorts are summarized in Table 2.

Amendment Cohort
By using the 99th centile cutoffs from Bonafide et al, 22  Flow diagram summarizing the composition of the validation cohort.  Fig 3). For the secondary outcome, both lactate (AUC, 0.89; 95% CI, 0.71 to 1) and CRP (AUC, 0.83; 95% CI, 0.68 to 0.98) revealed good predictive ability ( Table 3, Supplemental Fig 6). Several standard performance metrics of CRP and lactate, as well as each of the 4 scoring systems, are presented in Tables 3 and 4.

DISCUSSION
With our data, we show that a simple 4-item score reveals improved discriminant ability compared with the age-adjusted qSOFA and NICE high-risk criteria. Our novel, rapid, bedside score for children, the LqSOFA score, has revealed good discriminative ability in predicting CC admission in febrile ED patients in a large independent validation cohort.
In the ED, it is important to identify the "needle in the haystack": those children requiring further investigation and urgent treatment. The use of a sepsis-specific score in a pediatric ED may be of limited value because of the low prevalence, whereas predicting which children might require CC admission may be more useful.

LqSOFA
For predicting CC admission, the LqSOFA demonstrated good prognostic value. With a cutoff of $2, the score revealed excellent specificity but was limited by a relatively low sensitivity, consistent with sensitivities reported for the qSOFA in both adults and children. [7][8][9][10] Given that the these scores are designed as tools to aid clinician decision-making, rather than simply predict outcome, it has been argued that this rate of falsenegatives is unacceptably high. 10 However, a cutoff of $1 for the

FIGURE 2
Receiver operating characteristic curves for each scoring system or blood biomarker in predicting CC admission in the amendment cohort. PEWS data are not available because it was not in use at the hospital when amendment cohort data were collected.
LqSOFA produced a more favorable balance of sensitivity and specificity. Overall, the LqSOFA demonstrates superior discriminative ability than that reported with the qSOFA in the pediatric population, both in the current study and compared with previous research. 6,9,30 In the current study, the LqSOFA revealed the greatest positive predictive value of all 4 scoring systems, although this was only 34%. The low values for positive predictive value seen across all scoring systems likely reflect the low prevalence of sepsis within the ED population and highlight the difficulty in identifying such cases. There has rightly been a focus in recent years on preventing cases of sepsis being missed, leading to the prioritization of sensitivity over other performance measures. In the current study, NICE high-risk criteria achieved excellent sensitivity, greater than the LqSOFA and PEWS, but at the cost of limited specificity. The potential disadvantages of relying on such scoring systems, in overdiagnosis and overtreatment of sepsis, are significant. 31 Therefore, scoring systems with high sensitivity but poor specificity should be avoided in favor of more balanced scoring systems, ideally those that are quick and simple to calculate. The use of CRT and HR as a proxy for cardiovascular dysfunction means the LqSOFA can be calculated rapidly by using routine clinical observations and without any equipment, such as appropriately sized BP cuffs, a significant advantage over other scoring systems. In addition, shock in children may be present before hypotension is measured 3 ; therefore, these parameters may detect impending shock earlier. This facilitates its potential for standardized use across many other settings including primary care and resource-poor settings. The large number of age categories in the LqSOFA potentially provides greater accuracy but also increases the complexity of the score in comparison with other systems with fewer age categories (eg, NICE high-risk criteria). However, when used electronically, this greater complexity would not equate to a more time-consuming or complicated tool for the clinician; the number of components in any given scoring system has a far greater impact in this regard. Overall, the primary limitation of the use of the LqSOFA is the lower sensitivity achieved in comparison with PEWS. However, the excellent specificity suggests the LqSOFA may be useful in determining which children are at low risk of CC admission or in situations in which discriminant ability is prioritized over sensitivity.
qSOFA For predicting both the primary and secondary outcome, the qSOFA performed less well than the LqSOFA and PEWS. Our results are consistent with previous research. Schlapbach et al 6 reported an AUC of 0.64 for the qSOFA in predicting mortality in a PICU cohort. A limitation of their study is the use of a PICU population, given that "quick" organ dysfunction scores are designed to alert the clinician to those at risk for sepsis, and most PICU patients will already have some sort of organ dysfunction and/or organ support. In an ED study, Van Nassau et al 9 assessed the ability of the qSOFA to predict PICU transfer and/or mortality, reporting an AUC of 0.72 and a sensitivity of 50%, although their study used a relatively small sample (n = 864). With our study, we provide further evidence that the qSOFA lacks the prognostic ability to justify its introduction into routine practice in the pediatric ED.
A further limitation of using the qSOFA in the pediatric population, highlighted by the current study, is the infrequency of BP measurements in the pediatric ED. In the validation cohort, 8354 (68.2%) patients did not have their BP recorded while in the ED; this is consistent with previous research. 32 The lack of BP data may have affected the performance of the qSOFA; however, to exclude these patients from the study would mean excluding the vast majority of eligible patients. The results generated from such an analysis would provide little information on the potential "realworld" performance of the score. BP is included in many PEWS, 33 however, BP may not be measured at triage; NICE sepsis guidance recommends BP is measured only when tachycardia or prolonged CRT are present. 23 The infrequency of BP measurements may lead to missed opportunities for the early detection of sepsis. In the current study, almost one-quarter of those admitted to CC within 48 hours did not have their BP measured while in the ED, suggesting the absence of BP measurement is not confined to "well" children. A score that does not require BP measurement, such as LqSOFA, therefore presents a significant advantage.

PEWS
PEWS demonstrated the greatest discriminative ability for both the primary and secondary outcomes, consistent with previous research. 33 Most PEWS are designed to identify deterioration in hospitalized children and have therefore not been validated in an ED setting. A previous ED study revealed that PEWS was an independent predictor of ICU admission within 48 hours, or death within 30 days, in unselected pediatric ambulance patients presenting to the ED. 34 The large number of different heterogeneous PEWS in use is a disadvantage because this leads to inconsistency in detecting and responding to acute illness. qSOFA and especially LqSOFA therefore offer an advantage over PEWS as standardized tools to improve sepsis recognition. Importantly, PEWS was the only scoring system in routine clinical use during the study, and hence PEWS scores were known by treating physicians. This may have influenced decisions regarding transfer to CC, potentially altering the performance of the score in comparison with the other scoring systems, which had not been applied clinically. Furthermore, compared with only 3 and 4 items in the qSOFA and LqSOFA, respectively, PEWS often include several more variables and is therefore more complicated and time-consuming to calculate (Supplemental Fig 5). 33 Although a granular score may perform better than a parsimonious score such as ours, it may be less feasible in practice for rapid assessment in a busy ED.

NICE High-Risk Criteria
A previous study compared the NICE high-risk criteria in adults, with qSOFA, National Early Warning Score, and SIRS for their ability to predict mortality in an ED and ward setting. 35 NICE high-risk criteria achieved a sensitivity of only 58.9% and were not independently associated with an adverse outcome. In the current study, for the primary outcome we found good discriminative ability suggesting NICE high-risk criteria correlate well with risk of CC admission, and the criteria correctly identified all 5 sepsis-related deaths. However, 55% of validation cohort patients were positive for NICE high-  8risk criteria, suggesting the criteria lack the specificity to be useful alone as a screening tool for possible sepsis in the ED. Given that the guidelines recommend senior review for children with NICE high-risk criteria features, 23 implementation is likely to have huge resource implications in an already stretched health service. Importantly, our analysis did not include many other NICE high-risk criteria features such as cyanosis, nonblanching rash, and "appears ill to health care worker," suggesting the NICE high-risk criteria could be simplified.

Lactate and CRP
A strong association between lactate and mortality in pediatric sepsis has previously been reported. 36 In the current study, lactate revealed good discriminative ability for sepsisrelated mortality, providing further support for including lactate measurement in risk stratification in pediatric sepsis. 37 CRP revealed poor versus good discrimination for the primary versus secondary outcomes, respectively, which is consistent with several studies in which researchers report CRP as a poorer predictor of sepsis than other available biomarkers. 38

Comparison of All Scores
All 4 scoring systems in the current study used different age-specific thresholds. Schlapbach et al 6 noted the lack of age-specific cutoffs for the original qSOFA and therefore applied those used in the corrected 2005 Pediatric Sepsis definitions. 19,39 These age-specific thresholds are notably lower than those described by Bonafide et al 22 and used in the LqSOFA. NICE high-risk criteria and the PEWS in the current study generally use thresholds in between these 2. This lack of consensus in agespecific thresholds may explain a large part of the difference in prognostic ability seen between the scoring systems and highlights the urgent need for standardization of reference ranges. The Bonafide et al 22 reference ranges for hospitalized children are closer to the febrile child population than the O'Leary et al 21 reference ranges for low acuity children presenting to the ED or the Fleming et al 20 reference ranges for healthy children used in our PEWS and are therefore the best ranges to identify children with severe enough illness to require CC admission.
For operationalization of the score in the ED, a highly sensitive score (NICE high-risk criteria $1) could be used Percentage of validation cohort patients admitted to CC by LqSOFA score, qSOFA score, NICE high-risk criteria, and PEWS, annotated with raw figures above each bar.
as a screening test in triage to identify the group at risk for sepsis, followed by a highly specific score (LqSOFA$2) to identify those children at highest risk of poor outcome. This can be implemented electronically (such as reported by Balamuth et al 40 ), with an LqSOFA score $2 prompting immediate senior review.

Strengths and Limitations
To our knowledge, this is the largest study in which the performance of quick assessment scores, including the qSOFA, was assessed in a pediatric ED population. Our population reflects a real-world highincome ED setting, with a low prevalence of sepsis. We compared different age-specific cutoffs and 2 other prediction scores for sepsis: NICE high-risk criteria and qSOFA. The LqSOFA can be calculated rapidly across a variety of different health care settings, because no equipment is required. Limitations include use of retrospective data, patients from a single center, criteria of CC admission which may depend on local practice, and the low mortality. In addition, hypothermic patients presenting with sepsis may have been missed.

CONCLUSIONS
Overall, with the findings of the current study, we demonstrate the superior performance of LqSOFA and question the use of the qSOFA to identify patients at risk for sepsis in the pediatric ED. The LqSOFA requires further evaluation in other settings before recommending more widespread use as entry criterion into randomized controlled trials in the pediatric ED.

ACKNOWLEDGMENTS
We thank members of the research study team who helped recruit patients into the observational, prospective study which formed the amendment cohort. We also thank the children included in the prospective study, and their parents for giving consent for them to participate in the study. We also extend thanks to the nursing and medical staff in the ED at Alder Hey Children's NHS Foundation Trust for their contribution to the study.