medRxiv preprint doi: https://doi.org/10.1101/2024.12.10.24318795; this version posted December 11, 2024.

The copyright holder for this

preprint (which was not certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity.

It is made available under a CC-BY 4.0 International license .

Algorithm Refinement of the National Early Warning

Score 2 (NEWS2): Study Protocol
Chris Plummer, BSc, PhD, BM, BCh, FRCP 1,2,* Madison Milne-Ives, BAS, MSc,2,5, Cen Cong, BEng,
MA,3 Lynsey Threlfall, MBBS, MRCP,1,2 Edward Meinert, MA, MSc, MBA, MPA, MPH, PhD, CEng
FBCS EUR ING3,4
1
The Newcastle Upon Tyne Hospitals NHS Foundation Trust, Newcastle upon Tyne, NE7 7DN, UK
2
NIHR Newcastle Biomedical Research Centre, Newcastle University, Newcastle upon Tyne, NE1
7RU, UK
3
Translational and Clinical Research Institute, Newcastle University, Newcastle upon Tyne, UK
4
Department of Primary Care and Public Health, Imperial College London, London, UK
5
Centre for Health Technology, School of Nursing and Midwifery, University of Plymouth,
Plymouth, UK

Correspondence: [*] Correspondence to: Chris Plummer, National Institute for Health and Care
Research Newcastle Biomedical Research Centre, Newcastle University, Newcastle, UK.
Tel: +44(0) 191 2081148; e-mail; [email protected]

NOTE: This preprint reports new research that has not been certified by peer review and should not be used to guide clinical practice.
 medRxiv preprint doi: https://doi.org/10.1101/2024.12.10.24318795; this version posted December 11, 2024. The copyright holder for this
preprint (which was not certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity.

It is made available under a CC-BY 4.0 International license .

Abstract
Introduction: The second iteration of the National Early Warning Score (NEWS2) has been
widely adopted for predicting patient deterioration in healthcare settings using routinely
collected physiological observations. The use of NEWS2 has been shown to reduce in-hospital
mortality, but it has limited accuracy in the prediction of clinically important outcomes,
especially over longer time periods. The increasing implementation of digital patient
observations and health records presents an opportunity to investigate whether the addition of
individual patient characteristics and information about their care-setting, would improve the
predictive accuracy of the score.
Methods and analysis: This protocol describes the work to determine whether the performance
of the current NEWS2 system could be improved by the use of additional variables. The project
has been designed after an extensive scoping review of existing literature on NEWS2 and an
exploration of retrospective cohort data in The Newcastle upon Tyne Hospitals NHS Foundation
Trust, with input from key clinical stakeholders.
Ethics and dissemination: The project has received competitive funding following peer-review,
from the NIHR Newcastle Biomedical Research Centre as an Interdisciplinary Research Award.
Ethical approval has been requested. Findings are expected to be produced by June 2025, and
will be disseminated at symposia, conferences and in journal publications.
Keywords: National Early Warning Score, clinical deterioration, performance, proof-of-concept

Strengths and limitations of this study

Strengths:
-​ This work highlights the importance of investigating the use of additional clinical
variables to those used in NEWS2, in the development of a new early warning score
-​ The study design was informed by an evidence synthesis of the literature
Limitations:
-​ Some retrospective data sets may be of low quality and/or incomplete
-​ External validation will be needed to test algorithm generalisability
 medRxiv preprint doi: https://doi.org/10.1101/2024.12.10.24318795; this version posted December 11, 2024. The copyright holder for this
preprint (which was not certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity.

It is made available under a CC-BY 4.0 International license .

1. Introduction
The second National Early Warning Score (NEWS2) system is a widely used tool for the
systematic documentation and identification of clinical deterioration. It has been demonstrated
to be effective at reducing in-hospital mortality [1–4], it facilitates effective communication
between clinicians and enables timely interventions to improve patient outcomes, but NEWS2
has limited positive and negative predictive accuracy [5–7], particularly in predicting adverse
events beyond 24 hours [8–11].
The use of digital technologies in healthcare presents an opportunity to evaluate whether the
inclusion of additional routinely collected variables would improve the predictive accuracy of an
early warning score and thus reduce patient risk. The same technologies could then be used to
provide clinical decision support using artificial intelligence-derived algorithms that would not
be practical using paper observation charts and healthcare records [1,11].
This further personalisation of risk prediction is increasingly important in our ageing population.
Older adults are particularly vulnerable to sudden changes in physiological status [13], but this
group was not included in the development of NEWS2 [14]. Improved predictive accuracy in this
growing frail and vulnerable group could significantly improve patient outcomes [14–16].
In an evaluation of the performance of the original NEWS, it demonstrated consistent predictive
accuracy across multiple patient groups [5]. However, the performance of NEWS2 varies
between care settings, and between patients with different characteristics [11,17,18]. NEWS2
has been found to have significantly lower specificity, sensitivity, and positive predictive value
for those who were at risk of type II respiratory failure (T2RF), but higher specificity and
positive predictive value for those with documented T2RF status, then the original NEWS [17].
The accuracy of NEWS has also been found to vary between care settings [18]. These
observations suggest that the inclusion of individualised patient and care setting characteristics
may improve the predictive accuracy, sensitivity and specificity of the system and thus improve
outcomes in a range healthcare settings and in diverse patient groups.
To determine which variables may improve the risk prediction of NEWS2 we have reviewed six
databases (CINAHL, PubMed, Embase, ScienceDirect, Cochrane Library and Web of Science)
[19]. Our preliminary findings show the demographic variables including age and ethnicity
[6,20], trend data [21–23], and adjustments to component weighting [12,24] may improve on
the performance of NEWS2 in predicting clinical deterioration.
The principal aim of this work is to develop an algorithm that can improve on the predictive
accuracy and thus clinical value of the NEWS2 system. The objectives of this study are:
1.​ To determine which additional variables have been previously used to enhance the
accuracy of clinical early warning scores, including NEWS2;
2.​ To collect a large set of patient data that links observations, demographic, and
admissions data with key deterioration-related outcomes;
 medRxiv preprint doi: https://doi.org/10.1101/2024.12.10.24318795; this version posted December 11, 2024. The copyright holder for this
preprint (which was not certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity.

It is made available under a CC-BY 4.0 International license .

3.​ To use these data to train an artificial intelligence (AI) model to determine which
variables, and what weightings, provide optimal predictive accuracy;
4.​ To test the accuracy, sensitivity, and specificity of the algorithm;
5.​ To consider how the new model could be applied across a wide range of healthcare
settings, including different physiological observation equipment, different electronic
patient record systems, or in paper-based clinical record systems.

2. Methods and Analysis

The study is a retrospective cohort study using approximately 7 years (2017-2023) of historical,
anonymised patient data from the Newcastle upon Tyne Hospitals NHS Foundation Trust to
explore ways in which a new alerting system could improve on NEWS2. We will recruit PPIE
members to join the project steering committee through well-established PPIE processes in the
Newcastle BRC Informatics and Precision Care for an Ageing Population theme and Newcastle
University’s Clinical Trials Unit. PPIE members will be included in all project management group
meetings and will provide their lived experience with hospital monitoring to highlight any
concerns or barriers they foresee in the adoption and implementation of a modified NEWS
system. We will recruit at least one older adult to ensure that we are including a perspective
that has been missing in the development of previous NEWS systems.
Data collection: We will use a large number of routinely collected patient variables linked with
individual patient outcomes and use these linked data to determine which variables, and with
which algorithm, best predicts outcomes. The Newcastle Hospitals NHS Foundation Trust has an
established information governance system for the use of anonymised patient data for research,
which has been successfully used in the ADMISSION [32] and AI-Multiply [33] studies. A data
scientist at the Trust will exclude all data from patients who have opted out of their data being
used for research purposes through local or national processes. The data will then be
de-identified before the research team can access it within the Trust’s IT network for training
and testing the algorithm.

Figure 1. Study logic diagram

 medRxiv preprint doi: https://doi.org/10.1101/2024.12.10.24318795; this version posted December 11, 2024. The copyright holder for this
preprint (which was not certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity.

It is made available under a CC-BY 4.0 International license .

Data analysis: Deidentified data from the Trust’s clinical data marts, including patient
observations, admissions information, demographic data, and outcomes data, will be divided
into training and testing subsets. Using these data sets, we will train and test an algorithm that
optimises the variables and their weightings to predict the risk of key clinical outcomes,
including mortality, ITU admission, sepsis and cardiac arrest, to demonstrate a proof of concept
for a modified scoring system. By the end of this study in April 2025, we will have trained and
tested an algorithm based on 7 years of historical patient data. We will then test the new
system on data in other NHS Trusts across the UK to assess its generalisability. In 2026, we will
conduct implementation studies in Newcastle and then in other NHS Trusts to assess the
adoption of the system and gather initial data on patient outcomes to construct a learning
healthcare system to evaluate real-world performance of the algorithm and facilitate continual
improvement. By 2029, we aim to implement the system in NHS Trusts across the UK and
conduct large-scale trials to evaluate its effectiveness at improving patient outcomes.
Intended outputs: The aim of this project is to improve on the predictive accuracy of the
NEWS2 scoring system, particularly its accuracy over more than 24-hours, and its predictive
value in older patients and children. It will investigate whether using the currently collected
data differently and the inclusion of additional data would result in an improved algorithm.
There are two main ways in which this could improve patient outcomes: 1) Improving the
system’s ability to accurately predict patients who will deteriorate within and beyond 24-hours
would give healthcare professionals the opportunity to optimise their care and prevent
deterioration, and 2) reducing false positive alerts would allow care to be concentrated where it
will achieve the greatest benefit.
To be of national and international importance, the tool must meet the needs of patients
throughout their life-course. It is clearly important to examine the predictive accuracy of any
new early warning system in older people, but there is also limited outcome data for use of such
systems in children. On 3 November 2023, NHS England announced the roll-out of a national
Paediatric Early Warning System (PEWS) [25], and we will collaborate with the PEWS team in
Newcastle Hospitals to develop a new evidence-based clinical alerting system for deteriorating
children.
The tool must also be usable in the widest possible range of healthcare settings. As many
hospitals worldwide do not have a high degree of digital maturity [26,27] and rely on
paper-based documentation [28,29], we aim to develop a scalable paper- or smart-phone
application-based system that can achieve the highest possible predictive accuracy and clinical
value from an improved algorithm [30].
The overall aim is to incorporate an improved early warning system in an integrated Learning
Health System. This requires the technical capacity to routinely collect outcomes data and
provide structured opportunities to test and improve the algorithm for a changing population. It
also needs robust mechanisms for capturing and using clinical feedback on its implementation,
successes, and barriers in a wide range of care contexts [31].
 medRxiv preprint doi: https://doi.org/10.1101/2024.12.10.24318795; this version posted December 11, 2024. The copyright holder for this
preprint (which was not certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity.

It is made available under a CC-BY 4.0 International license .

3. Ethics and Dissemination

An application for the ethical approval of this study has been submitted to the UK The Health
Research Authority Integrated Research Approval System and is under consideration at the time
of manuscript submission. The study findings are expected to be produced in April 2025 and will
be disseminated symposia, conferences, clinical meetings and academic peer-reviewed
publications. PPIE members will help guide the dissemination of the results of the study to the
population through publicly accessible forums. Further funding applications will be prepared to
continue this programme of work by applying the new algorithm to other Trusts across the UK
and other hospital systems around the world, to test its generalisability and refine its
performance. Patient data will be stored only on secure NHS Trust systems. Other data available
to the research team will be stored securely on Newcastle University systems until the study is
complete. Analyses generated by the study will be available upon publication for 10 years from
the PI on reasonable request.

4. Conclusion
This work has identified potential areas of improvement to the NEWS2 system and outlined a
project for improving on its ability to predict adverse outcomes through the use of additional
patient data and/or changes to the scoring process. The project’s outputs will be shared widely
to facilitate collaboration between healthcare professionals, researchers, and policymakers to
support the implementation of improved clinical early warning systems in diverse populations.
 medRxiv preprint doi: https://doi.org/10.1101/2024.12.10.24318795; this version posted December 11, 2024. The copyright holder for this
preprint (which was not certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity.

It is made available under a CC-BY 4.0 International license .

Conflicts of Interest
The authors have no conflict of interest to report.

Abbreviations
NEWS = National Early Warning Score

Funding
This work was supported by the National Institute for Health and Care Research (NIHR)
Newcastle Biomedical Research Centre (BRC) based at the Newcastle upon Tyne Hospitals NHS
Foundation Trust, Newcastle University, and the Cumbria, Northumberland, and Tyne and Wear
(CNTW) NHS Foundation Trust. The views expressed in this publication are those of the authors
and not necessarily those of the NIHR or BRC, or any of the authors’ affiliated universities. The
open access publication fee was paid by the Newcastle BRC. The funding body was not involved
in the study design, data collection or analysis, or the writing and decision to submit the article
for publication.

CRediT author statement

Chris Pummer: Conceptualisation, Funding acquisition, Writing - review & editing

Madison Milne-Ives: Methodology, Writing - Original Draft
Cen Cong: Writing - Original Draft, Writing - Review & Editing
Lynsey Threlfall: Writing - review & editing
Edward Meinert: Conceptualisation, Supervision, Writing - review & editing
 medRxiv preprint doi: https://doi.org/10.1101/2024.12.10.24318795; this version posted December 11, 2024. The copyright holder for this
preprint (which was not certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity.

It is made available under a CC-BY 4.0 International license .

References
1 ​ Holland M, Kellett J. The United Kingdom’s National Early Warning Score: should everyone use it? A
narrative review. Intern Emerg Med. 2023;18:573–83. doi: 10.1007/s11739-022-03189-1

2 ​ Finnikin S, Wilke V. What’s behind the NEWS? National Early Warning Scores in primary care:
National Early Warning Scores in primary care. Br J Gen Pract. 2020;70:272–3. doi:
10.3399/bjgp20x709361

3 ​ Myrstad M, Ihle-Hansen H, Tveita AA, et al. National Early Warning Score 2 (NEWS2) on admission
predicts severe disease and in-hospital mortality from Covid-19 - a prospective cohort study. Scand J
Trauma Resusc Emerg Med. 2020;28:66. doi: 10.1186/s13049-020-00764-3

4 ​ Scott LJ, Redmond NM, Garrett J, et al. Distributions of the National Early Warning Score (NEWS)
across a healthcare system following a large-scale roll-out. Emerg Med J. 2019;36:287–92. doi:
10.1136/emermed-2018-208140

5 ​ Price C, Prytherch D, Kostakis I, et al. Evaluating the performance of the National Early Warning
Score in different diagnostic groups. Resuscitation. 2023;193:110032. doi:
10.1016/j.resuscitation.2023.110032

6 ​ Carr E, Bendayan R, Bean D, et al. Evaluation and improvement of the National Early Warning Score
(NEWS2) for COVID-19: a multi-hospital study. BMC Med. 2021;19:23. doi:
10.1186/s12916-020-01893-3

7 ​ Williams B. The National Early Warning Score: from concept to NHS implementation. Clin Med.
2022;22:499–505. doi: 10.7861/clinmed.2022-news-concept

8 ​ Holland M, Kellett J. A systematic review of the discrimination and absolute mortality predicted by
the National Early Warning Scores according to different cut-off values and prediction windows. Eur
J Intern Med. 2022;98:15–26. doi: 10.1016/j.ejim.2021.12.024

9 ​ Wei S, Xiong D, Wang J, et al. The accuracy of the National Early Warning Score 2 in predicting early
death in prehospital and emergency department settings: a systematic review and meta-analysis.
Annals of Translational Medicine. 2023;11. doi: 10.21037/atm-22-6587

10 ​ Mcneill H, Khairat S. Impact of Intensive Care Unit Readmissions on Patient Outcomes and the
Evaluation of the National Early Warning Score to Prevent Readmissions: Literature Review. JMIR
Perioperative Medicine. 2020;3:e13782. doi: 10.2196/13782

11 ​ Zhang K, Zhang X, Ding W, et al. The Prognostic Accuracy of National Early Warning Score 2 on
Predicting Clinical Deterioration for Patients With COVID-19: A Systematic Review and
Meta-Analysis. Front Med. 2021;8:699880. doi: 10.3389/fmed.2021.699880

12 ​ Clarke J, Gallifant J, Grant D, et al. Predictive value of the National Early Warning Score 2 for
hospitalised patients with viral respiratory illness is improved by the addition of inspired oxygen
fraction as a weighted variable. BMJ Open Respir Res. 2023;10:e001657. doi:
 medRxiv preprint doi: https://doi.org/10.1101/2024.12.10.24318795; this version posted December 11, 2024. The copyright holder for this
preprint (which was not certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity.

It is made available under a CC-BY 4.0 International license .

10.1136/bmjresp-2023-001657

13 ​ Clegg A, Young J, Iliffe S, et al. Frailty in Older People. Lancet. 2013;381:752. doi:
10.1016/S0140-6736(12)62167-9

14 ​ Vardy ER, Lasserson D, Barker RO, et al. NEWS2 and the older person. Clin Med . 2022;22:522–4.
doi: 10.7861/clinmed.2022-0426

15 ​ Kemp K, Alakare J, Harjola V-P, et al. National Early Warning Score 2 (NEWS2) and 3-level triage scale
as risk predictors in frail older adults in the emergency department. BMC Emerg Med. 2020;20:1–9.
doi: 10.1186/s12873-020-00379-y

16 ​ Sze S, Pan D, Williams CML, et al. Letter to the Editor: Variability but not admission or trends in
NEWS2 score predicts clinical outcome in elderly hospitalised patients with COVID-19. J Infect.
2021;82:159. doi: 10.1016/j.jinf.2020.05.063

17 ​ Pimentel MAF, Redfern OC, Gerry S, et al. A comparison of the ability of the National Early Warning
Score and the National Early Warning Score 2 to identify patients at risk of in-hospital mortality: A
multi-centre database study. Resuscitation. 2019;134:147–56. doi:
10.1016/j.resuscitation.2018.09.026

18 ​ Zaidi H, Bader-El-Den M, McNicholas J. Using the National Early Warning Score (NEWS/NEWS 2) in
different Intensive Care Units (ICUs) to predict the discharge location of patients. BMC Public
Health. 2019;19:1231. doi: 10.1186/s12889-019-7541-3

19 ​ Milne-Ives M, Riccalton V, Plummer C, et al. Modifications to the National Early Warning Score: a
scoping review protocol. BMJ Open. 2024;14:e089061. doi: 10.1136/bmjopen-2024-089061

20 ​ Nissen SK, Candel BGJ, Nickel CH, et al. The impact of age on predictive performance of National
Early Warning Score at arrival to emergency departments: Development and external validation.
Ann Emerg Med. 2022;79:354–63. doi: 10.1016/j.annemergmed.2021.09.434

21 ​ Forster S, McKeever TM, Churpek M, et al. Predicting outcome in acute respiratory admissions using
patterns of National Early Warning Scores. Clin Med. 2022;22:409–15. doi:
10.7861/clinmed.2022-0074

22 ​ Gonem S, Taylor A, Figueredo G, et al. Dynamic early warning scores for predicting clinical
deterioration in patients with respiratory disease. Respir Res. 2022;23:203. doi:
10.1186/s12931-022-02130-6

23 ​ Zhu Y, Chiu Y-D, Villar SS, et al. Dynamic individual vital sign trajectory early warning score
(DyniEWS) versus snapshot national early warning score (NEWS) for predicting postoperative
deterioration. Resuscitation. 2020;157:176–84. doi: 10.1016/j.resuscitation.2020.10.037

24 ​ Malycha J, Farajidavar N, Pimentel MAF, et al. The effect of fractional inspired oxygen concentration
on early warning score performance: A database analysis. Resuscitation. 2019;139:192–9. doi:
10.1016/j.resuscitation.2019.04.002

25 ​ England NHS. NHS England » New NHS warning system to spot deterioration in children’s health
faster.
 medRxiv preprint doi: https://doi.org/10.1101/2024.12.10.24318795; this version posted December 11, 2024. The copyright holder for this
preprint (which was not certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity.

It is made available under a CC-BY 4.0 International license .

https://www.england.nhs.uk/2023/11/new-nhs-warning-system-to-spot-deterioration-in-childrens-
health-faster/ (accessed 3 November 2023)

26 ​ Koebe P, Bohnet-Joschko S. What’s next in hospital digitization? A Delphi-based scenario study. Eur J
Futures Res. 2023;11:1–18. doi: 10.1186/s40309-023-00222-0

27 ​ Geissler A, Hollenbach J, Haring M, et al. A nationwide digital maturity assessment of hospitals –

Results from the German DigitalRadar. Health Policy Technol. 2024;13:100904. doi:
10.1016/j.hlpt.2024.100904

28 ​ Hufnagl C, Doctor E, Behrens L, et al. DIGITISATION ALONG THE PATIENT PATHWAY IN HOSPITALS.
Published Online First: 2019.

29 ​ Fröhlich D, Bittersohl C, Schroeder K, et al. Reliability of paper-based routine documentation in

psychiatric inpatient care and recommendations for further improvement. Front Psychiatry.
2019;10:954. doi: 10.3389/fpsyt.2019.00954

30 ​ Ma M-W, Gao X-S, Zhang Z-Y, et al. Extracting laboratory test information from paper-based reports.
BMC Med Inform Decis Mak. 2023;23:251. doi: 10.1186/s12911-023-02346-6

31 ​ Hardie T, Horton T, Thornton-Lee N, et al. Developing learning health systems in the UK: Priorities
for action. The Health Foundation 2022.

32​ ADMISSION Collaborative, Newcastle University https://research.ncl.ac.uk/admissioncollab/

33​ AI-MULTIPLY Newcastle University https://ai-multiply.co.uk/