Skip to main content

Research Repository

Advanced Search

Development, implementation and evaluation of an early warning system improvement programme for children in hospital: the PUMA mixed-methods study

Allen, Davina; Lloyd, Amy; Edwards, Dawn; Grant, Aimee; Hood, Kerenza; Huang, Chao; Hughes, Jacqueline; Jacob, Nina; Lacy, David; Moriarty, Yvonne; Oliver, Alison; Preston, Jennifer; Sefton, Gerri; Skone, Richard; Strange, Heather; Taiyari, Khadijeh; Thomas-Jones, Emma; Trubey, Robert; Tume, Lyvonne; Powell, Colin; Roland, Damian


Davina Allen

Amy Lloyd

Dawn Edwards

Aimee Grant

Kerenza Hood

Jacqueline Hughes

Nina Jacob

David Lacy

Yvonne Moriarty

Alison Oliver

Jennifer Preston

Gerri Sefton

Richard Skone

Heather Strange

Khadijeh Taiyari

Emma Thomas-Jones

Robert Trubey

Lyvonne Tume

Colin Powell

Damian Roland


Objectives: (1) Identify, through systematic review, the following: evidence for core components of effective paediatric track-and-trigger tools and paediatric early warning systems, and contextual factors consequential for paediatric track-and-trigger tool and early warning system effectiveness. (2) Develop and implement an evidence-based paediatric early warning system improvement programme (i.e. the PUMA programme). (3) Evaluate the effectiveness of the PUMA programme by examining clinical practice and core outcomes trends. (4) Identify ingredients of successful implementation of the PUMA programme. Review methods: The quantitative reviews addressed the following two questions: how well validated are existing paediatric track-and-trigger tools and their component parts for predicting inpatient deterioration? How effective are paediatric early warning systems (with or without a tool) at reducing mortality and critical events? The qualitative review addressed the following question: what sociomaterial and contextual factors are associated with successful or unsuccessful paediatric early warning systems (with or without tools)?. Design: Interrupted time series and ethnographic case studies were used to evaluate the PUMA programme. Qualitative methods were deployed in a process evaluation. Setting: The study was set in two district general and two tertiary children’s hospitals. Intervention: The PUMA programme is a paediatric early warning system improvement programme designed to harness local expertise to implement contextually appropriate interventions. Main outcome measures: The primary outcome was a composite metric, representing children who experienced one of the following in 1 month: mortality, cardiac arrest, respiratory arrest, unplanned admission to a paediatric intensive care unit or unplanned admission to a high-dependency unit. Paediatric early warning system changes were assessed through ethnographic ward case studies. Results: The reviews showed limited effectiveness of paediatric track-and-trigger tools in isolation, and multiple failure points in paediatric early warning systems. All sites made paediatric early warning system changes; some of the clearer quantitative findings appeared to relate to qualitative observations. Systems changed in response to wider contextual factors. Limitations: Low event rates made quantitative outcome measures challenging. Implementation was not a one-shot event, creating challenges for the interrupted time series in conceptualising ‘implementation’ and ‘post-intervention’ periods. Conclusions: Detecting and acting on deterioration in the acute hospital setting requires a whole-systems approach. The PUMA programme offers a framework to support ongoing system-improvement work; the approach could be used more widely. Organisational-level system change can affect clinical outcomes positively. Alternative outcome measures are required for research and quality improvement. Future work: The following further research is recommended: a consensus study to identify upstream indicators of paediatric early warning system performance; an evaluation of OUTCOME approach in other clinical areas; an evaluation of supernumerary nurse co-ordinator role; and an evaluation of mandated system improvement.


Allen, D., Lloyd, A., Edwards, D., Grant, A., Hood, K., Huang, C., …Roland, D. (2022). Development, implementation and evaluation of an early warning system improvement programme for children in hospital: the PUMA mixed-methods study. Health and Social Care Delivery Research, 10(1), 307.

Journal Article Type Article
Acceptance Date Jan 1, 2022
Publication Date Jan 24, 2022
Deposit Date Apr 4, 2022
Publicly Available Date Oct 27, 2022
Journal Health and Social Care Delivery Research
Print ISSN 2755-0060
Electronic ISSN 2755-0079
Publisher NIHR Journals Library
Peer Reviewed Peer Reviewed
Volume 10
Issue 1
Pages 307
Public URL


Published article (24.9 Mb)

Publisher Licence URL

Copyright Statement
Copyright © 2022 Allen et al. This work was produced by Allen et al. under the terms of a commissioning contract issued by the Secretary of State for Health and Social Care. This is an Open Access publication distributed under the terms of the Creative Commons Attribution CC BY 4.0 licence, which permits unrestricted use, distribution, reproduction and adaption in any medium and for any purpose provided that it is properly attributed. See: For attribution the title, original author(s), the publication source – NIHR Journals Library, and the DOI of the publication must be cited.

You might also like

Downloadable Citations