skip to navigation | skip to content

The Clinical Excellence Commission The eChartbook


Between the Flags (BTF) Program For more information, click here
Improving Recognition and Response to Deteriorating Patients
For more information about this CEC program, click here
 


Why is this important?


Failure to recognise and respond to deteriorating patients is a significant issue, not only in New South Wales (NSW) public hospitals, but in hospitals and health care organisations around the world [1-5]. These serious adverse events are often preventable as there are documented warning signs of deterioration in up to 80 per cent of cases, which provide an opportunity for intervention [6],[ 7].


The high incidence of preventable adverse events and deaths in hospitals has triggered the development of Rapid Response Systems (RRS) designed to help clinicians recognise deterioration in their patients and enable them to initiate an appropriate response [8]. RRS are now widely used in hospitals around the world as patient safety net systems [9-12]. These systems have their origins in pioneering work at Liverpool Hospital in NSW, Australia in the early 1990's [13]. A number of studies have now reported the beneficial effect of RRS on inpatient mortality, cardiac arrests, unplanned intensive care unit (ICU) admissions and improved staff job satisfaction and culture [14-22]. There is evidence to suggest that there is a dose-response relationship between the number of Rapid Response calls and a reduction in mortality and other serious events such as cardiac arrests and unplanned admissions to ICU [22], [29].


In addition to reductions in adult mortality, a recent meta-analysis has shown reduction in paediatric and adult cardiac arrest rates, by approximately one third [23]. Compared to other clinical interventions (such as new drugs or procedures), this is a large improvement. No other similar interventions can demonstrate such large reductions in mortality or cardiac arrest rates.


In January 2010, NSW Health took the lead by introducing the Clinical Excellence Commission's (CEC's) statewide program called Between the Flags (BTF) [24], with the aim to: Improve early recognition and response to clinical deterioration and thereby reduce potentially preventable deaths and serious adverse events in patients who receive their care in NSW public hospitals.


Between the Flags was implemented in response to a recommendation from a major health review, the Garling Commission of Inquiry, which identified the CEC's program as an opportunity to improve recognition and response to deteriorating patients [25]. The system uses the analogy of Surf Life Saving Australia's lifeguards and lifesavers who keep swimmers safe by observing them and ensuring they don't venture into unsafe areas; and if they get into trouble, that rescue occurs rapidly.


BTF is the largest and most comprehensive system of its kind anywhere in the world. A five-element strategy was introduced in all NSW public hospitals, including governance structures, standardised calling criteria (incorporated in a suite of observation charts), Clinical Emergency Response Systems(including minimum standards for escalation), specially developed education materials and standard key performance indicators, which together provide a safety net for deteriorating patients.


BTF provides evidence in support of the Australian Commission on Safety and Quality in Health Care National Standard 9 - Recognising and Responding to Clinical Deterioration in Acute Health Care [26],[ 27].



References


[1] Institute of Medicine (IOM). To Err Is Human: Building a safer Health System: Committee on Quality of Health Care in America. Washington D.C.: National Academy Press, 1999.
[2] Wilson RM, Runciman WB, Gibberd RW, Harrison BT, Newby L, Hamilton JD. The Qualityin Australian Health Care study. Med J Aust 1995;163:458-71.
[3] Brennan TA, Leape LL, Laird NM, et al. Incidence of adverse events and negligence in hospitalised patients: results of the Harvard Medical Practice Study I. N Engl J Med 1991; 324: 370-77.
[4]Davis P, Lay-Yee R, Briant R, et al. Adverse events in New Zealand public hospitals I: occurrence and impact N Z Med J 2002; 115: 271.
[5] Baker GR, Norton PG, Flintoft V, et al. The Canadian Adverse Events Study: the incidence of adverse events among hospital patients in Canada.CMAJ 2004; 170: 1678-86.
[6] Buist MD, Jarmolowski E, Burton PR, et al: Recognising clinical instability in hospital patients before cardiac arrest or unplanned admission to intensive care. A pilot study in a tertiary-care hospital. Med Journal of Australia 1999; 171:22-25.
[7] Hodgetts TJ et al. The identification of risk factors for cardiac arrest and formulation of activation criteria to alert a medical emergency team, Resuscitation 2002; 54:125-131
[8] DeVita MA, Bellomo R, Hillman K, Kellum J, Rotondi A, Teres D, et al. Findings of the first consensus conference on medical emergency teams. Crit Care Med. 2006;34(9):2463-78. [4] Hillman K, Chen J, Brown D. A Clinical Model for Health Services Research - The Medical Emergency Team. J Crit Care 2003;18(3):195-99.
[9] Ball C, Kirby K, Williams S. Effect of the critical care outreach team on patient survival to discharge from hospital and readmission to critical care: non-randomised population based study. BMJ. 2003 November 1; 327(7422): 1014.
[10]England K, Bion JF. Introduction of medical emergency teams in Australia and New Zealand: a multicentre study. Crit Care 2008;12:151.
[11] Steel AC, Reynolds SF. The growth of rapid response systems. Jt Comm J Qual Patient Saf 2008;34:489-95.
[12] Winters BD, Pham J, Pronovost PJ. Rapid response teams - walk, don't run. JAMA 2006;296:1645-7.
[13] Lee A, Bishop G, Hillman KM, Daffurn K. The medical emergency team. Anaesth Intensive Care 1995; 23: 183-186.
[14] Buist MD, Moore GE, Bernard SA, Waxman BP, Anderson JN, Nguyen TV. Effects of a medical emergency team on reduction of incidence of and mortality from unexpected cardiac arrests in hospital: preliminary study. BMJ 2002;324(7334):387-90.
[15] Cretikos MA, Chen J, Hillman KM, Bellomo R, Finfer SR, Flabouris A. The effectiveness of implementation of the medical emergency team (MET) system and factors associated with use during the MERIT study Crit Care Resusc. 2007;9(2):206-12.
[16] Dacey MJ, Mirza ER, Wilcox V, Doherty M, Mello J, Boyer A, et al. The effect of a rapid response team on major clinical outcome measures in a community hospital. Crit Care Med 2007;35(9):2076-82.
[17] Daly ML, Powers J, Orto V, Rogers M, Dickinson T, Fabris M, et al. Innovative solutions: leading the way: an innovative approach to support nurses on general care units with an early nursing intervention team Dimens. Crit Care Nurs. 2007;26(1):15-20.
[18] Galhotra S, DeVita MA, Simmons RL, Dew MA. Mature rapid response system and potentially avoidable cardio-pulmonary arrests in hospital Qual.Saf Health Care 2007;16(4):260-65.
[19] Jones D, Egi M, Bellomo R, GoldsmithD. Effect of the medical emergency team on long-term mortality following major surgery Crit Care 2007;11(1):R12.
[20] McFarlan SJ, Hensley S. Implementation and outcomes of a rapid response team. J Nurs.Care Qual 2007;22(4):307-13, quiz.
[21] Hillman K, ChenJ, Cretikos M, Brown D, Bellomo R, Doig G, et al. Introduction of medical emergency team (MET) system - a cluster-randomised controlled trial. Lancet 2005;365:2091-97.
[22] Chen J, Bellomo R, Flabouris A, Hillman K, Finfer S. The relationship between early emergency team calls and serious adverse events. Crit Care Med 2009;37(1):148-53.
[23] Chan PS, Jain R, Nallmothu BK, Berg RA, Sasson C. Rapid response teams: a systematic review and meta-analysis. Arch Intern Med 2010;170:18-26.
[24] Hughes C, Pain C, Braithwaite J, et al. BMJQual Saf Published Online First: 16 April 2014 doi:10.1136/bmjqs-2014-002845
[25] Garling P. First Report of the Special Commission of Inquiry into Acute Care Services in NSW Public Hospitals. http://healthactionplan.nsw.gov.au/garling-report.php, 2008
[26]Australian Commission on Safety and Quality in Health Care (2010). National Consensus Statement: Essential Elements for Recognising and Responding to Clinical Deterioration, ACSQHC, Sydney www.safetyandquality.gov.au/ourwork/recognition-and-response-to-clinical-deterioration/the-national-consensusstatement/
[27] Australian Commission on Safety and Quality in Health Care (September 2011). National Safety and Quality Health Service Standards, ACSQHC, Sydney www.safetyandquality.gov.au/publications/national-safety-and-quality-health-servicestandards/
[28] Clinical Excellence Commission (CEC) 2013. Safer Systems Better Care - Quality Systems Assessment Statewide Report 2012. Sydney: CEC
[29] Jones, D & Bellomo, R. 'Introduction of a rapid response system: why we are glad we MET', Critical Care 2006,10:121.


Findings


This data shows a significant inverse relationship between the Rapid Response call rate and the cardiac arrest rate in NSW hospitals following the introduction of the BTF program in Aug 2010 (Charts BF01 and BF03). Since its introduction to Dec 2016, the Rapid Response rate has increased by 156 per cent and the cardiac arrest rate has decreased by 52 per cent in NSW compared to the baseline period. These are both statistically significant (p < 0.01). This supports the premise that implementation of its five elements, including a consistent approach to the Clinical Emergency Response Systems, has had an impact on cardiac arrests in NSW hospitals.


Note:

1). In late 2013, the Between the Flags Standard Adult General Observation (SAGO) Charts were implemented in hospitals across South Eastern Sydney Local Health District (SESLHD). SESLHD decided to use a two tiered response to the Red Zone criteria on the charts. This resulted in a marked increase in Rapid Response calls for SESLHD from the periods Jul–Dec 2013 onwards. This marked increase can also be seen in the St Vincent’s Hospital Network data from July 2014 for similar reasons.


2). The two key performance indicator (KPI) definitions were updated in 2013/14 to outline the requirements to report Rapid Response calls and cardiac arrest rates separately for Adult, Paediatric and Maternity patients. In addition, greater emphasis was placed on the exclusion criteria of calls made in the Emergency Departments. When interpreting any change in Rapid Response call or cardiac arrest rates in 2013/14, this should be taken into account.


3). In some smaller facilities (e.g. multipurpose service, peer group F3) there are times that the numerator of the key performance indicators are larger than the denominator. This occurs when there were no acute separations (denominator) at the facility but there were still patients in the facility who had a Rapid Response or cardiac arrest call (numerator) during that month and therefore a denominator of 0 and a numerator greater than 0 is recorded.


Implications


There is now encouraging evidence that Between the Flags is achieving its aim of reducing the harm to patients from a failure to recognise and respond to deterioration. There is also good evidence that Between the Flags is strongly supported by staff who believe it is having a positive impact on patient safety [28]. However, the improvements demonstrated are likely to be attributable to a range of strategies that the NSW public health system has put in place to improve patient safety, and not just BTF.


The BTF program has further highlighted the barriers, challenges and obstacles to escalation that exist in the NSW public health system, as evidenced by the CEC's 2012 Quality Systems Assessment (QSA) report [28] which has identified the following barriers: team feel situation is under control and escalation not required; staff failure to recognise; not wanting to 'bother' doctors; and, staff not knowing when to escalate. Efforts to address these barriers, challenges and obstacles area focus for improvement to patient safety in NSW. There are significant opportunities to build on the successes of this program and others and address some of these challenges.



References


[28] Clinical Excellence Commission (CEC) 2013. Safer Systems Better Care - Quality Systems Assessment Statewide Report 2012. Sydney: CEC


What we don't know


The current literature does not tell us what the optimal Rapid Response rate should be for individual or different kinds of hospitals. A target of > 20 Rapid Response calls per 1,000 admissions has been set as a benchmark across the State. Jones and Bellomo, however, report that the Rapid Response rate stabilised at 40 calls per 1,000 admissions in their study [29] . It is likely that a rate of about this level is necessary for an effective system.


We also don't know which elements of the program are most important in improving recognition and response to deteriorating patients. Research being conducted at the University of NSW aims to identify which elements are most effective.


At this stage, it has proven difficult to collect data from Clinical Reviews*, because they are quite often an informal process. We don't know what impact the Clinical Review rate will have on the Rapid Response rate, nor do we know the impact of the hospital's culture on the effectiveness of the program's implementation.


Note: * Clinical Review definition from the policy directive PD2013_049 Recognition and Response to patients who are clinically deteriorating “A patient review undertaken within 30 minutes by the attending medical team, or designated responder, as defined in the local Clinical Emergency Response System".



References


[29] Jones, D & Bellomo, R. 'Introduction of a rapid response system: why we are glad we MET', Critical Care 2006, 10:121.

Chart BF01 - Deteriorating Patients - Rapid Response calls
 

Monthly Rapid Response call rate per 1,000 acute separations*, NSW, Aug 2010 - Apr 2017


eChartbook

Source: NSW Ministry of Health, Clinical Excellence Commission.

* Public hospitals only. # Aug-Dec 2010, 5 months of Rapid Response call data.



Chart BF02 - Deteriorating Patients - Rapid Response calls
 
Six-monthly Rapid Response call rate per 1,000 acute separations* by LHD/SN, NSW, Jul 2012 - Dec 2016

eChartbook

Source: NSW Ministry of Health, Clinical Excellence Commission.

* Public hospitals only.



Chart BF02a - Deteriorating Patients - Rapid Response calls
 
Annual Rapid Response call rate per 1,000 acute separations* by LHD/SN, NSW, 2013 - 2016

eChartbook

Source: NSW Ministry of Health, Clinical Excellence Commission.

* Public hospitals only.



Chart BF02b - Deteriorating Patients - Rapid Response calls
 
Six-monthly number and rate (per 1,000 acute separations*) of Rapid Response calls by hospital peer group,
Jan 2015 - Dec 2016


eChartbook

Source: NSW Ministry of Health, Clinical Excellence Commission. * Public hospitals only.

Note: The NSW Hospital Peer Groups used in the analysis are based on the NSW Peer Hospital Groups 2011/12 definitions. The table shows Number of Rapid Response calls / Number of acute separations, by peer group and time period.



Chart BF03 - Deteriorating Patients - Cardiopulmonary arrest rate
 
Monthly unexpected cardiopulmonary arrest rate per 1,000 acute separations*, NSW,
Aug 2010 - Apr 2017

eChartbook

Source: NSW Ministry of Health, Clinical Excellence Commission.

*Public hospitals only. # Aug-Dec 2010, 5 months of unexpected cardiopulmonary arrest data.



Chart BF04 - Deteriorating Patients - Cardiopulmonary arrest rate
 
Six-monthly unexpected cardiopulmonary arrest rate per 1,000 acute separations* by LHD/SN & NSW,
Jul 2012 - Dec 2016

eChartbook

Source: NSW Ministry of Health, Clinical Excellence Commission.

* Public hospitals only.



Chart BF04a - Deteriorating Patients - Cardiopulmonary arrest rate
 
Annual unexpected cardiopulmonary arrest rate per 1,000 acute separations* by LHD/SN, 2013 - 2016

eChartbook

Source: NSW Ministry of Health, Clinical Excellence Commission.

*Public hospitals only.



Chart BF04b - Deteriorating Patients – Cardiopulmonary arrest rate
 

Six-monthly number and rate (per 1,000 acute separations*) of unexpected cardiopulmonary arrests by hospital peer group,
Jan 2015 - Dec 2016


eChartbook

Source: NSW Ministry of Health, Clinical Excellence Commission.  *Public hospitals only.

Note: The NSW Hospital Peer Groups used in the analysis are based on the NSW Peer Hospital Groups 2011/12 definitions. The table shows Number of unexpected cardiopulmonary arrests / Number of acute separations, by peer group and time period.



Chart BF02d - Deteriorating Patients - Rapid Response calls
 

Rapid Response call rate per 1,000 acute separations* by LHD/SN, NSW, Aug 2010 - Dec 2016
(dynamic chart: default view)

Source: NSW Ministry of Health, Clinical Excellence Commission.

*Public hospitals only.



Chart BF04d - Deteriorating Patients - Cardiopulmonary arrest rate
 

Six-monthly unexpected cardiopulmonary arrest rate per 1,000 acute separations* by LHD/SN, Aug 2010 - Dec 2016
(dynamic chart: default view)

Source: NSW Ministry of Health, Clinical Excellence Commission.

*Public hospitals only.


End Matter


Contributors
Drafted by: CEC Between the Flags team
Data analysis by: CEC eChartbook team
Reviewed by: CEC Between the Flags team
Edited by: CEC eChartbook team


Suggested citation
Clinical Excellence Commission (access year). eChartbook Portal: Safety and Quality of Healthcare in New South Wales. Sydney: Clinical Excellence Commission. Available at: http://www.cec.health.nsw.gov.au/echartbook/cec-indicators-intro-chartbook/btf Accessed (insert date of access).


© Clinical Excellence Commission 2017
This work is copyright. Apart from any use as permitted under the Copyright Act 1968, no part may be reproduced without prior written permission from the Clinical Excellence Commission (CEC). Requests and enquiries concerning reproduction and rights should be directed to the Director, Information Management, Locked Bag 8, Haymarket, NSW 1240.


Evidence-base for this initiative


Reported elsewhere
Monthly NSW Ministry of Health (MoH) Performance Report


Data Definitions


Chart: BF01, BF02 and BF02a

Admin Status: Current, Apr 2017

Indicator Name: Deteriorating Patients - Rapid Response calls

Description: Rapid Response call rate per 1,000 acute separations (public hospitals only) by LHD/SN and NSW, Aug 2010 - Apr 2017

Dimension: Access to acute care

Clinical Area: The effectiveness of Clinical Emergency Response Systems in NSW hospitals

Data Inclusions: All acute admitted patients, including recovery (adults: 16 years and over, paediatrics: children 16 years and under, maternity: women in maternity services) in NSW public hospitals

Data Exclusions: Non-admitted patients, residential aged care patients, patients in operating theatre and ICU

Numerator: The number of Rapid Response calls made for patients with criteria within the Red Zone as outlined on the appropriate NSW Health Standard Observation chart. This includes both advanced life support/cardio-respiratory arrest calls.

Denominator: Acute separations counted as stays, not episodes

Standardisation: None (crude rate per 1,000 acute separations was calculated)

Data Source: Clinical Excellence Commission and NSW Ministry of Health (NSW public health care facilities). Power chart - Rapid Response Data Collection; Paper-based Rapid Response Record Form; Switchboard Rapid Response activation record.

Comments: The optimum Rapid Response calling rate is currently unknown. There is evidence, however, that a higher rate of calls to the Rapid Response system is desirable and associated with better outcomes. This is because a higher call rate may indicate that patients who are clinically deteriorating are being identified and reviewed promptly. Initially, as the program matures, it is expected that the Rapid Response rate would increase. As LHDs work to embed the Clinical Review (Yellow Zone) process, however, it would be expected that clinical deterioration would be treated earlier and this would lead to a reduced requirement for Rapid Response.
 
Chart: BF02b

Admin Status: Current, Dec 2016

Indicator Name: Deteriorating Patients - Rapid Response calls

Description: Six-monthly Rapid Response call rate per 1,000 acute separations (public hospitals only) by Hospital peer group, Jan 2015 - Dec 2016

Dimension: Access to acute care

Clinical Area: The effectiveness of Clinical Emergency Response Systems in NSW hospitals

Data Inclusions: All acute admitted patients, including recovery (adults: 16 years and over, paediatrics: children 16 years and under, maternity: women in maternity services) in NSW public hospitals

Data Exclusions: Non-admitted patients, residential aged care patients, patients in operating theatre and ICU

Numerator: The number of Rapid Response calls made for patients with criteria within the Red Zone as outlined on the appropriate NSW Health Standard Observation chart. This includes both advanced life support/cardio-respiratory arrest calls.

Denominator: Acute separations counted as stays, not episodes

Standardisation: None (crude rate per 1,000 acute separations was calculated)

Data Source: Clinical Excellence Commission and NSW Ministry of Health (NSW public health care facilities). Power chart - Rapid Response Data Collection; Paper-based Rapid Response Record Form; Switchboard Rapid Response activation record.

Comments: 1. The optimum Rapid Response calling rate is currently unknown. There is evidence, however, that a higher rate of calls to the Rapid Response system is desirable and associated with better outcomes. This is because a higher call rate may indicate that patients who are clinically deteriorating are being identified and reviewed promptly. Initially, as the program matures, it is expected that the Rapid Response rate would increase. As LHDs work to embed the Clinical Review (Yellow Zone) process, however, it would be expected that clinical deterioration would be treated earlier and this would lead to a reduced requirement for Rapid Response.

2. Peer Group definition used by ACSQHC when displaying BTF data in their evaluation of Standard 9

A1 = principal referral hospital.
A2 = paediatric specialist hospital.
A3 = tertiary referral hospital. B1 – major hospital with between 17 000-35 000 acute weighted separations (AWS) and specialist services.
B2 = major hospital with 10 000-35 000 AWS and no specialist services.
C1 – district group (4000-10 000 AWS)
C2 district group (2000-4000 AWS)
D1a community with surgery (200-2000)
D1b – community without surgery
F3 – multipurpose including aged care
F4 – subacute care

Source: NSW Ministry of Health, Clinical Excellence Commission. Public hospitals only. Note: The peer groups used in the analysis are based on the NSW Peer Hospital Groups 2011/12 definitions.
 
Chart: BF03, BF04 and BF04a

Admin Status: Current, Apr 2017

Indicator Name: Deteriorating Patients - Unexpected cardiopulmonary arrest rate

Description: Unexpected cardiopulmonary arrest rate per 1,000 acute separations (public hospitals only) by LHD/SN and NSW, Aug 2010 - Apr 2017

Dimension: Access to acute care

Clinical Area: To provide an outcome measure of the effectiveness of the Between the Flags program

Data Inclusions: All acute admitted patients, including recovery (adults: 16 years and over, paediatrics: children 16 years and under, maternity: women in maternity services) in NSW public hospitals

Data Exclusions: Non-admitted patients, residential aged care patients, patients in operating theatre and ICU

Numerator: The number of patients who have experienced a cardio-respiratory arrest without a documented Not for Resuscitation/Not for CPR/Allow a Natural Death order

Denominator: Acute separations counted as stays, not episodes

Standardisation: None (crude rate per 1,000 acute separations was calculated)

Data Source: Clinical Excellence Commission and NSW Ministry of Health (NSW public health care facilities). Power chart - Rapid Response Data Collection; Paper-based Rapid Response Record Form; Switchboard Rapid Response activation record.

Comments: The desired outcome for the BTF program is fewer instances of cardio-respiratory arrest through earlier recognition and response to clinical deterioration. Cardio-respiratory arrest is defined as the absence of pulse or respiratory effort and unconsciousness, necessitating the start of cardio-pulmonary resuscitation.
 
Chart: BF04b

Admin Status: Current, Dec 2016

Indicator Name: Deteriorating Patients - Unexpected cardiopulmonary arrest rate

Description: Six-monthly unexpected cardiopulmonary arrest rate per 1,000 acute separations (public hospitals only) by Hospital peer group, Jan 2015 - Dec 2016

Dimension: Access to acute care

Clinical Area: To provide an outcome measure of the effectiveness of the Between the Flags program

Data Inclusions: All acute admitted patients, including recovery (adults: 16 years and over, paediatrics: children 16 years and under, maternity: women in maternity services) in NSW public hospitals

Data Exclusions: Non-admitted patients, residential aged care patients, patients in operating theatre and ICU

Numerator: The number of patients who have experienced a cardio-respiratory arrest without a documented Not for Resuscitation/Not for CPR/Allow a Natural Death order

Denominator: Acute separations counted as stays, not episodes

Standardisation: None (crude rate per 1,000 acute separations was calculated)

Data Source: Clinical Excellence Commission and NSW Ministry of Health (NSW public health care facilities). Power chart - Rapid Response Data Collection; Paper-based Rapid Response Record Form; Switchboard Rapid Response activation record.

Comments: 1. The desired outcome for the BTF program is fewer instances of cardio-respiratory arrest through earlier recognition and response to clinical deterioration. Cardio-respiratory arrest is defined as the absence of pulse or respiratory effort and unconsciousness, necessitating the start of cardio-pulmonary resuscitation.

2. Peer Group definition used by ACSQHC when displaying BTF data in their evaluation of Standard 9

A1 = principal referral hospital.
A2 = paediatric specialist hospital.
A3 = tertiary referral hospital. B1 – major hospital with between 17 000-35 000 acute weighted separations (AWS) and specialist services.
B2 = major hospital with 10 000-35 000 AWS and no specialist services.
C1 – district group (4000-10 000 AWS)
C2 district group (2000-4000 AWS)
D1a community with surgery (200-2000)
D1b – community without surgery
F3 – multipurpose including aged care
F4 – subacute care

Source: NSW Ministry of Health, Clinical Excellence Commission. Public hospitals only.
Note: The peer groups used in the analysis are based on the NSW Peer Hospital Groups 2011/12 definitions.