Is there a problem that needs fixing? CEC masterclass on using data for improvement
7 November 2023
On 26 October eminent statistician and Senior Improvement Advisor at the Institute for Healthcare Improvement (IHI) Lloyd Provost led a Clinical Excellence Commission masterclass, "Variation and Data for Improvement".
The Clinical Excellence Commission Chief Executive Adj Prof Michael C. Nicholl introduced Lloyd and opened the masterclass in quoting W. Edwards Deming, one of the founders of data for improvement science, "Without data you're just a person with an opinion".
Leaders in patient safety and quality improvement from across NSW Health attended to fine tune their skills using data to improve healthcare systems and processes, with the goal to provide more reliable and safe healthcare.
Spotting an issue with healthcare data
When looking at examples of variations in healthcare data, we ask: is what we see just normal fluctuation or has something significantly changed? For example, we could consider the daily average length of stay of a patient in an Emergency Department (ED). Are the day-to-day changes due to random differences in patients that present, overcrowding at peak times or seasonal variation in some types of presentations, or are they due to a significant event like a recent change to the physical layout, or changes to test requests and availability used in the ED?
Mr Provost explained how we use the Shewhart chart of a data set to tell the difference between common cause variation and special cause variation - when something has significantly changed.
Julianne Jones is QIDS MatIQ (Quality Improvement Data System Maternity Safety Intelligence System) Midwifery Lead at the CEC.
"This has given me a lot of clarity and enhanced my understanding of how Shewhart charts can help define the variation in the process. I can now better explain to clinicians how we can highlight special causes in data variation," she said.
Julianne works with and trains clinicians and clinical governance teams across the 15 NSW Local Health Districts that offer maternity services to use the CEC's maternity data intelligence system, QIDS MatIQ. The system automatically creates the Shewhart charts and identifies when maternity and neonatal data is indicating that there may be an issue that maternity teams are encouraged to investigate.
Steve Bowden, Associate Director Patient Safety at the CEC, said the masterclass was a valuable opportunity to explore opportunities for improvement with Mr Provost.
"If we can improve our knowledge and experience with data, we may be able to incite curiosity, learning and appropriate action," he said.
Data used for improvement is different to data for accountability
Goodhart's rule says when a measure (used for improvement) becomes a target, it ceases to be a good measure.
Lloyd Provost talked about how we can miss important improvement opportunities if we don't use data specially selected for this use. Data classified in commonly used traffic light visualisations – using red, yellow and green – for accountability may not show trends important for patient safety and improvement.
"If we use data for accountability, and we are not looking to improve, we are missing an opportunity." said Malcolm Green, Acting Director Safety Quality and Governance at Sydney Children's Hospitals Network – an attendee at the masterclass.
Malcolm echoed the opinion of many at the masterclass.
"The masterclass was an opportunity to learn from an absolute expert in the field," he said.
CEC Director Capability and Culture Karen Patterson, Professor Lloyd Provost, CEC Chief Executive Adj Professor Michael Nicholl.