Applied Measurement and Variation Workshop
This workshop will give participants practical experience with the key tools for understanding safety and quality data. The session will be interactive where participants will teach and learn from each other. They will also need to bring a laptop (with QI Macros loaded) to enable them to practice graphing data in Excel.
At the end of the workshop, participants will be able to:
- Select appropriate data collection tools for examining data for improvement
- Create charts using Excel and/or statistical process control software
- Analyse the charts in terms of improvement approach.
This workshop will build on the knowledge learnt at;
- CEC monthly Webinar/PEXIP Virtual Education Offering on Measurement for Improvement: Run Charts, Pareto Charts & Histograms.
- CEC Improvement Science two day Workshop
- CEC Introduction to Measurement for Improvement one day Workshop
Dates of program
The program is typically offered four times per year. For workshop dates, please view the CEC's Upcoming Events page
Mode of delivery and duration
Face to Face from 9am until 4:30pm – see hour by hour program below.
To have completed at least one of these education offerings:
- Listen to 1 hour PEXIP / Webinar (CEC Virtual Education Offering) on "Introduction Measurment for Improvement: Run & Pareto Charts and Histograms." See CEC Website Events page for dates/times of this 1 hour WebEx
- CEC Improvement Science 2 day Workshop
- CEC Introduction to Measurement for Improvement 1 day Workshop
It is recommended that participants refresh their understanding of the Quality Tools on the CEC Quality Improvement Academy Website, in particular:
- Model for Improvement & PDSA
- Basic Statistics
- Driver Diagrams
- Pareto Charts & the 80/20 Rule
- Run Charts
- Control Charts
- Scatter Diagrams
OPTIONAL: Also consider listening to the short video series on Improvement Science to understand the context and importance of measurement in quality improvement. Videos on the CEC YouTube Channel here. Or these videos can be viewed via HETI My Health Learning – Course name: Improvement Science – HETI Course Code: 161814499 (best via Chrome)
It would also be advantageous to view these IHI videos prior to the workshop:
- Family of Measures: Outcome, Process & Balancing Measures ~8 mins
- Run Chart - Part 1 ~7 mins
- Run Chart - Part 2 ~6 mins
- Using Run & Control Charts to understand Variation by Robert Lloyd ~56 mins
Pre-Workshop Homework (basic chart development in Excel): Before the workshop we expect you to have completed some pre-workshop charts in Excel. The data for these charts is contained in this Excel Spreadsheet download. Please create these charts in Excel and bring this completed file with you to the workshop (on USB).
Use the data in the attached Excel file to make:
- 2 Run charts
- 1 Histogram
- 1 Pareto chart
Software that enables the creation of Statistical Process Control (SPC) charts is essential for utilising skills learnt in this program. There are many software options available at varying levels of sophistication and pricing.
The Clinical Excellence Commission does not endorse or support any particular software. Appendix 6 of the Health Care Data Guide (2011) by Provost and Murray provides a list of characteristics to consider when Purchasing SPC Software.
Program participants' organisations may already have selected a preferred software choice and be able to provide licences. It is required that participants have a commitment from their organisation to purchase or provide a licence for the participant for SPC software prior to commencing the workshop.
At the workshop itself, QI Macros, an add‐in to Excel designed for use by non‐statisticians, will be required to be installed, even if the participants are using other software in their organisations. This will facilitate a focus on the creation and understanding of charts rather than differences in technology.
In order to maximise learning from the practical sessions at the workshop, it is essential that participants have appropriate skills in using Excel. In particular, participants should be able to:
- Prepare data (clean, format and arrange data for analysis)
- Click and select continuous and non–continuous cells
- Utilise formulas to analysis data (such as =SUM(), =SUMIF(), =COUNT(), =COUNTIF(), =COUNTIFS(), addition, subtraction, multiplication, division)
- Copy and paste formulas, data (including how to 'fix' cells included within the formula)
- Transpose rows to columns and vice versa
Resources to learn these skills are readily available by searching the internet. CEC also has a simple Excel file with examples of these formulas. It is a good idea to bring an EXTERNAL mouse as there will be a lot of cutting and pasting.
Participants are to bring along a laptop (and power cord) to work from and install two (2) simple software prior to the workshop (if using a work Laptop please see your work IT Administrator to install the QI Macros software).
1. Install on your Laptop Excel Data Analysis Tool-Pak
Excel has functions which enable creation of many of the most common charts for examining data for improvement. Some of these require activation of the Data Analysis add‐in available in the program:
a. Open Excel, go to 'File' – then 'Options'
b. Select 'Add‐ins' on the left
c. At the bottom of the window at 'Manage', make sure 'Excel add‐ins' is selected, click 'Go'
d. Tick "Analysis tool‐pak" and click 'OK'
e. You will now have a new set of tools available under the Data tab in Excel: Data Analysis (enable add in)
2. Install on your Laptop QI Macros
QI Macros software is needed to run some charts at the session:
a. Access a free 30‐day trial (make sure this will not expire before the session!). You will need an administrator to install it for you if you are using a work laptop.
b. You need to ensure that the correct control chart rules are applied – those used by the Institute for Healthcare Improvement (IHI): go to 'QI Macros' tab, select 'Control Charts (SPC)' then 'Control Chart' rules and then 'Healthcare‐IHI'
c. Once installed, the tools are available on the QI Macros tab. You should check that this tab is visible BEFORE the workshop. Troubleshooting for different versions of Excel is available online.
Learning about measurement and variation is stronger when participants are able to apply skills to their own data. Participants are encouraged to bring along real data relating to their organisation and/or their own work. This should include at least one set of data over time – this may be incidents, waiting times, budgets, audit results, by day, week, month, quarter (aim for at least 12 separate data points). If you have denominator data (occupied bed days, number of surgeries, etc) bring those too.
Evaluation of program
Participants will be asked to give feedback at the conclusion of the program via a short survey. It is an expectation that all participants complete the course evaluation survey. Participants will also be expected to complete a Pre-Post knowledge questionnaire.
It is expected that participants will participate fully in the practical sessions and be fully engaged.
Certificates will be issued to participants who complete the program.
Participation in this program will qualify participants for continuing professional development hours, as required by their respective National Board.
- Free for NSW Health Staff
- Non-NSW Health staff $660
How to register
Go to the CEC Events Calendar, select your workshop date and complete the registration form.
Checklist for participants
- Register before close date
- Complete all pre-reading and watch all videos
- Load software onto laptop a few days prior to workshop and test the software
- Do Pre-workshop Charts (bring completed charts on USB)
- On day of workshop bring: Laptop (with QI Macros loaded), power cord, mouse, pen & paper and USB of Excel homework.
Senior Manager, Knowledge, Evaluation and Research
B.Pharm, MBA (Health Management), Cert IV TAE, MSHP, MPS
Evette joined the Clinical Excellence Commission in 2012 establishing the Quality Use of Antimicrobials in Healthcare Program, and represents NSW on the National Antimicrobial Stewardship Network. She is a pharmacist with a long standing interest in infectious diseases and pharmacy education and training.
Evette has experience in the community and hospital sector, where she has held positions in clinical, management and educator roles. Evette led the statewide antimicrobial stewardship program for 6 years and is an associate investigator for the National Centre for Antimicrobial Stewardship. Since completing the IHI Improvement Advisor Training Program in 2017, Evette has worked as an improvement advisor on two large CEC Collaboratives.
Senior Manager, Mental Health Patient Safety Program
BHealth (Health Service Management), Diploma Management, Diploma Business, FISQua, IA
Kimberley's passion for quality and safety in healthcare began early in her career leading health projects and programs in government and non-government organisations.
In 2005 Kimberley joined the NSW Clinical Excellence Commission leading the inaugural NSW Hand Hygiene Campaign and Between the Flags Program.
Between March 2014 and January 2018, Kimberley was a key member of the Senior Executive for South Eastern Sydney Local Health District, Mental Health Service, leading the Service Development and Innovation portfolio. In this role, she was responsible for Service Development, Innovation, and Improvement Science, and led the SESLHD Mental Health Patient Safety Program to reduce harm to consumers and increase the capability of staff to build a culture of improvement locally in partnership with consumers.
Kimberley embarked on the International Society for Quality in Health Care (ISQua) Fellowship Programme receiving Fellowship in January 2017, and she undertook the Improvement Advisor Program at the Institute for Healthcare Improvement (IHI) in Boston, graduating in September 2017.
In January 2018, Kimberley joined the Clinical Excellence Commission as Improvement Advisor, supporting the NSW Falls in Hospitals Collaborative, and supporting the IHI Improvement Advisor Program participants, and National Perineal Tears Collaborative. In November 2018, she was appointed Senior Manager for the NSW Mental Health Patient Safety Program.
Senior Manager, Adult Patient Safety
RN, BN, MN (Critical Care)
Following the completion of a Master of Nursing in Critical Care, Malcolm went on to a variety of roles in Intensive Care across the NSW Public Health System and NHS England.
eHealth, Acting Senior Manager - Safety & Quality, Clinical Engagement and Patient Safety
B App Sci (MRA), Masters Sci (Applied Statistics), FISQua
Wendy has over twenty-five years experience in quality management in healthcare with leadership roles in Victoria and NSW.
In 2009, she joined the CEC as a member of the Quality Systems Assessment (QSA) team, developing and managing the QSA On-site Visit and Verification Program.
In March 2015, Wendy was appointed as the Senior Manager of the CEC's Quality Improvement Academy where she managed the Improvement Science and Clinical Leadership Programs with over 400 participant enrolments each year.
In October, 2020 Wendy was seconded to the NSW eHealth Clinical Engagement and Patient Safety team as their Senior Manager of Safety & Quality.
Wendy is a RABQSA-AU & BSBAUD402B Certified Auditor and successfully completed the ISQua Fellowship Program in June 2014, the CEC Executive Clinical Leadership Program in August 2014 and the USA Institute for Healthcare Improvement (IHI) Improvement Advisor Program in February, 2019. Wendy is passionate about improving the quality and safety of healthcare for current and future generations.
Information from the following resources and websites were used to create this workshop:
- Lloyd Provost & Sandra Murray, "The Health Care Data Guide - Learning from Data for Improvement", Published 2011 by Jossey-Bass, USA.
- Langley G, Moen R, Nolan K, Nolan T, Norman C, Provost L, "The Improvement Guide, A Practical approach to Enhancing Organisational Performance", 2nd Edition. Published 2009 Jossey-Bass, USA.
- Neviene Torki, "Statistical Techniques - Process Improvement - The Link", Published 1992.
- David Moore & George McCabe, "Introduction to the Practice of Statistics" - Second Edition, Published 1993.
- Institute for Healthcare Improvement www.ihi.org – particularly resources from the IHI Improvement Advisor Program (Wave 53 in 2018).
- NSW Health GEM Workstar – Clinical Practice Improvement module.
- Clinical support systems program - What is clinical practice improvement? By R. M. Wilson and B. T. Harrison from QaRNS, Royal North Shore Hospital, Sydney, New South Wales, Australia. Internal Medicine Journal 2002; 32: pages 460–464.
- Dr Brent James – Clinical Practice Improvement Program (slides from 2000 program run by NSW Health Department).
- Easy Guide to Clinical Practice Improvement – A guide for healthcare professionals - NSW Health Department (published October 2002).
- CEC Clinician's Guide to Quality & Safety published by the Clinical Excellence Commission in 2016.
- CEC Masters Clinician's Guide to Quality & Safety published by the Clinical Excellence Commission in 2018.
- CEC QARS & QIDS programs
- UK Healthcare Quality Improvement Partnership (HQIP) website - What is clinical audit?
- UK Clinical Audit Support Centre
- Healthcare Improvement Scotland website – ihub
- Clinical Excellence Commission QI Academy web site