This review explores a selection of existing methods that measure ambulance service performance and identifies a performance framework that could be useful in a variety of contextual settings. It drew on a selection of well-known performance frameworks currently in use within emergency, health and ambulance systems. It is recommended that ambulance authorities adopt a health services performance framework in preference to the widely used emergency services framework with its heavy emphasis on time intervals such as response times. This suggested framework should be able to accommodate a wide range of validated clinical and non-clinical performance indicators that better reflect the position of ambulance services as part of the health system.
References
1.
BaragwanathCA, Metropolitan Ambulance Service: Fulfilling a vital community need. 1997, Special Report No. 50, Auditor-General of Victoria.
2.
BaragwanathCA, Victorian Rural Ambulance Services: Fulfilling a vital community need. 1997, Special Report No. 51, Auditor-General of Victoria.
3.
Audit Office of New South Wales, Performance Audit Report: Ambulance Service of New South Wales: readiness to respond. 2001, The Audit Office of New South Wales.
4.
McCoyJ, ‘Ambulance Service Victoria, A partnership to achieve quality performance’. Australian Journal of Emergency Care. 1996; Vol 3, No 3, pp. 30–31.
5.
GuppyL, and WoollardM, ‘Emergency ambulance services: performance management and review’. Pre-hospital Immediate Care. 2000; Vol 4, pp. 40–45.
6.
ShapiroSE, ‘Outcomes of prehospital care: do we really make a difference?’Journal of Emergency Nursing. 2000; Vol 26, No 3, pp. 239–241.
7.
SoboEJ, AndrieseS, StroupC, MorganD, and KurtinP, ‘Developing indicators for emergency medical services (EMS) system evaluation and quality improvement’. Joint Commission Journal on Quality Improvement. 2001; Vol 27, No 3, pp. 138–154.
8.
BarrettB, and GulyHR, ‘How accurate are the times held by the ambulance control computer’. Pre-hospital Immediate Care. 1999; Vol 3, pp. 206–208.
9.
CallahamM, ‘Quantifying the scanty science of prehospital emergency care’. Ann Emerg Med December 1997; Vol 30, pp. 785–790.
10.
MarcumL, and CallahamM, ‘The not-so-scientific value of EMS’. Ann Emerg Med, 1998; Vol 32, No 5, pp. 634–636.
11.
BrazierH, MurphyAW, LynchC, and BuryG, ‘Searching for the evidence in pre-hospital care: a review of randomised controlled trials. On behalf of the Ambulance Response Time Sub-Group of the National Ambulance Advisory Committee’. Journal of Accident and Emergency Medicine. 1999, Vol 16, No 1, pp. 18–23.
12.
BriceJH, GarrisonHG, and EvansAT, ‘Study design and outcomes in out-of-hospital emergency medicine research: a ten-year analysis’. Prehospital Emergency Care. 2000; Vol 4, pp. 144–150.
13.
SpaiteDW, CrissEA, ValenzuelaTD, and HarveyW, ‘Developing a Foundation for the Evaluation of Expanded-Scope EMS: A Window of Opportunity That Cannot Be Ignored’. Ann Emerg Med, 1997; Vol 30, No 6. pp. 791–796.
14.
SpaiteDW, ValenzuelaTD, MeislinHW, CrissEA, HinsbergP, ‘Prospective validation of a new model for evaluation of EMS systems by in-field observation of specific time intervals in prehospital care’. Ann Emerg Med1993; Vol 22, pp. 638–645.
15.
SpaiteDW, MaioR, GarrisonHG, DesmondJS, GregorMA, StiellIG, CaytenCG, ChewJLJr, MacKenzieEJ, MillerDR, and O'MalleyPJ, ‘Emergency Medical Services Outcomes Project (EMSOP) II: developing the foundation and conceptual models for out-of-hospital outcomes research’. Ann Emerg Med.2001; Vol 37, pp. 657–663.
16.
MooreL, ‘Measuring quality and effectiveness of prehospital EMS’. Prehospital Emergency Care. 1999; Vol 3, No 4, pp. 325–31.
17.
McCleanSA, MaioRF, SpaiteDW, and GarrisonMD, ‘Emergency medical services outcome research: evaluating the effectiveness of prehospital care’. Prehospital Emergency Care. 2002; Vol 6, No 2, pp. S52–56.
18.
MacFarlaneC, and BennCA, ‘Evaluation of emergency medical services systems: a classification to assist in determination of indicators’. Emerg Med J. 2003 Vol 20, pp. 188–191.
19.
CarringtonDJ, ‘Review of emergency ambulance performance standards’. Care of the Critically Ill. 1997; Vol 13, No 1, pp. 19–20.
20.
MaioRF, GarrisonHG, SpaiteDW, DesmondJS, GregorMA, CaytenCG, ChewJLJr, HillEM, JoyceSM, MacKenzieEJ, MillerDR, O'MalleyPJ, StiellIG, ‘Emergency Medical Services Outcomes Project I (EMSOP I): Prioritizing conditions for outcomes research’. Ann Emerg Med April 1999; Vol 33, pp. 423–432.
21.
MaioRF, GarrisonHG, SpaiteDW, DesmondJS, GregorMA, StiellIG, CaytenCG, ChewJLJr, MacKenzieEJ, MillerDR, and O'MalleyPJ. ‘Emergency Medical Services Outcomes Project (EMSOP) IV: pain measurement in out-of-hospital outcomes research’. Ann Emerg Med. August 2002; Vol 40, pp. 172–179.
22.
GarrisonHG, MaioRF, SpaiteDW, DesmondJS, GregorMA, O'MalleyPJ, StiellIG, CaytenCG, ChewJLJr, MacKenzieEJ, and MillerDR. ‘Emergency Medical Services Outcomes Project III (EMSOP III): the role of risk adjustment in out-of-hospital outcomes research’. Ann Emerg Med. July 2002; Vol 40, pp. 79–88.
23.
DuckettS, ‘Policy challenges for the Australian health care system’. Australian Health Review. 1999; Vol 22, No 2, pp. 130–147.
24.
AdayLA, BegleyCE, LairsonDR, and SlaterCH, Evaluating the Healthcare System: Effectiveness, Efficiency and Equity. AHSR, Chicago, 1998.
25.
SheldonT, ‘Promoting health care quality: what role performance indicators?’Quality in Health Care. 1998; Vol 7, Supplement, pp. S45–50.
26.
HancockL, (Ed.) Health Policy in the Market State. Allen & Unwin, Sydney, 1999.
27.
National Health Performance Committee, Measuring performance in the Australian health system: towards a national performance assessment framework. Discussion Paper. Sydney, 2000.
28.
CSIRO Mathematical and Information Sciences. ‘Measuring service performance tells the SA Ambulance Service if it is effective’. 2001, www.cmis.csiro.au/opm/CaseStudies/SAAS.htm [accessed 05/06/03].
29.
National Health Ministers‘ Benchmarking Working Group, Second National report on health sector performance indicators. Australian Health Ministers’ Conference, 1998.
30.
HumphreysJS, and Mathews-CoweyS, Models of Health Service Delivery for Small Rural and Remote Communities. La Trobe University, Bendigo, 1999.
31.
SandersonI, ‘Evaluation, policy learning and evidence-based policy making’. Public Administration. 2002; Vol 80, No 1, pp. 1–22.
32.
DohertyJ, and PriceM, ‘Evaluation of rural ambulance services.’World Health Forum. 1998; Vol 19, No 3, pp. 315–319.
33.
O'MearaP, ‘Professional and community expectations of rural ambulance services in Australia’, Pre-hospital Immediate Care. 2001; Vol 5, No 1, pp. 27–30.
34.
SpaiteDW, MaioR, GarrisonHG, DesmondJS, GregorMA, StiellIG, CaytenCG, ChewJL, MacKenzieEJ, MillerDR, and O'MalleyPJ, ‘Emergency Medical Services Outcomes Project (EMSOP) II: Developing the Foundation and Conceptual Models for Out-of-Hospital Outcomes Research’. Ann Emerg Med, 1999; Vol 37, No 6, pp. 657–663.
35.
StiellIG, ‘Cardiac Arrest in Your Community: Are There Weak Links in the Chain of Survival?’Canadian Medical Association Journal. 1993; Vol 149, No 5, pp. 563–565.
36.
StiellIG, HebertPC, WellsGA, LaupacisA, VandemheemK, DreyerJF, EisenhauerMA, GibsonJ, HigginsonLA, KirbyAS, MahonJL, MaloneyJP, and WeitzmanBN, ‘The Ontario trial of active compression-decompression cardiopulmonary resusitation for in-hospital and prehospital cardiac arrest’. JAMA. 1996; Vol 275, No 18, pp. 1417–1423.
37.
StiellIG, WellsGA, SpaiteDW, LyverMB, MunkleyDP, FieldBJ, DagnoneE, MaloneyJP, JonesGR, LiuinstraLG, JermynBD, WardR, and DeMaioV, ‘The Ontario Prehospital Advanced Life Support (OPALS) Study: Rationale and Methodolgy for Cardiac Arrest Patients’. Annals of Emergency Medicine. 1998; Vol 32, No 2, pp. 180–190.
38.
StiellIG, WellsGA, DeMaioVJ, SpaiteDW, FieldBJ3rd, MunkleyDP, LyverMB, LiuinstraLG, and WardR, ‘Modifiable factors associated with improved cardiac arrest survival in a multicenter basic life support/defibrillation system: OPALS Study Phase I results. Ontario Prehospital Advanced Life Support’. Annals of Emergency Medicine. 1999; Vol 33, No 1, pp. 44–50.
39.
StiellIG, WellsGA, FieldBJ, SpaiteDW, DeMaioVJ, WardR, MunkleyDP, LyverMB, LiuinstraLG, CampeauT, MaloneyJ, and DagnoneE, ‘Improved out-of-hospital cardiac arrest survival through the inexpensive optimization of an existing defibrillation program: OPALS Study Phase II. Ontario Prehospital Advanced Life Support’. JAMA. 1999; Vol 281, No 13, pp. 1175–1181.
40.
Steering Committee for the Review of Commonwealth/State Service Provision, Report on Government Services Volume 1, 2003. AusInfo, Canberra.
41.
CadiganRT, and BugarinCE, ‘Predicting Demand for Emergency Ambulance Service’, Ann Emerg Med, 1989; Vol 18, pp. 618–621.
42.
JacobsIG, and OxerHF, ‘A review of pre-hospital defibrillation by ambulance officers in Perth, Western Australia’, Medical Journal of Australia, Dec 1990; Vol 153, No 11–12, pp. 662–664.
43.
ClarkMJ, PurdieJG, and FitzgeraldG, ‘Determinants of pre-hospital care non-usage for patients with emergency care needs’. Pre-hospital Immediate Care. 2000; Vol 4, pp. 90–96.
44.
MorriseyMA, OhsfeldtRL, JohnsonV, and TreatR, ‘Rural Emergency Medical Services: patients, destinations, times and services’. J. Rural Health. 1995; Vol 11, No 4, pp. 286–294.
45.
ConeDC, DavidsonSJ, and NquyenQ, ‘A Time-Motion Study of the Emergency Medical Services Turnaround Interval’Annals of Emergency Medicine, 1998; Vol 31, No 2, pp. 241–246.
46.
StiellIG, HebertPC, WellsGA, LaupacisA, VandemheemK, DreyerJF, EisenhauerMA, GibsonJ, HigginsonLA, KirbyAS, MahonJL, MaloneyJP, and WeitzmanBN, ‘The Ontario trial of active compression-decompression cardiopulmonary resuscitation for in-hospital and prehospital cardiac arrest’. JAMA. 1996; Vol 275, No 18, pp. 1417–1423.
47.
RobinsonJS, DaviesMK, JohnsBM, and EdwardsSN, ‘Out-of-hospital cardiac arrests treated by the West Midlands Ambulance Service over a 2-year period’. European Journal of Anaesthesiology. 1998; Vol 15, No 6, pp. 702–709.
48.
Health Office Circular HSC (IS)67/WHSC(IS)57.Organisation of Ambulance Services: Standards and Incentive Bonus Schemes. Department of Health and Social Security/Welsh Office, 1974.
49.
Department of Health and Social Security – UK. Review of Ambulance Services Standards, Emergency and Urgent Targets: Final Report. 1990.
50.
NichollJ, TurnerJ, and MartinD, The Future of Ambulance Services in the United Kingdom. Ambulance Service Association, 2001.
51.
DunfordJ, DomeierRM, BlackwellT, MearsG, OvertonJ, Rivera-RiveraEJ, and SworR, ‘Performance Measurements in Emergency Medical Services.’Prehospital Emergency Care, 2002; Vol 6, No 1, pp. 92–98.
52.
ErichJ, ‘The State of EMS: “Three simple digits” face complex challenges’. The Journal of Emergency Care, Rescue and Transportation. 2001, www.emsmagazine.com/issues/state.html [accessed 21/05/2001].
53.
SethiD, KwanI, KellyAM, RobertsI, and BunnF, ‘Advanced trauma life support training for ambulance crews (Cochrane Review)’. The Cochrane Library, Issue 2, 2003.
54.
TippettV, ClarkM, WoodsS, and FitzgeraldG, ‘Towards a national research agenda for the ambulance and pre-hospital sector in Australia’. Journal of Emergency Primary Health Care. 2003; Vol 1, Issue 1–2.
55.
MahonyK, ‘Management and the creation of occupational stressors in an Australian and a UK Ambulance Service’. Australian Health Review. 2001; Vol 24, No 4, pp. 135–145.
56.
BledsoeBE, ‘EMS mythology. EMS myth #7. System status management (SSM) lowers response times and enhances patient care’. Emerg Med Serv. 2003; Vol 32, No 9, pp. 158–9.
57.
TurnerJ, and NicholJ, The Performance of Staffordshire Ambulance Service. A Review, Medical Care Research Unit of the University of Sheffield. 2002.
58.
PepePE, ‘Controversies in the prehospital management of major trauma’. Emerg Med, 2000; Vol 12, pp. 180–189.
59.
JacobsI, ‘Prehospital care: A plea for more research’. Emergency Medicine. 2000; Vol 12, pp. 175–176.
60.
SayreMR, WhiteLJ, BrownLH, and McHenrySD, ‘The National EMS Research Agenda Executive Summary’. Ann Emerg Med, 2002; Vol 40, No 6, pp. 636–643.
61.
WoollardM, LeavesS, ChartersK, PittK, BottellJ, and DonnellyP, ‘Learning the alphabeta: developing a research capability in the Welsh Ambulance Service’. Pre-hospital Immediate Care. 2000; Vol 4, pp. 100–101.
62.
DaviesHTO, and NutleySM, ‘The Rise and Rise of Evidence in Health Care’, Public Money & Management, January-March 1999; pp. 9–16.
63.
O'MearaP, Models of Ambulance Service Delivery for Rural Victoria. PhD Thesis, UNSW, 2002.
