Home News Archive Publications Conferences Latest Evidence Contact Details
Background Search/ Site Map Presentations Guidelines Regional Groups Links

Regional Activity

Each regional group will report their current clinical effectiveness activity on these pages.

Wales
Last updated 18/06/01

Emergency Ambulance Services: Performance Management And Review

Guppy, L., Woollard, M. Emergency Ambulance Services: Performance Management And Review. Pre-hospital Immediate Care. 2000;4:40-45. Reproduced with the permission of BMJ Publishing Group.

This is copyrighted and must not be reproduced without permission of the authors.  

Authors

Lawrence Guppy, Welsh Ambulance Services NHS Trust, South East Region

Malcolm Woollard, Pre-hospital Emergency Research Unit, Welsh Ambulance Services NHS Trust / University of Wales College of Medicine

Corresponding Author:

Malcolm Woollard
Pre-hospital Emergency Research Unit
Lansdowne Hospital
Sanatorium Road
Cardiff
CF1 8UL
Tel: 02920 233651 ext. 2930
Fax:  02920 237930
Email: [email protected]

Emergency Ambulance Services: Performance Management And Review

Introduction

This discussion paper attempts to identify the necessary developments required in simple performance management techniques to meet the challenges of the new ambulance response standards.[1],[2],[3] In this respect “performance management” encompasses the development of appropriate indicators and targets, the implementation of robust measurement systems and the devolution of management responsibilities to those officers who are best placed to effect the necessary improvements. This paper is primarily concerned with time management, logistics and organisational matters. It does not seek to be a primer in sophisticated system status management or ‘high performance’ techniques, but rather focuses on simple strategies that do not require the availability of advanced information or command and control systems, although the authors hope that services using these techniques would ultimately aim to introduce these more advanced methods. The paper also does not attempt to deal with the measurement of clinical skills but recognises their immense importance in terms of overall performance monitoring.

It is clear that additional funding alone will not automatically ensure that the new ambulance response standards will be achieved. It is of equal, if not greater, importance that current resources are organised in such a way to ensure that the best possible performance and value for money is provided for the users and purchasers of our service.[4]

There are several references throughout this paper to various performance levels of an existing service. These examples, although largely based on factual data, have been anonymised but are used to demonstrate or develop a particular point.

The critical success factors in moving towards the achievement of the new standards are, in the author’s view, as follows:

  • Management of each component of the EMS (999 call) episode
  • Development of demand/performance tables
  • Introduction of effective deployment plans (System Status Management)
  • Implementation of new working patterns

It is recognised that all of the above cannot be achieved within a short space of time. The introduction of new working patterns, for example, will require considerable negotiation. However, it can hopefully be demonstrated that much can be done in the short term to produce immediate improvements. In terms of information requirements it is not a pre-requisite for state of the art command and control systems to be available, however desirable they may be, to make inroads into the above. All the facts and statistics in this document have been compiled from the collection and analysis of a dozen or so data fields.

Management of the EMS Episode

For the purposes of this paper the EMS episode is described as the period from which the call is received to the time the crew becomes available (clear). The following diagram identifies the relevant time periods within the episode.

           
   
     
 


A

B C D E F G
       
   

A = Call processing time
B = Control allocation time
C = Crew mobilisation time
D = Travelling time to scene
E = Time spent on scene
F = Travelling time to hospital
G = Time spent at hospital

Figure 1: Components of the EMS Episode

The response time forms only part of the episode and cannot be managed as a single entity. It is, however, the result of three factors, B + C + D, which can very much be managed and improved. The following sections deal with each of the time periods which make up the episode and give reasons for the importance of their monitoring. The tables which follow each section suggest targets, where applicable, and the level of manager who should be responsible for their achievement. Recommendations for information analysis and reporting periods are also made. The targets suggested can be amended to suit individual organisations but it is, however, of vital importance to overall performance that they are both challenging and achievable.

Time Period A – Call Processing Time

Time period A is the time between the receipt of call and the patient’s condition and location being confirmed. For the purposes of this paper it does not include the ring time between the exchange contacting the ambulance control and the call being answered. It is however proposed that this additional period be monitored and managed as “ring time performance”. Under the new government reporting arrangements time period A will not form part of the response time. However, this period is still patient critical and should receive the same attention as if it were a constituent part of the response time. Given that there are skill factors involved in this event and that performance will vary by individual call takers, it is essential that this period is monitored and subject to performance review. This will highlight additional training needs of those members of staff whose performance falls below acceptable levels.

Target 1 Location/condition confirmed in 45 sec for 50% of calls
Target 2 Location/condition confirmed in 60 sec for 95% of calls
Responsible Control team leader
Information Analysed by individual call taker and by team
Reporting Weekly and monthly

Table 1: Performance management recommendations for ‘call time.’

Time Period B – Control Allocation Time

This time period is the first element of the response time and it is essential that this period is monitored and managed to ensure a fighting chance is given to meeting overall performance targets. Once again skills will vary between individual allocators and the following is proposed.

Target 1 50% of calls allocated within 15 sec
Target 2 95% of calls allocated within 30 sec
Responsible Control team leader
Information Analysed by individual allocator and by team
Reporting Weekly and monthly

Table 2: Performance management recommendations for ‘control allocation time.’

Time Period C – Crew Mobilisation Time

Perhaps the most dramatic improvement to the overall response time can be made via the monitoring and management of crew mobilisation times. By way of example these arrangements were piloted in one division of the authors’ ambulance service and have shown the degree of improvement that can be made.

The following graph charts the mobilisation times and the corresponding Orcon results in this division for three separate one month periods, i.e. the beginning of the monitoring period, the midway point and the latest results.

Figure 2:Impact of Managing Crew Mobilisation Times on Orcon Performance.

These mobilisation times have been analysed by crew member, by station and by shift pattern (i.e. time of day). Results have been fed back to individual staff to correct deficiencies in performance and to recognise excellence. The results have also been used to highlight problems at particular stations – for example electrically raised doors have been fitted along with a remote control panel positioned next to the emergency phone. Improvements in mobilisation times are a major factor in improving Orcon performance.

Target 1 50% of calls mobilised within 30 sec
Target 2 95% of calls mobilised within 60 sec
Responsible Local operations manager (e.g. divisional ambulance officer / station manager)
Information Analysed by crew member, by shift and by station
Reporting Weekly and monthly

Table 3: Performance management recommendations for ‘mobilisation times.’

Time Period D – Travelling Time to Scene

The management of travelling time to scene poses different problems to the other response time elements. A degree of sensitivity must be used to ensure that a culture is not developed whereby the individual crew member takes any unnecessary driving risks. For this reason it is not recommended that times by crew member are compiled.

However, broader information on travelling times is necessary to measure the effectiveness of deployment plans and the variances in performance between Unitary Authorities or other reporting areas. The following is therefore recommended.

Target 1 70% within 5 minutes
Target 2 95% within 15 minutes
Responsible System status manager
Information Analysed by time of day, day of week and location
Reporting Weekly and monthly.

Table 4: Performance management recommendations for ‘travelling time to scene.’

Time Period E – Scene Time

This time period is clearly more about quality of service and clinical effectiveness than anything else. However, in terms of the data collected it is interesting to note how much variance there is by crew member and by time of day. For example, the latest monitoring period for one division of out Trust showed a range in average on scene times between individual crew members from 8 minutes to 17 minutes. Clear protocols need to be defined in terms of optimum scene times for varying clinical conditions. These can then be measured and monitored accordingly. A uniform target may be inappropriate but exception reports should be produced for scene times which are obviously greater than the norm.

Target 1 Exception reports for scene times in excess of 30 mins
Target 2 N/A
Responsible Clinical auditor
Information Analysed by crew member, by shift and by station
Reporting Weekly and monthly

Table 5: Performance management recommendations for ‘scene time.’

Time Period F – Travelling Time to Hospital

This time period will obviously vary greatly according to area and patient condition. It is recommended that this period be monitored for the purposes of exception reporting only and that targets are therefore inappropriate.

Target 1 Exception reports for travelling times in excess of 30 mins (adjusted by area)
Target 2 N/A
Responsible Clinical auditor
Information Analysed by crew member, by shift and by station
Reporting Weekly and monthly

Table 6: Performance management recommendations for ‘travelling time to hospital.’

Time Period G – Time Spent at Hospital

The analysis of crew time at hospital can be expected to show wide variances between individual hospitals, crew members, and at different times of day. The graph below shows an example of the average time spent at various hospitals between the period June to August 1998.

Figure 3: Average time spent at various hospitals

The challenges here are to understand the reason for the significant differences in times between, say, hospitals D and A. This time period is a significant drain on resources. In one division alone we estimated that if hospital turnaround times were reduced by five minutes this would release the equivalent of one additional ambulance per shift. The impact of time spent at hospitals on performance is clearly significant. The following is therefore recommended.

Target 1 50% less than 10 minutes
Target 2 95% less than 15 minutes
Responsible Local operations manager (e.g. divisional ambulance officer / station manager)
Information Analysed by crew member, by shift and by station
Reporting Weekly and monthly

Table 7: Performance management recommendations for ‘time spent at hospital.’

Demand and Performance Tables

In order to capitalise on improvements in response times achieved through performance management, it is necessary to develop ‘demand and performance’ tables. Quite simply these measure performance by geographical areas that are considerably smaller than those of either Health Authorities or Unitary Authorities. Typically Health Authority areas should be further divided into thirty or forty sub areas.

Analysis at this tighter geographic level is vitally important to be able to target resources to those areas with high demand but low performance. The tables below demonstrate the result of having performance aligned with demand and the subsequent effect this has on overall response times. In this example an imaginary unitary authority has been divided into nine sub areas. The Orcon performance for these sub areas ranges between 30% to 80% of responses within 8 minutes. Because demand is aligned with performance, the overall result for the unitary authority is 63%.

Location

No of Responses

% in 8 Mins Demand Position Performance Position
Area A

400

80

1

1

Area B

300

70

2

2

Area C

200

60

3

3

Area D

100

55

4

4

Area E

90

50

5

5

Area F

80

45

6

6

Area G

70

40

7

7

Area H

60

35

8

8

Area I

50

30

9

9

Table 8: The impact of aligning performance with demand.

Conversely the following table below shows performance in reverse to demand. The overall level here drops to 42%.

Location

No of Responses

% in 8 Mins Demand Position Performance Position
Area A

400

30

1

9

Area B

300

35

2

8

Area C

200

40

3

7

Area D

100

45

4

6

Area E

90

50

5

5

Area F

80

55

6

4

Area G

70

60

7

3

Area H

60

70

8

2

Area I

50

80

9

1

Table 9: The impact of performance as the inverse of demand.

The use of this type of table is key to the preparation of effective deployment plans. Given that ambulance services have finite resources the appropriate approach must be to provide the greatest good to the greatest number of people.

It is not, however, suggested that areas of low demand are in any way unimportant. The very fact that performance is measured in low demand areas provides the opportunity to make improvements in service to these locations. The development and use of these tables provides the vital information required in order to determine those areas (which may well be of low demand) which are suitable for the introduction of first responder schemes. Being able to predict the call volume and the variances between time of day and day of week will ensure that such schemes can be set up effectively. Once first responder schemes are introduced minimum levels of performance can be set for all areas to ensure equity.

Development of a ‘real’ demand versus performance table for one of our divisions led to us securing funding and support for the opening of a new ambulance station. The following results show the before and after Orcon performance position.

 

Demand Position

Performance Position

% in 8 Minutes

Before new station

2

27

24

After new station opened

2

8

64

Table 10: The impact of opening a new ambulance station on Orcon performance.

Not surprisingly this new station has made a substantial contribution to the overall improvement in the performance figures for this division. Equally unsurprisingly, everyone ‘knew’ that this station was needed. However it was only on presentation of a demand / performance table to the Health Authority that they finally agreed (after twenty years of ignoring our requests) to fund this initiative.

It could be argued that rather than build a new ambulance station the service could have simply utilised a standby point at the same location and minimised cost. This would have indeed fulfilled the same objective in terms of an increase in performance. Many services in the USA have minimised the number of ambulance stations they utilise, preferring instead to have a small number of very large ‘garages’ from which the majority of vehicles are deployed at the beginning of their shift. A small number of UK services have also implemented this model, the aim of which is cost minimisation (by reducing capital spend and subsequent revenue consequences) and to ensure ease of flexibility in changing the deployment of resources as demand patterns change over time.

This model appears to work well in urban areas but deserves further analysis. For example, in some high performance ambulance services in the USA it has been recognised that the relatively low call frequency at certain standby points makes it unfair to ask crews to remain in their ambulance for protracted periods of time, and so facilities are provided such as a rented flat or a room in commercial premises. In rural areas the distance needed to travel from a single central ambulance station to appropriate standby points makes this concept unfeasible in both economic and performance terms.

In the case of the new ambulance station discussed here, performance data clearly identified the need for a resource to be positioned at this location on a regular basis. However whilst the call volume for this area is high relative to the rest of the division, call frequency is low. A crew based on standby at this point may, on occasion, have to wait hours to be assigned to a call. It was felt appropriate, therefore, to provide facilities to ensure adequate comfort. On examination of what was available in the required area we discovered a small industrial estate with a unit available to rent. Very little alteration was required to the premises and we were able to secure a short lease with the option to renew. This ensured the cost to the Trust was minimised – particularly as the Health Authority was prepared to fund the revenue costs as a result of the evidence provided to them. Flexibility in deployment has been maintained, as when demand dictates crews still go out to standby at other locations, and should demand patterns change over a longer time scale we have flexibility in our lease, allowing us to move to a more suitable alternative location.

Performance management is not a proscriptive religion with only one set of solutions. Rather it should be seen as a menu from which various options may be selected to suit local circumstances, although the general principles will always hold. Additionally staff must be treated in a humane manner or industrial relations will inevitably suffer, and this of itself can have a significant negative effect on performance. High staff turnover due to poor morale is a feature of at least some high performance ambulance services in the USA, and lessons must be learned if we are to conserve our most precious resource – our staff.

Demand / performance tables are also of assistance in determining appropriate staffing levels for ambulance control rooms. Call answering time can be plotted against call volume by time of day. Clearly one would expect that low periods of demand would be associated with excellent call answering times, but it is also obvious that poor performance at times of high demand will result in a bigger impact on overall performance. Demand / performance tables can identify when staffing levels should be increased to maximise performance, and when staffing levels can be reduced without adversely impacting call answering times to support this.

Such an analysis of supply (staffing levels) against demand (workload) is also of critical relevance to designing rosters and systems status plans which determine the pre-planned availability of staffed ambulances (see below). Call volumes vary by time as well as by location, and detailed analysis of such variations and the subsequent development of responsive strategies is key to improving performance.

Demand varies by time of day, by day of week, and by season. For example the total workload for emergency vehicles in many ambulance services will at its highest on Mondays, between the hours of 11.00 and 14.00 hrs. This is predicated not by 999 call volume but by urgent requests from G.P.s. Many patients apparently prefer not to bother their doctor at the weekend, and this accounts for the ‘Monday’ peak. The time period for the peak is seen every day (Monday to Friday) because many G.P.s will be making house calls during this period and subsequently identifying patients who require ambulance transfer to hospital.

The pattern of demand for 999 calls, when analysed in isolation, is different. It is not surprising that almost all services will see a peak in 999 call volumes between 22.00 and 24.00 hours as pubs close and the effects of alcohol come to the fore. It is also not surprising that this peak tends to be higher on Fridays and Saturdays and at its lowest on Mondays.

Seasonal variations in demand vary based on the topography and demographic make up of the area served. In the author’s service, in urban areas demand starts to rise in late October, peaks in December, and declines in March. In coastal holiday areas, however, the situation is reversed and demand increases significantly in the summer months.

A temporal analysis of demand cannot be separated from a contemporaneous geographical analysis . Variations in demand by time of day and day of week will themselves be subject to variability in their patterns for various locations. For example, call volumes are likely to be relatively higher in residential areas after working hours and at weekends. Road accidents will occur more frequently on commuter routes during ‘rush hour.’ Calls to shopping centres, leisure facilities, and in rural areas are likely to be higher at particular times of day and at weekends.

Rota patterns should be adjusted to reflect these temporal demands by supplying a matched number of operational resources. System status plans must be formulated by time of day, day of week, and season to ensure that these resources are deployed to those geographical locations with highest demand for any given time.[5]

Production and Use of Simple and Sophisticated System Status Plans

The development of demand and performance tables is an essential tool in the production of effective System Status Management (SSM) plans[6]. System status management is a complex process which requires the development of many plans, based on real call data, which detail the required positioning of un-tasked vehicles according to both time of day and day of week. The analysis of demand and performance at the narrower geographical level gives the necessary information to determine those locations with the highest probability of a call.

Simple plans which do not vary over a twenty four hour period can be developed relatively easily and implemented using simple manual dispatch methods (such as, for example, placing markers on a magnetic map). However to provide maximum effectiveness these must ultimately be replaced with sophisticated multiple plans which change according to the time of the day and the day of the week. Whilst these can be developed using simple information systems the (potentially) hundreds of different plans which result can only be implemented in the real world when control staff are supported by sophisticated computer-aided dispatch systems. Plans also need to be subject to continuous review to ensure that lessons learned from their use are translated into purposive amendments and that they are kept up to date with changing patterns of demand.

As an example the following table shows part of a simple SSM plan for one division. The full plan extends down to the level of the availability of a single vehicle.

Available Vehicles

Available Vehicles

Available Vehicles

Available Vehicles

Available Vehicles

Available Vehicles

12

11

10

9

8

7

Locations

Locations

Locations

Locations

Locations

Locations

A

B

B

B

B

B

B

C

C

C

C

C

C

D

D

D

D

D

D

E

E

E

F

G

E

F

F

F

G

H

F

G

G

G

H

J

G

H

H

H

J

L

H

I

I

J

L

 

I

J

J

L

 

J

K

L

 

K

L

 

L

 

Figure 4: Simple System Status Plan

This plan is utilised for only sixteen hours of each day as all stations in the very sparsely populated area covered by this division operate a ‘standby’ system (crews respond from home) at night. Despite this very challenging environment the plan’s use has considerably improved performance since its implementation in April of this year. The following graph charts Orcon performance for this division since implementation of the plan, compared to the same period in the previous year.

Figure 5: Impact on response times of the introduction of a simple system status plan

Complex System Status Plans (SSPs) have the potential for a much bigger impact on performance. The ultimate aim must be to have different plans available for every hour of every day and for every season to best ensure that resources are closely matched to demand. Indeed the new performance standards mandate that these plans reflect not only the level of demand but also the type of demand. Clearly SSPs should target resources to peaks in category A calls rather than to peaks in category B calls. It must be emphasised, however, that such a detailed approach can produce literally hundreds of different plans and that control staff can only be realistically expected to utilise these effectively if supported by sophisticated computerised command and control systems. A full description of the science and art of SSM is outside the scope of this article.

Review of Current Rota Patterns

Another major factor in the achievement of the new response standards relates to the level of resource deployment (i.e. number of operational vehicles) at any given time of day or day of week. Historically ambulance services have had a three shift system producing, for example, crew levels of 12, 12, 10. Traditionally these staffing levels are inflexible and do not recognise changes in demand by time of day or day of week or as a result of other seasonal factors such as bank holidays or unusual weather conditions.

The chart below shows the overall demand for one division by hour of day for one month against the level of resources (i.e. number of ambulances) provided.

Figure 6: Comparison of staffing pattern with demand profile.

This simple graph clearly demonstrates that a rigid rota system does not reflect the pattern of demand. For instance night time cover reduces at 2300 hours – one of the busiest slots in the 24 hour period. Additionally there is no recognition of the surge in workload between the hours of 10.00 and 15.00. Each day of the week has its own discreet demand patterns and may subsequently require a different pattern of cover. It is clear that in order to meet the new response targets there is an urgent need to move towards more flexible and effective rota arrangements. Neither simple eight or twelve hour shift patterns can meet this need. Only complex mixtures of shifts of several different lengths (annualised hours) can hope to meet this requirement whilst ensuring humane working arrangements for ambulance personnel. [7]

Conclusion

The need for an effective system of performance management and review is borne out by the following:

  • Levels of public expectation for high class public sector services have risen dramatically during the last few years and will continue to do so.
  • Increased media exposure of the Ambulance Service has led to a much higher profile. This linked with the well-publicised changes to response standards is likely to increase local awareness of the standards of service which we provide.
  • New income for current and future services will be difficult to attract unless it can be clearly demonstrated to commissioners and external audit organisations that the best possible value for money is already being achieved.

The authors believe that a formal system of performance management with clear targets and goals will ensure that staff and management efforts will be focused on service delivery issues. Relatively simple strategies which are not dependent on the availability of highly sophisticated information and command and control systems can produce dramatic improvements in performance. The new response standards provide an ideal opportunity to effect the organisational and cultural changes necessary to provide a truly high performance Ambulance Service.

Acknowledgements

The authors gratefully acknowledge the contribution provided by two anonymous reviewers in improving the first draft of this paper.

Conflict of interest: none.

Funding: none.

References

[1] Chapman R. et al. Review of Ambulance Performance Standards: Final Report of the Steering Group. London: NHS Executive, 1996.
[2] Department of Health. DGM (96) 162. London: Department of Health, 1996
[3] Welsh Office. DGM (98) 29. Cardiff: Welsh Office, 1998.
[4] Audit Commission. A Life in the Fast Lane: Value for Money in Emergency Ambulance Services. London: Audit Commission, 1998.
[5] Stout JL. System Status Management. The Strategy of Ambulance Placement. J Emerg Med Serv 1983;8(5):22-32.
[6] Stout JL, System Financing in Roush WR, ed. Principles of EMS Systems (second edition). Dallas: American College of Emergency Physicians, 1994: 453-473.
[7] Stout JL, Peak-Load Staffing. What’s fair for Personnel and Patients? J Emerg Med Serv 1989;14(8):73-4,76.