Faculty of Industrial Engineering and Management

1
Can We Make Simulation More Accessible To
Emergency Department Decision Makers
David Sinreich and Yariv Marmor
Faculty of Industrial Engineering and
Management Technion Israel Institute of
Technology
Emergency Multidisciplinary Research Unit SMBD -
Jewish General Hospital, August 29, 2005
2
The Healthcare Industry and Numbers

• The annual Canadian expenditure on healthcare in
  2001 was estimated at 64.2 billion (2100 per
  person).
• According to 2005 issue of healthcare in Canada
  (CIHI) the healthcare expenditure in 2004 grow to
  about 100 billion (3200 per person) and
  accounted for 10 of the GDP.
• Hospitals represented 30 of the total healthcare
  expenditure in 2004.

3
The Healthcare Industry and Numbers

• The annual U.S. expenditure on healthcare in 2003
  was estimated at 1.5 trillion (5000 per
  person) and is expected to reach 2.8 trillion by
  the year 2011.
• Healthcare accounted for 13.2 of the GDP in 2000
  and may reach 17 of the GDP by 2011.
• Hospitals represented 31.7 of the total
  healthcare expenditure in 2001. This expenditure
  is expected to decrease to 27 by 2012.
• According to the American College of Emergency
  Physicians (June 2003), the cost of Emergency
  Department (ED) operations amounted to 5 of the
  total US healthcare expenditure.

4
The Healthcare Industry and Numbers

• The ICBS reports that the annual healthcare
  spending in Israel in 2003 reached 10.1 billion
  (1700 per person), which accounts for 8.8 of
  the GDP.
• This level of expenditure is similar to other
  OECD countries such as Germany 10.9, France
  9.7, Sweden 9.2 and Australia 9.1 (number
  reflect expenditure in 2002)
• Hospitals accounted for 26.1 of the national
  expenditure on healthcare in 2001.

5
The Healthcare Industry and Numbers

• The ICBS reports that the annual healthcare
  spending in Israel in 2003 reached 10.1 billion
  (1700 per person), which accounts for 8.8 of
  the GDP.
• This level of expenditure is similar to other
  OECD countries such as Germany 10.9, France
  9.7, Sweden 9.2 and Australia 9.1 (number
  reflect expenditure in 2002)
• Hospitals accounted for 26.1 of the national
  expenditure on healthcare in 2001.

These numbers are a clear indication that
increasing the efficiency and productivity of
hospital and ED operations is critical to the
success of the entire healthcare system
6
The Emergency Department

• The ED serves as the hospitals gate keeper and
  is the most difficult department to manage.
• The ED has to handle efficiently and effectively
  a random arrival stream of patients.
• The ED has to be highly versatile and flexible.
• The ED is required to have the ability to react
  quickly to fast unfolding events.

7
The Emergency Department

• There are 2.5 million patient visits each year at
  the 25 EDs in Israel. This translates to about
  270 patient visits on average per day.
• On average there 30 - 40 beds in these EDs.
• Based on these number, there are around 7 - 8
  patient turnarounds a day, this translates to an
  average length of stay of 3 - 3.5 hours.
• In reality there are 4 - 6 times more patient
  arrivals during pick hours (between 11 13 and
  19 22) compared to other hours of the day.

8
Simulation of Healthcare and ED Systems

• Hospital management is reluctant to accept
  change, particularly if it comes from a
  'black-box' type of tool.
• Management often does not realize the benefits of
  using simulation-based analysis tools.
• Management is well aware of the time and cost
  that have to be invested in building detailed
  simulation models.
• Management believes that spending money on
  operational issues only diverts funds from
  patient care.
• Lack of experts with experience in modeling
  large, complex systems.

9
Modeling Options
The Model's Basic Building Blocks
Generic Processes
High abstraction level Flexible enough to model
any system and scenario Difficult to use requires
knowledge and experience
Medium abstraction level Flexible enough to model
any system which uses a similar process Simple
and intuitive to use after a brief and short
introduction
Low abstraction level Can only model and analyze
the system it was designed for Simple and easy to
use after a quick explanation
10
Increasing Acceptance of Simulation in Healthcare

• It is essential to build up the models
  credibility
• Hospital management should be directly involved
  in the development of simulation projects.
• The development of simulation projects should be
  done in-house by hospital personal.

11
Increasing Acceptance of Simulation in Healthcare

• General, and flexible

• Include default values for most of the system
  parameters.

• Simple to use

• Include a decision support system

12
Essential Basic Condition
For the tool to be general and flexible The
process patients go through when visiting an ED
has to be determined mainly by the patient type
(Internal, Orthopedic, Surgical etc.) rather than
by the hospital in which it is performed.
13
The Field Study

• Funded by the Israel National Institute for
  Health Policy (NIHP).
• 5 out of the 25 27 general hospitals operating
  in Israel participated in the study.
• Hospitals 1 and 3 are large (over 700 beds).
  Hospital 5 is medium (400 - 700 beds). Hospital 2
  and 4 are small (less than 400 beds).
• Hospital 5 is a regional hospital and the rest
  are inner-city hospitals.
• Hospitals 1 and 3 are level 1 trauma centers and
  the rest are level 2 centers.

14
The Field Study

• Teams of supervised students equipped with
  standardized code lists of the different process
  elements conducted time and motion studies in the
  selected hospitals.
• Data was also gathered from each hospitals
  information system.
• Additional data was gathered trough interviews
  with the hospital top management, ED chief
  physician and ED head nurse.

15
Processes and Patient Types

• Through observations, gathered data and
  interviews, 19 individual process charts each
  representing a typical patient types were
  identified.

Patient Type Hospital
Fast-Track, Internal, Surgical, Orthopedic 1
Internal, Surgical, Orthopedic 2
Walk-In Internal, Walk-In Orthopedic, Walk-In Surgical, Internal, Trauma 3
Internal Acute, Internal/Surgical Minor, Orthopedic 4
Fast-Track, Internal, Surgical, Orthopedic 5
16
Process Chart
17
(No Transcript)
18
The Similarity Measure - Activities
i
j
bji
eij
bij
e122,
a120.33
b122,
b212
19
The Similarity Measure - Relationships
c12 020000000 2
d12 002201010 6
r12 2/(62) 0.25
20
The Sensitivity of the Similarity Measure

• The similarity measure is sensitive to
• The absence of an activity or to additional
  activities a resource is expected to perform.
• The absence of a relationship or to an additional
  relationship between activities.
• The similarity measure is not sensitive to
• The order in which activities are expected to be
  performed.

21
Clustering the Patient Processes
Average Similarity Level 0.44
22
Clustering the Patient Processes

• Full enumeration and ranking was used to
  determine the best way to divide the processes
  into

Three clusters
Four clusters
Two clusters
23
Clustering the Processes Into Two Groups

• The first group included all the internal patient
  types from all 5 hospitals 1Int, 1FT, 2Int,
  3Int, 3W_Int, 4Int_S, 4Int_A, 5Int, 5FT.
• The best combined average similarity value for
  two clusters (0.579, 0.571) was 0.575.

24
Clustering the Patient Processes

• Full enumeration and ranking was used to
  determine the best way to divide the processes
  into

Three clusters
Four clusters
Two clusters
25
Clustering the Processes Into Three Groups

• The best combined average similarity value for
  three clusters was 0.638.
• The chosen clustering option was ranked as number
  17 with a combined average similarity value of
  (0.656, 0.746, 0.544) 0.623.

26
When good is better than best (Petroski 1994)
1
1

1

1

2
2
2
3
3

3

3

3
4
4
4

5
5

5

5

I

O

S

FT

I
O
S
I
O_W

S_W

I_W

T
I_S
O
I_A

I
O

S

FT

43

5
2
80
64
33
52
59

32

23
32

43
89
46
70

66
24

55

88

66
40
40
89
74
14

60

37
12

59
35
86
36

32
77

62

53


61
49
59
74
31

58

47
31

68
58
61
39

37
42

89

79

56
32
51
44

37

29
28

45
67
57
43

48
23

56

94


44
53
63

25

18
39

38
59
25
73

89
23

40

66


67
9
43

23
6

45
30
71
50

32
72

51

46

31

45

38
32

62
46
55
45

37
51

71

64

10

9

27

27
57
7
51

53
1

23

52


84
16

88
28
52
15

14
25

51

46


30

100
16
31
5

5
2

41

37


36
29
3
26

23
0

19

45

34
46
24

23
22

64

54

39
72

61
27

54

79

26

18
53

56
57

Average Similarity Level 0.656
77
32

27

59

16
25

55

Average Similarity Level 0.746

52
32

Average Similarity Level 0.544
65
Combined Average Similarity Level 0.623
27
Clustering the Patient Processes

• Full enumeration and ranking was used to
  determine the best way to divide the processes
  into

Three clusters
Four clusters
Two clusters
28
Clustering the Processes Into Four Groups

• The best combined average similarity value for
  four clusters was 0.683.
• The chosen clustering option was ranked as number
  76 with a combined average similarity value of
  (0.669, 0.746, 0.654, 0.558) 0.666.
• The first group included all acute internal
  patient types 1Int, 2Int, 3Int, 4Int_S, 4Int_A,
  5Int.
• The second group included most orthopedic
  patients types 1O, 2O, 4O, 5O.
• The third group included most surgical patients
  types 1S, 2S, 3O_W, 3S_W, 3T, 5S.

• The forth group included all ambulatory patients
  types 1FT, 3Int_W, 5FT.

29
Clustering the Patient Processes

• Full enumeration and ranking was used to
  determine the best way to divide the processes
  into

Three clusters
Four clusters
Two clusters

• The clustering options chosen were compared to
  the best similarity result of 1000 random
  clustering solutions into 4 groups
• For a selection probability (0.25, 0.25, 0.25,
  0.25) the best combined average similarity value
  was 0.554.
• For a selection probability (0.315, 0.21, 0.315,
  0.16) the best combined average similarity value
  was 0.56.

30
Conclusions
Based on this analysis it is safe to argue that
in the hospitals that participated in this study,
patient type has a higher impact in defining the
operation process than does the specific hospital
in which the patients are treated.
31
Increasing Acceptance of Simulation in Healthcare

• General, and flexible

• Include default values for most of the system
  parameters.

• Simple to use

• Include a decision support system

32
The Relative Precision of the Time Elements

• Since a time study is basically a statistical
  sampling process, it is important to estimate the
  precision of the gathered data.

Average Duration and Standard Deviation over all
observed elements i for patient type p at all the
hospitals
The number of times element i was observed for
each patient type p
The maximum number of times patient type p goes
through an element that is only performed once
during the ED process
Precision as a proportion of the gathered element
33
The Relative Precision of the Time Elements
The contribution of element i to the total
process time of patient type p
The relative weight of element i for patient type
p
The relative precision of element i
The relative precision for patient type p
Over 20,000 process elements were observed and
recorded.
34
Precision of the Different Time Elements
Patient Types Patient Types Patient Types Patient Types Patient Types Element Precision
Element Internal Surgical Orthopedic Trauma Fast-Track
Vital Signs 3.6 5.7 8.9 6.7 3.2 2.2
E.C.G. Check 3.6 11.3 16.0 13.1 9.7 3.0
Treatment Nurse 5.5 12.6 11.1 10.8 15.6 3.9
Follow-up Nurse 10.1 47.5 43.0 19.7 50.1 7.9
Instructions Prior to Discharge 16.5 30.7 29.1 25.2 43.2 11.9
First Examination 4.6 6.3 4.4 7.4 10.2 2.8
Second or Third Examination 6.7 11.4 8.0 11.8 30.2 4.3
Follow-Up Physician 5.9 27.8 26.0 32.9 ---- 5.4
Hospitalization /Discharge 11.0 13.0 19.3 32.9 15.0 7.5
Handling Patient and Family 6.5 15.9 9.3 9.5 18.4 4.6
Treatment Physician 11.3 12.9 15.4 21.2 49.9 7.1
Patient Precision 5.2 9.4 8.1 9.5 7.6
35
Conclusions
The combined precision values indicate, that
aggregating element duration regardless of
patient type and the hospital in which the
patients are treated, improves the precision
levels of all the different elements.
36
Increasing Acceptance of Simulation in Healthcare

• General, and flexible

• Include default values for most of the system
  parameters.

37
Increasing Acceptance of Simulation in Healthcare

• General, and flexible

• Include default values for most of the system
  parameters.

• Simple to use

• Include a decision support system

38
The Structure of the Simulation Tool
Decision Support System
Graphical User Interface based on the Generic
Process
ARENAs Simulation Model
Mathematical Models
39
(No Transcript)
40
Imaging Center
41
Specialists
42
Scheduling Medical Staff
43
The Structure of the Simulation Tool
Decision Support System
Graphical User Interface based on the Generic
Process
ARENAs Simulation Model
Mathematical Models
44
Mathematical Model Development
The following mathematical models were developed
based on the gathered information

• Patient arrivals to the ED

• Patient Arrivals at the Imaging Center

• Staffs walking time

45
Estimating the Patient Arrival Process

• The gathered data reveals that the number of
  patients arriving at the ED differs from hour to
  hour and from day to day
• Statistical tests reveal that the square-root of
  the patients' arrival process can be described by
  a normal distribution.

• Let Xpihd be a random variable normally
  distributed with a mean of that represents
  the square-root of the number of patients of type
  p who arrive at the ED of hospital i at hour h on
  day d.

46
Estimating the Patient Arrival Process
The patient arrival process is similar for all
the hospitals surveyed therefore it was decided
to combine the gathered data from all hospitals
47
Estimating the Patient Arrival Process
The number of patients of type p who arrive at
hospital i at hour h on day d
48
Estimating the Patient Arrival Process
In the case a new hospital whishes to use the
simulation tool all that is needed are the
values obtained from the hospital's computerized
information systems.
The rest of the process, which includes
calculating the formulas, is performed
automatically by the simulation tool.
Patient Type Patient Type Patient Type
Hospital Internal Surgical Orthopedic
1 1.180 1.293 1.187
2 0.958 1.038 0.840
4 0.862 0.669 0.974
It is clear from these factors that hospital 1 is
larger than the other two hospitals
49
Validating The Model
Internal Patients on Monday
Internal Patients on Saturday
50
Validating The Model
Surgical Patients on Wednesday
51
Validating The Model
The distribution of the residuals between the
predicted patient arrivals and the actual patient
arrivals.
Moments Mean 0.0000844 Std Dev 0.6003367 Std
Err Mean 0.0025241 upper 95 Mean 0.0050316 lowe
r 95 Mean -0.004863 N 56571


Shapiro-Wilk goodness of fit tests reveal that
the residuals can be described by a normal
distribution with a mean close to 0, and a
standard deviation of 0.6.
52
Mathematical Model Development
The following mathematical models were developed
based on the gathered information

• Patient arrivals to the ED

• Patient Arrivals at the Imaging Center

• Staffs walking time

53
The Patient Arrival Process to the Imaging Center

• To accurately estimate the turnaround time ED
  patients experience at the imaging center it is
  important to estimate the following
• patients' walking time
• Waiting time at the imaging center
• the time it takes to perform an X-ray
• the time it takes the radiologist to view the
  X-ray to return a diagnose
• Imaging centers (X-ray, CT and ultrasound) are
  not always ED-dedicated. In some cases these
  centers serve the entire hospital patient
  population.

54
The Patient Arrival Process to the Imaging Center

• In these cases two different patient types are
  sent to the imaging center for service
• patients who come from the ED
• patients who come from all other hospital wards
• These two streams interact and interfere with
  each other and compete for the same resources
• In these case it is imperative to estimate the
  hospital patient arrival process.

55
Estimating The Imaging Center Arrival Process

• The gathered data reveals that the number of
  hospital patients arriving at the imagining
  center differs from hour to hour and from day to
  day and from month to month
• Statistical tests reveal that the square-root of
  the hospital patients' arrival process can be
  described by a normal distribution.

56
Estimating The Imaging Center Arrival Process

• A linear regression model was used to estimate
  the stream of hospital patients. In order to
  maintain the model's linearity, four separate
  regression sub-models were developed.

• A sub-model to estimate the arrivals between 6 AM
  and 12 midnight on weekdays.
• A sub-model to estimate the arrivals between 6 AM
  and 12 midnight on weekends.
• A sub-model to estimate the arrivals between 12
  midnight and 6 AM on weekdays and weekends.
• A sub-model to estimate the arrivals between 12
  noon and 5 PM in the cases the central imaging
  center only operates part of the day.

57
Estimating The Imaging Center Arrival Process
The number of hospital patients of type p who
arrive at imagining center of hospital i at hour
h on day d and on month m
58
Validating The Model
Hospital Patient Arrivals to the Imaging Center
on A Tuesday
59
Validating The Model
The distribution of the residuals between the
predicted patient arrivals and the actual patient
arrivals.
Moments Mean -1.62e-14 Std Dev 0.8030956 Std
Err Mean 0.0075429 upper 95 Mean 0.0147854 lowe
r 95 Mean -0.014785 N 11336


Shapiro-Wilk goodness of fit tests reveal that
the residuals can be described by a normal
distribution with a mean close to 0, and a
standard deviation of 0.8.
60
Mathematical Model Development
The following mathematical models were developed
based on the gathered information

• Patient arrivals to the ED

• Patient Arrivals at the Imaging Center

• Staffs walking time

61
Estimating the Staffs Walking Time

• Observations show that the medical staff spends a
  considerable amount of time, during each shift,
  walking between the different activity points in
  the ED.

• patient beds
• medicine cabinet
• nurse's station
• ED main counter

• The estimation model is based on the following
  parameters

• The distances between the different activity
  points
• The number of beds each staff member is in charge
  of
• The ED space dimensions each staff member
  operates in.

62
Estimating the Staffs Walking Time
Physician's mean walking time when treating
patient type p (sec)
Nurse's mean walking time when treating patient
type p (sec)
Width, Length of the space in which the medical
staff operates (cm)
Walking distance from the area's centroid to the
ED counter (cm)
Walking distance from the area's centroid to the
procedure room (cm)
Walking distance from the area's centroid to the
medicine cabinet (cm)
Walking distance from the area's centroid to the
nurse's station (cm)
Number of patient beds in the ED room
63
Estimating the Staffs Walking Time
Physicians Walking Model
Nurses Walking Model
64
Validating The Model

• The fit of the above models as indicated by R2 is
  0.737 for the physician's walking model and 0.675
  for the nurse's walking models.
• The variance analysis shows the both models and
  all their parameters are significant.
• The residual analyses of the physicians' and
  nurses' estimation walking models reveal that in
  both cases residuals are normally distributed
  with a mean of 0.
• The models have been used in a setting different
  from the ones that were used in the initial
  development.

65
Validating The Model

• The single factor ANOVA in both cases reveals
  that the null hypothesis, (there is no
  statistical difference between the model and
  observation results) can not be rejected.
• P-value for the physicians model was 0.28
• P-value for the nurses model was 0.74

66
The Structure of the Simulation Tool
Decision Support System
Graphical User Interface based on the Generic
Process
ARENAs Simulation Model
Mathematical Models
67
Increasing Acceptance of Simulation in Healthcare

• General, and flexible

• Include default values for most of the system
  parameters.

• Simple to use

• Include a decision support system

68
The Decision Support Module
69
Model Validation

• The validation process is comprised of two
  stages
• Five simulation models were created using the
  developed tool in conjunction with the suggested
  default values and the other specific values for
  each of the five EDs that participated in the
  study.
• Ten 60-day simulation runs were performed for
  each of the five EDs.
• The performance of each of these models was
  compared to the actual data that was obtained
  from each of hospital's information systems
  (250,000 data entries that represent around 2.5
  years of data).

70
Model Validation

• Statistical significance of the differences
  between the simulation and the averages obtained
  from the information system.

• Practical significance of the differences between
  the information system's and simulation averages.

Practical Difference
Statistical Significance
Time
71
Model Validation
Comparison of the Results Obtained for the ED in
Hospital 1
P-Value Practical Difference Simulation Std. Simulation Average (10 runs) Database Average (2 years) Patient Type
0.33 6.7 13 182 195 Internal
0.18 6.6 10 211 198 Surgical
0.28 4.5 7 150 157 Orthopedic
Comparison of the Results Obtained for the ED in
Hospital 2
P-Value Practical Difference Simulation Std. Simulation Average (10 runs) Database Average (2 years) Patient Type
0.67 2.2 20 399 408 Internal
0.75 1.7 11 240 236 Surgical
0.28 6.1 9 156 166 Orthopedic
72
Model Validation
Comparison of the Results Obtained for the ED in
Hospital 3
P-Value Practical Difference Simulation Std. Simulation Average (10runs) Database Average (2 years) Patient Type
0.31 6.5 18 261 279 Internal
0.09 14.4 13 125 146 Surgical
0.59 6.0 15 142 134 Orthopedic
Comparison of the Results Obtained for the ED in
Hospital 4
P-Value Practical Difference Simulation Std. Simulation Average (10 runs) Database Average (2 years) Patient Type
0.32 10.6 17 178 161 Internal
0.59 5.7 16 149 158 Surgical
0.68 1.6 6 127 125 Orthopedic
73
Model Validation
Comparison of the Results Obtained for the ED in
Hospital 5
P-Value Practical Difference Simulation Std. Simulation Average (10 runs) Database Average (2 years) Patient Type
0.48 6.7 13 143 134 Fast-Track
0.14 14.5 19 197 172 Internal
0.06 8.4 8 103 95 Surgical
0.32 14.8 6 93 81 Orthopedic
74
Model Validation
Internal Patients During a Weekday in the ED of
Hospital 1
Orthopedic Patients During a Weekend day in the
ED of Hospital 3
75
Model Validation
Internal Patients During a Weekday in the ED of
Hospital 4
76
Model Validation

• A sixth ED was chosen and data on its operations
  was gathered from the hospital's information
  systems and through observations.
• A simulation model was created using the tool's
  default values augmented by some of the gathered
  data and ten 60-day simulation runs were
  performed.

Comparison of the Results Obtained for the ED in
Hospital 6
P-Value Practical Difference Simulation Std Simulation Average (10 runs) Database Average (2 years) Patient Type
0.36 9.5 16 161 147 Internal
0.67 3.2 11 149 154 Surgical
0.09 13.8 7 132 116 Orthopedic
77
Model Validation
Surgical Patients During a Weekday in the ED of
Hospital 6
Internal Patients During a Weekend day in the ED
of Hospital 6
78
Conclusions
If we use the statement
The suggested unified generic process can be
used to model any arbitrary ED"
as a scientific hypothesis and try to find a
system for which the statement is not true, each
failure increases our confidence in the model.
So far we have failed to reject the statement
eight times
79
Acknowledgment
To the Israeli National Institute for Health
Policy and Health Services Research NIHP To all
the students from the IEMgmt. Faculty and the
Research Center for Human Factors and Work Safety
which assisted in gathering the data and
analyzing it and especially to Almog Shani and
Ira Goldberg