Title: Faculty of Industrial Engineering and Management
1Can We Make Simulation More Accessible To
Emergency Department Decision Makers
David Sinreich and Yariv Marmor
Faculty of Industrial Engineering and
Management Technion Israel Institute of
Technology
Emergency Multidisciplinary Research Unit SMBD -
Jewish General Hospital, August 29, 2005
2The Healthcare Industry and Numbers
- The annual Canadian expenditure on healthcare in
2001 was estimated at 64.2 billion (2100 per
person). - According to 2005 issue of healthcare in Canada
(CIHI) the healthcare expenditure in 2004 grow to
about 100 billion (3200 per person) and
accounted for 10 of the GDP. - Hospitals represented 30 of the total healthcare
expenditure in 2004.
3The Healthcare Industry and Numbers
- The annual U.S. expenditure on healthcare in 2003
was estimated at 1.5 trillion (5000 per
person) and is expected to reach 2.8 trillion by
the year 2011. - Healthcare accounted for 13.2 of the GDP in 2000
and may reach 17 of the GDP by 2011. - Hospitals represented 31.7 of the total
healthcare expenditure in 2001. This expenditure
is expected to decrease to 27 by 2012. - According to the American College of Emergency
Physicians (June 2003), the cost of Emergency
Department (ED) operations amounted to 5 of the
total US healthcare expenditure.
4The Healthcare Industry and Numbers
- The ICBS reports that the annual healthcare
spending in Israel in 2003 reached 10.1 billion
(1700 per person), which accounts for 8.8 of
the GDP. - This level of expenditure is similar to other
OECD countries such as Germany 10.9, France
9.7, Sweden 9.2 and Australia 9.1 (number
reflect expenditure in 2002) - Hospitals accounted for 26.1 of the national
expenditure on healthcare in 2001.
5The Healthcare Industry and Numbers
- The ICBS reports that the annual healthcare
spending in Israel in 2003 reached 10.1 billion
(1700 per person), which accounts for 8.8 of
the GDP. - This level of expenditure is similar to other
OECD countries such as Germany 10.9, France
9.7, Sweden 9.2 and Australia 9.1 (number
reflect expenditure in 2002) - Hospitals accounted for 26.1 of the national
expenditure on healthcare in 2001.
These numbers are a clear indication that
increasing the efficiency and productivity of
hospital and ED operations is critical to the
success of the entire healthcare system
6The Emergency Department
- The ED serves as the hospitals gate keeper and
is the most difficult department to manage. - The ED has to handle efficiently and effectively
a random arrival stream of patients. - The ED has to be highly versatile and flexible.
- The ED is required to have the ability to react
quickly to fast unfolding events.
7The Emergency Department
- There are 2.5 million patient visits each year at
the 25 EDs in Israel. This translates to about
270 patient visits on average per day. - On average there 30 - 40 beds in these EDs.
- Based on these number, there are around 7 - 8
patient turnarounds a day, this translates to an
average length of stay of 3 - 3.5 hours. - In reality there are 4 - 6 times more patient
arrivals during pick hours (between 11 13 and
19 22) compared to other hours of the day.
8Simulation of Healthcare and ED Systems
- Hospital management is reluctant to accept
change, particularly if it comes from a
'black-box' type of tool. - Management often does not realize the benefits of
using simulation-based analysis tools. - Management is well aware of the time and cost
that have to be invested in building detailed
simulation models. - Management believes that spending money on
operational issues only diverts funds from
patient care. - Lack of experts with experience in modeling
large, complex systems.
9Modeling Options
The Model's Basic Building Blocks
Generic Processes
High abstraction level Flexible enough to model
any system and scenario Difficult to use requires
knowledge and experience
Medium abstraction level Flexible enough to model
any system which uses a similar process Simple
and intuitive to use after a brief and short
introduction
Low abstraction level Can only model and analyze
the system it was designed for Simple and easy to
use after a quick explanation
10Increasing Acceptance of Simulation in Healthcare
- It is essential to build up the models
credibility - Hospital management should be directly involved
in the development of simulation projects. - The development of simulation projects should be
done in-house by hospital personal.
11Increasing Acceptance of Simulation in Healthcare
- Include default values for most of the system
parameters.
- Include a decision support system
12Essential Basic Condition
For the tool to be general and flexible The
process patients go through when visiting an ED
has to be determined mainly by the patient type
(Internal, Orthopedic, Surgical etc.) rather than
by the hospital in which it is performed.
13The Field Study
- Funded by the Israel National Institute for
Health Policy (NIHP). - 5 out of the 25 27 general hospitals operating
in Israel participated in the study. - Hospitals 1 and 3 are large (over 700 beds).
Hospital 5 is medium (400 - 700 beds). Hospital 2
and 4 are small (less than 400 beds). - Hospital 5 is a regional hospital and the rest
are inner-city hospitals. - Hospitals 1 and 3 are level 1 trauma centers and
the rest are level 2 centers.
14The Field Study
- Teams of supervised students equipped with
standardized code lists of the different process
elements conducted time and motion studies in the
selected hospitals. - Data was also gathered from each hospitals
information system. - Additional data was gathered trough interviews
with the hospital top management, ED chief
physician and ED head nurse.
15Processes and Patient Types
- Through observations, gathered data and
interviews, 19 individual process charts each
representing a typical patient types were
identified.
Patient Type Hospital
Fast-Track, Internal, Surgical, Orthopedic 1
Internal, Surgical, Orthopedic 2
Walk-In Internal, Walk-In Orthopedic, Walk-In Surgical, Internal, Trauma 3
Internal Acute, Internal/Surgical Minor, Orthopedic 4
Fast-Track, Internal, Surgical, Orthopedic 5
16Process Chart
17(No Transcript)
18The Similarity Measure - Activities
i
j
bji
eij
bij
e122,
a120.33
b122,
b212
19The Similarity Measure - Relationships
c12 020000000 2
d12 002201010 6
r12 2/(62) 0.25
20The Sensitivity of the Similarity Measure
- The similarity measure is sensitive to
- The absence of an activity or to additional
activities a resource is expected to perform. - The absence of a relationship or to an additional
relationship between activities. - The similarity measure is not sensitive to
- The order in which activities are expected to be
performed.
21Clustering the Patient Processes
Average Similarity Level 0.44
22Clustering the Patient Processes
- Full enumeration and ranking was used to
determine the best way to divide the processes
into
Three clusters
Four clusters
Two clusters
23Clustering the Processes Into Two Groups
- The first group included all the internal patient
types from all 5 hospitals 1Int, 1FT, 2Int,
3Int, 3W_Int, 4Int_S, 4Int_A, 5Int, 5FT. - The best combined average similarity value for
two clusters (0.579, 0.571) was 0.575.
24Clustering the Patient Processes
- Full enumeration and ranking was used to
determine the best way to divide the processes
into
Three clusters
Four clusters
Two clusters
25Clustering the Processes Into Three Groups
- The best combined average similarity value for
three clusters was 0.638. - The chosen clustering option was ranked as number
17 with a combined average similarity value of
(0.656, 0.746, 0.544) 0.623.
26When good is better than best (Petroski 1994)
1
1
1
1
2
2
2
3
3
3
3
3
4
4
4
5
5
5
5
I
O
S
FT
I
O
S
I
O_W
S_W
I_W
T
I_S
O
I_A
I
O
S
FT
43
5
2
80
64
33
52
59
32
23
32
43
89
46
70
66
24
55
88
66
40
40
89
74
14
60
37
12
59
35
86
36
32
77
62
53
61
49
59
74
31
58
47
31
68
58
61
39
37
42
89
79
56
32
51
44
37
29
28
45
67
57
43
48
23
56
94
44
53
63
25
18
39
38
59
25
73
89
23
40
66
67
9
43
23
6
45
30
71
50
32
72
51
46
31
45
38
32
62
46
55
45
37
51
71
64
10
9
27
27
57
7
51
53
1
23
52
84
16
88
28
52
15
14
25
51
46
30
100
16
31
5
5
2
41
37
36
29
3
26
23
0
19
45
34
46
24
23
22
64
54
39
72
61
27
54
79
26
18
53
56
57
Average Similarity Level 0.656
77
32
27
59
16
25
55
Average Similarity Level 0.746
52
32
Average Similarity Level 0.544
65
Combined Average Similarity Level 0.623
27Clustering the Patient Processes
- Full enumeration and ranking was used to
determine the best way to divide the processes
into
Three clusters
Four clusters
Two clusters
28Clustering the Processes Into Four Groups
- The best combined average similarity value for
four clusters was 0.683. - The chosen clustering option was ranked as number
76 with a combined average similarity value of
(0.669, 0.746, 0.654, 0.558) 0.666. - The first group included all acute internal
patient types 1Int, 2Int, 3Int, 4Int_S, 4Int_A,
5Int. - The second group included most orthopedic
patients types 1O, 2O, 4O, 5O. - The third group included most surgical patients
types 1S, 2S, 3O_W, 3S_W, 3T, 5S.
- The forth group included all ambulatory patients
types 1FT, 3Int_W, 5FT.
29Clustering the Patient Processes
- Full enumeration and ranking was used to
determine the best way to divide the processes
into
Three clusters
Four clusters
Two clusters
- The clustering options chosen were compared to
the best similarity result of 1000 random
clustering solutions into 4 groups - For a selection probability (0.25, 0.25, 0.25,
0.25) the best combined average similarity value
was 0.554. - For a selection probability (0.315, 0.21, 0.315,
0.16) the best combined average similarity value
was 0.56.
30Conclusions
Based on this analysis it is safe to argue that
in the hospitals that participated in this study,
patient type has a higher impact in defining the
operation process than does the specific hospital
in which the patients are treated.
31Increasing Acceptance of Simulation in Healthcare
- Include default values for most of the system
parameters.
- Include a decision support system
32The Relative Precision of the Time Elements
- Since a time study is basically a statistical
sampling process, it is important to estimate the
precision of the gathered data.
Average Duration and Standard Deviation over all
observed elements i for patient type p at all the
hospitals
The number of times element i was observed for
each patient type p
The maximum number of times patient type p goes
through an element that is only performed once
during the ED process
Precision as a proportion of the gathered element
33The Relative Precision of the Time Elements
The contribution of element i to the total
process time of patient type p
The relative weight of element i for patient type
p
The relative precision of element i
The relative precision for patient type p
Over 20,000 process elements were observed and
recorded.
34Precision of the Different Time Elements
Patient Types Patient Types Patient Types Patient Types Patient Types Element Precision
Element Internal Surgical Orthopedic Trauma Fast-Track
Vital Signs 3.6 5.7 8.9 6.7 3.2 2.2
E.C.G. Check 3.6 11.3 16.0 13.1 9.7 3.0
Treatment Nurse 5.5 12.6 11.1 10.8 15.6 3.9
Follow-up Nurse 10.1 47.5 43.0 19.7 50.1 7.9
Instructions Prior to Discharge 16.5 30.7 29.1 25.2 43.2 11.9
First Examination 4.6 6.3 4.4 7.4 10.2 2.8
Second or Third Examination 6.7 11.4 8.0 11.8 30.2 4.3
Follow-Up Physician 5.9 27.8 26.0 32.9 ---- 5.4
Hospitalization /Discharge 11.0 13.0 19.3 32.9 15.0 7.5
Handling Patient and Family 6.5 15.9 9.3 9.5 18.4 4.6
Treatment Physician 11.3 12.9 15.4 21.2 49.9 7.1
Patient Precision 5.2 9.4 8.1 9.5 7.6
35Conclusions
The combined precision values indicate, that
aggregating element duration regardless of
patient type and the hospital in which the
patients are treated, improves the precision
levels of all the different elements.
36Increasing Acceptance of Simulation in Healthcare
- Include default values for most of the system
parameters.
37Increasing Acceptance of Simulation in Healthcare
- Include default values for most of the system
parameters.
- Include a decision support system
38The Structure of the Simulation Tool
Decision Support System
Graphical User Interface based on the Generic
Process
ARENAs Simulation Model
Mathematical Models
39(No Transcript)
40Imaging Center
41Specialists
42Scheduling Medical Staff
43The Structure of the Simulation Tool
Decision Support System
Graphical User Interface based on the Generic
Process
ARENAs Simulation Model
Mathematical Models
44Mathematical Model Development
The following mathematical models were developed
based on the gathered information
- Patient arrivals to the ED
- Patient Arrivals at the Imaging Center
45Estimating the Patient Arrival Process
- The gathered data reveals that the number of
patients arriving at the ED differs from hour to
hour and from day to day - Statistical tests reveal that the square-root of
the patients' arrival process can be described by
a normal distribution.
- Let Xpihd be a random variable normally
distributed with a mean of that represents
the square-root of the number of patients of type
p who arrive at the ED of hospital i at hour h on
day d.
46Estimating the Patient Arrival Process
The patient arrival process is similar for all
the hospitals surveyed therefore it was decided
to combine the gathered data from all hospitals
47Estimating the Patient Arrival Process
The number of patients of type p who arrive at
hospital i at hour h on day d
48Estimating the Patient Arrival Process
In the case a new hospital whishes to use the
simulation tool all that is needed are the
values obtained from the hospital's computerized
information systems.
The rest of the process, which includes
calculating the formulas, is performed
automatically by the simulation tool.
Patient Type Patient Type Patient Type
Hospital Internal Surgical Orthopedic
1 1.180 1.293 1.187
2 0.958 1.038 0.840
4 0.862 0.669 0.974
It is clear from these factors that hospital 1 is
larger than the other two hospitals
49Validating The Model
Internal Patients on Monday
Internal Patients on Saturday
50Validating The Model
Surgical Patients on Wednesday
51Validating The Model
The distribution of the residuals between the
predicted patient arrivals and the actual patient
arrivals.
Moments Mean 0.0000844 Std Dev 0.6003367 Std
Err Mean 0.0025241 upper 95 Mean 0.0050316 lowe
r 95 Mean -0.004863 N 56571
Shapiro-Wilk goodness of fit tests reveal that
the residuals can be described by a normal
distribution with a mean close to 0, and a
standard deviation of 0.6.
52Mathematical Model Development
The following mathematical models were developed
based on the gathered information
- Patient arrivals to the ED
- Patient Arrivals at the Imaging Center
53The Patient Arrival Process to the Imaging Center
- To accurately estimate the turnaround time ED
patients experience at the imaging center it is
important to estimate the following - patients' walking time
- Waiting time at the imaging center
- the time it takes to perform an X-ray
- the time it takes the radiologist to view the
X-ray to return a diagnose - Imaging centers (X-ray, CT and ultrasound) are
not always ED-dedicated. In some cases these
centers serve the entire hospital patient
population.
54The Patient Arrival Process to the Imaging Center
- In these cases two different patient types are
sent to the imaging center for service - patients who come from the ED
- patients who come from all other hospital wards
- These two streams interact and interfere with
each other and compete for the same resources - In these case it is imperative to estimate the
hospital patient arrival process.
55Estimating The Imaging Center Arrival Process
- The gathered data reveals that the number of
hospital patients arriving at the imagining
center differs from hour to hour and from day to
day and from month to month - Statistical tests reveal that the square-root of
the hospital patients' arrival process can be
described by a normal distribution.
56Estimating The Imaging Center Arrival Process
- A linear regression model was used to estimate
the stream of hospital patients. In order to
maintain the model's linearity, four separate
regression sub-models were developed.
- A sub-model to estimate the arrivals between 6 AM
and 12 midnight on weekdays. - A sub-model to estimate the arrivals between 6 AM
and 12 midnight on weekends. - A sub-model to estimate the arrivals between 12
midnight and 6 AM on weekdays and weekends. - A sub-model to estimate the arrivals between 12
noon and 5 PM in the cases the central imaging
center only operates part of the day.
57Estimating The Imaging Center Arrival Process
The number of hospital patients of type p who
arrive at imagining center of hospital i at hour
h on day d and on month m
58Validating The Model
Hospital Patient Arrivals to the Imaging Center
on A Tuesday
59Validating The Model
The distribution of the residuals between the
predicted patient arrivals and the actual patient
arrivals.
Moments Mean -1.62e-14 Std Dev 0.8030956 Std
Err Mean 0.0075429 upper 95 Mean 0.0147854 lowe
r 95 Mean -0.014785 N 11336
Shapiro-Wilk goodness of fit tests reveal that
the residuals can be described by a normal
distribution with a mean close to 0, and a
standard deviation of 0.8.
60Mathematical Model Development
The following mathematical models were developed
based on the gathered information
- Patient arrivals to the ED
- Patient Arrivals at the Imaging Center
61Estimating the Staffs Walking Time
- Observations show that the medical staff spends a
considerable amount of time, during each shift,
walking between the different activity points in
the ED.
- patient beds
- medicine cabinet
- nurse's station
- ED main counter
- The estimation model is based on the following
parameters
- The distances between the different activity
points - The number of beds each staff member is in charge
of - The ED space dimensions each staff member
operates in.
62Estimating the Staffs Walking Time
Physician's mean walking time when treating
patient type p (sec)
Nurse's mean walking time when treating patient
type p (sec)
Width, Length of the space in which the medical
staff operates (cm)
Walking distance from the area's centroid to the
ED counter (cm)
Walking distance from the area's centroid to the
procedure room (cm)
Walking distance from the area's centroid to the
medicine cabinet (cm)
Walking distance from the area's centroid to the
nurse's station (cm)
Number of patient beds in the ED room
63Estimating the Staffs Walking Time
Physicians Walking Model
Nurses Walking Model
64Validating The Model
- The fit of the above models as indicated by R2 is
0.737 for the physician's walking model and 0.675
for the nurse's walking models. - The variance analysis shows the both models and
all their parameters are significant. - The residual analyses of the physicians' and
nurses' estimation walking models reveal that in
both cases residuals are normally distributed
with a mean of 0. - The models have been used in a setting different
from the ones that were used in the initial
development.
65Validating The Model
- The single factor ANOVA in both cases reveals
that the null hypothesis, (there is no
statistical difference between the model and
observation results) can not be rejected. - P-value for the physicians model was 0.28
- P-value for the nurses model was 0.74
66The Structure of the Simulation Tool
Decision Support System
Graphical User Interface based on the Generic
Process
ARENAs Simulation Model
Mathematical Models
67Increasing Acceptance of Simulation in Healthcare
- Include default values for most of the system
parameters.
- Include a decision support system
68The Decision Support Module
69Model Validation
- The validation process is comprised of two
stages - Five simulation models were created using the
developed tool in conjunction with the suggested
default values and the other specific values for
each of the five EDs that participated in the
study. - Ten 60-day simulation runs were performed for
each of the five EDs. - The performance of each of these models was
compared to the actual data that was obtained
from each of hospital's information systems
(250,000 data entries that represent around 2.5
years of data).
70Model Validation
- Statistical significance of the differences
between the simulation and the averages obtained
from the information system.
- Practical significance of the differences between
the information system's and simulation averages.
Practical Difference
Statistical Significance
Time
71Model Validation
Comparison of the Results Obtained for the ED in
Hospital 1
P-Value Practical Difference Simulation Std. Simulation Average (10 runs) Database Average (2 years) Patient Type
0.33 6.7 13 182 195 Internal
0.18 6.6 10 211 198 Surgical
0.28 4.5 7 150 157 Orthopedic
Comparison of the Results Obtained for the ED in
Hospital 2
P-Value Practical Difference Simulation Std. Simulation Average (10 runs) Database Average (2 years) Patient Type
0.67 2.2 20 399 408 Internal
0.75 1.7 11 240 236 Surgical
0.28 6.1 9 156 166 Orthopedic
72Model Validation
Comparison of the Results Obtained for the ED in
Hospital 3
P-Value Practical Difference Simulation Std. Simulation Average (10runs) Database Average (2 years) Patient Type
0.31 6.5 18 261 279 Internal
0.09 14.4 13 125 146 Surgical
0.59 6.0 15 142 134 Orthopedic
Comparison of the Results Obtained for the ED in
Hospital 4
P-Value Practical Difference Simulation Std. Simulation Average (10 runs) Database Average (2 years) Patient Type
0.32 10.6 17 178 161 Internal
0.59 5.7 16 149 158 Surgical
0.68 1.6 6 127 125 Orthopedic
73Model Validation
Comparison of the Results Obtained for the ED in
Hospital 5
P-Value Practical Difference Simulation Std. Simulation Average (10 runs) Database Average (2 years) Patient Type
0.48 6.7 13 143 134 Fast-Track
0.14 14.5 19 197 172 Internal
0.06 8.4 8 103 95 Surgical
0.32 14.8 6 93 81 Orthopedic
74Model Validation
Internal Patients During a Weekday in the ED of
Hospital 1
Orthopedic Patients During a Weekend day in the
ED of Hospital 3
75Model Validation
Internal Patients During a Weekday in the ED of
Hospital 4
76Model Validation
- A sixth ED was chosen and data on its operations
was gathered from the hospital's information
systems and through observations. - A simulation model was created using the tool's
default values augmented by some of the gathered
data and ten 60-day simulation runs were
performed.
Comparison of the Results Obtained for the ED in
Hospital 6
P-Value Practical Difference Simulation Std Simulation Average (10 runs) Database Average (2 years) Patient Type
0.36 9.5 16 161 147 Internal
0.67 3.2 11 149 154 Surgical
0.09 13.8 7 132 116 Orthopedic
77Model Validation
Surgical Patients During a Weekday in the ED of
Hospital 6
Internal Patients During a Weekend day in the ED
of Hospital 6
78Conclusions
If we use the statement
The suggested unified generic process can be
used to model any arbitrary ED"
as a scientific hypothesis and try to find a
system for which the statement is not true, each
failure increases our confidence in the model.
So far we have failed to reject the statement
eight times
79Acknowledgment
To the Israeli National Institute for Health
Policy and Health Services Research NIHP To all
the students from the IEMgmt. Faculty and the
Research Center for Human Factors and Work Safety
which assisted in gathering the data and
analyzing it and especially to Almog Shani and
Ira Goldberg