Overview of Simulation Models and a Simulation Model for NHIS Field Operations and Cost Estimates

1
Overview of Simulation Models and a Simulation
Model for NHIS Field Operations and Cost Estimates

• Bor-Chung Chen
• Office of Railroad Safety
• Federal Railroad Administration/USDOT
• April 7, 2011

2
Staff Allocation Models

• Risk and Safety
• Safety Data
• Risk/Reliability Data
• Safety Data-Driven
• Workload and Activities
• Inspector Activity Report Data
• Demand-Driven
• Required by Regulations/Law

3
Example Model (1)

• NIP (National Inspection Plan) Staff Allocation
  Model
• Minimize damages of railroad accidents
• Regression analysis is used based on
  injuries/fatalities data
• Constraints on specified percentage changes of
  current inspector position slots

4
Example Model (2)

• Transportation Security Administrations Staff
  Allocation Model
• Three Components
• GRA Flight Data by GRA, Inc. baggage volume,
  flight passenger distribution, and load factor
• Regal Software by Regal Decision Systems
  airport configuration and screening process
• Sabre Software by Sabre Airline Solutions, Inc.
  scheduling process to determine staff needed in
  waiting time of 10 minutes (or 5) or less.

5
Example Model (3)

• Positive Train Control (PTC) Staff Allocation
  Model
• Four PTC Components
• Dispatch Center
• Communication Systems
• Locomotive Units
• Wayside Units
• Appropriateness/Effectiveness Index Score
• Maximize the total score of all inspector
  assignments to all inspector activities. (To
  allocate the skilled inspectors to the inspection
  activities in reaching the most effective
  performance)
• Constraints

6
Simulation Modeling An Operations Research
Method Optimization Save Resources and/or Improve
Data Quality
7
Operations Research (OR) seeks the determination
of the best (optimum) course of action of a
decision problem under the restriction of limited
resources
8
An optimization model is a decision-making tool
that recommends an answer (the goal to be
optimized) based on analyses of information
(constraints and decision variables). It consists
of three components

• The goal to be optimized,
• Constraints, and
• Decision variables

9
Operations Research Models

• Deterministic Models
• Linear Programming Models
• Integer Programming Models
• Network Flow Programming Models
• Nonlinear Programming Models
• Stochastic Models
• Inventory Models
• Queueing Models
• Queueing Networks and Decision Models
• Simulation Models

10
Types of OR Models

• Analytical Models
• The objective and constraints of the models can
  be expressed quantitatively or mathematically as
  functions of the decision variables.
• Simulation Models
• The relationship between input and output of the
  models are not explicitly stated the models
  break down the modeled system into basic or
  elemental modules that are then linked to one
  another by well-defined logical relationships.

11
An M/M/c Queueing System
12
Performance Measures of Queueing Systems
Arrival rate
Departure rate
Server utilization
Expected number of customers in queue
Expected number of customers in system
Expected waiting time in queue
Expected waiting time in system
13
Total Cost of A Queueing System (Taha2011)
Cost of Waiting
Total Cost
Cost of Operation
Cost Per Unit Time
Optimum Number of Servers (Tellers)
Number of Servers (Tellers)
14
Queueing Systems vs. Field Operations

• Queueing Systems
• The customers come to the servers
• The system is small and simple
• No traveling time involved

• Field Operations (Personal Visits)
• The servers (interviewers) go to the customers
  (respondents)
• The system is very large and complicated
• Server traveling time

15
Inbound vs. Outbound Telephone Call Centers

• Outbound
• Telemarketing
• Telephone Surveys
• Charities
• Politicians
• Some Companies

• Inbound
• 800 Customer Services
• Help Desks

16
Outbound Telephone Dialing Systemas a Closed
Queueing Network (Samuelson1999)
NA
D
Service Facility
Queue or Waiting Line
Party Does Not Answer
1
W
2
x x x x x x
A
Waiting To Dial
. . .
Party Answers
c
Lines with Parties Who Hang Up or Get Turned Away
N
S
R
17
Outbound Telephone Dialing SystemDecision
Variables

• Amount of time to anticipate service completions
• Obtaining the new party too early, resulting in
  an abandoned call and the need to start dialing
  again
• Cost of waiting too long, resulting in
  unnecessary idle time for the representatives
• Number of calls to attempt at once
• Two or more answer, we will have one or more
  abandoned calls
• None answers, we will have idle representative
  time

18
Objectives

• Develop a valid method of predicting cost,
  response rates, and timing of new or continuing
  surveys for the field operations.
• The simulation modeling will be followed by the
  optimization of the field operations if a
  simulation model is feasible and valid.

19
Definition of Discrete-Event Simulation

• Event Driven Each occurrence of an event changes
  the state of the system
• Using a model (implemented as a computer
  program), rather than experimenting with a real
  system

20
Steps of Simulation Study (Banks 1998)

• Model Conceptualization
• Data Collection
• Input Data Analysis
• Model Translation
• Verification and Validation
• Experimental Design
• Production Runs and Output Analysis

21
Model Conceptualization

• Problem Formulation
• Objectives and Project Plan
• The modeling begins simply and the model grows
  until a model of appropriate complexity has been
  developed with the objectives in mind.

22
Data Collection

• A data set for each variable from a survey is
  collected.
• Whenever possible, collect between 100 and 200
  observations.
• Collect a number of samples from different time
  periods, such as field operations (time of day
  and/or day of week)

23
Input Data Analysis and Modeling

• Assessing Independence
• Probability Plots
• Estimation of Parameters
• Goodness of Fit Tests
• Empirical Distributions
• Simulation Support Software
• ExpertFit (A. M. Law and Associates)
• StatFit (Geer Mountain Software Corporation)

24
Model Translation

• The conceptual model constructed is coded into a
  computer-recognizable form, an operational model.
• General-Purpose Software
• Manufacturing-Oriented Software
• Business Process Reengineering
• Simulation-Based Scheduling
• Field Operations? C, FORTRAN?

25
Random Number and Random Variate Generation

• Random (pseudorandom) numbers between 0 and 1
  from the uniform distribution, U(0,1) or RN(0,1)
• Use Inverse Transform Method to obtain a random
  variable, X

otherwise
26
Verification and Validation

• Verification concerns if the operational model is
  performing properly.
• Validation is the determination that the
  conceptual model is an accurate representation of
  the field operations (or the real system).

27
Verification and Validation Process

• It is an iterative process
• Add new details to the model
• Run the model
• Evaluate the results
• The results are not sufficiently accurate
• Identify other details (operations/input data)
• Go to step 1 and the cycle starts anew
• At some point, the model is determined to be
  close enough

28
Experimental Design

• For each scenario that is to be simulated,
  decisions need to be made concerning the length
  of the simulation run, the number of runs (also
  called replications), the manner of
  initialization, and controllable decision
  variables as required.

29
Production Runs and Output Analysis

• Production runs and their subsequent output
  analysis are used to estimate the performance
  measures (cost, timing, and response rates) for
  the scenario that are being simulated.
• Finite-Horizon Simulations
• Steady-State Simulations

30
Simulation Model of Simplified NHIS Field
Operations (Prototype)

• Ten FRs, 1050 cases, 105 cases per FR
• Each FR covers a PSU of 60 x 60 square miles
• FRs are given 17 days starting from a Monday
• All FRs start to work at 300 PM each day
• 2004 NHIS CHI data set for input modeling
• Visiting order Traveling Salesman Problem
• The model about 1900 lines of C code

31
Field Operation Inputs

• Frequency distribution of 28 outcomes
• Interview length distributions by outcomes
• Contact/No-Contact Bernoulli distribution
• Contact time distributions
• Uniform distributions for vehicle speed

32
Software Development for Field Operations
Simulation Modeling
Field Operations Inputs
Input Modeling
Field Operations Simulation Model
Output Analysis
Response Rates
Costs
Timing
33
Performance Measures

• Low Cost Direct Labor Cost (Hours and Mileage)
• Average number of personal visits per case
• High Response Rate
• Short Timing How long it takes each month (17
  days)
• It is called LHS

34
Preliminary Results

• 1000 independent replications with different
  seeds
• Cost 25,475
• Based on 10/hr and 0.35/mile
• Average number of PV 1.74
• Response Rate 86.04
• Timing 17 days

35
Response Rates of 2004 NHIS Q2
Region RR() Region RR()
Boston 86.09 Charlotte 90.62
New York 76.40 Atlanta 91.72
Philadelphia 82.86 Dallas 87.23
Detroit 93.46 Denver 92.04
Chicago 91.21 Los Angles 87.32
Kansas City 93.48
Seattle 86.99 National 88.63
36
Design of ExperimentsControllable Parameters

• Starting time1000 AM, 1200 noon, and 300 PM
• Number of FRs 10 and 15
• Timing 17 days vs. 11 days
• Area 3600 vs. 2401 square miles
• Cases per FR 105 vs. 70
• FR-Days 170 vs. 165

37
Selected Frequency Distributions of Contact
(C)/No-Contact (NC)
Hours Sun Mon Tue Wed Thur Fri Sat All
C NC 1000 1200 49.02 50.98 51.54 48.46 49.75 50.25 50.67 49.33 53.62 46.38 51.09 48.91 55.17 44.83 51.88 48.12
C NC 1200 1500 52.97 47.03 50.63 49.37 51.10 48.90 51.22 48.78 50.31 49.69 51.96 48.04 54.25 45.75 51.64 48.36
C NC 1500 2000 51.95 48.05 55.50 44.50 56.05 43.95 56.94 43.06 56.26 43.74 53.86 46.14 51.77 48.23 55.32 44.68
38
The Six Parameter Settingsfor the Experiments
S. T. FRs Days Area FR-Days Adj. Days
1 1000 10 17 3600 170 17.00
2 1200 10 17 3600 170 17.00
3 1500 10 17 3600 170 17.00
4 1000 15 11 2401 165 11.33
5 1200 15 11 2401 165 11.33
6 1500 15 11 2401 165 11.33
39
The Estimates of the PMs of the Six Parameter
Settings
Adjusted to 170 FR-Days
Cost() RR() AVs Cost() RR() AVs Saved()
1 25,375 86.19 1.72 25,375 86.19 1.72
2 25,238 86.86 1.71 25,238 86.86 1.71
3 25,475 86.04 1.74 25,475 86.04 1.74
4 20,722 82.23 1.68 21,349 84.72 1.73 15.86
5 20,575 83.50 1.66 21,199 86.03 1.71 16.00
6 20,589 83.88 1.67 21,213 86.42 1.72 16.73
40
Federal Statistics in the FY 2010 Budget

• Source http//www.copafs.org/reports/federal_stat
  istics_in_the_fy_2010_budget.aspx

(Total direct funding in millions) FY2008 Actual FY2009 Estimate FY2010 Request
Census Bureau Current Programs 232.8 263.6 289.0
Census Bureau Periodic Programs 1,234.0 3,906.3 7,115.7
Others 1,217.7 1,330.5 1,431.0
Total 2,684.5 5,500.4 8,835.7
41
Cost Estimates of the Replication with Seed 169001
Setting Total Time (hours) Wages () Total Distance (miles) Mileage () Total Cost ()
3 1,349.37 13,494 35,012 12,254 25,748
6 1,107.52 11,075 27,101 9,486 20,561
6(adj) 1,141.08 11,411 27,922 9,773 21,184
42
The Other Three Parameter Settingsfor the
Experiments
S. T. FRs Days Area FR-Days Adj. Days
1 1000 10 17 3600 170 17.00
2 1200 10 17 3600 170 17.00
3 1500 10 17 3600 170 17.00
7 1000 15 17 2401 255 11.33
8 1200 15 17 2401 255 11.33
9 1500 15 17 2401 255 11.33
43
The Estimates of the PMs of the Other Three
Parameter Settings
Cost() RR() AVs Cost Saved() RR Gain()
1 25,375 86.19 1.72
2 25,238 86.86 1.71
3 25,475 86.04 1.74
7 24,545 89.93 1.78 3.27 3.74
8 24,085 89.96 1.75 4.57 3.10
9 23,926 89.98 1.75 6.08 3.94
44
Optimum number of FRs
45
Conclusions

• Simulation models can be used for optimizing
  field operations
• Smaller PSU area is more cost effective
• Less time on the roads and more time knocking on
  the doors
• Not at the expense of the response rate
• Field operations can be completed sooner

46
Microsimulation of NHIS

• Physical Impediments and At-Home Patterns of
  Households
• Interviewer Strategies
• Multiple Visits of Completed Interviews
• Unrelated Persons Living in the Same House
• Classification of Interviewers
• Multiple Surveys
• Sample Designs

47
What Next?

• Most Recent NHIS CHI Data
• Classification of PSUs
• Population Densities
• Car Densities
• Traffic Statistics
• Development of A Simulation Language for Field
  Operations?

48
Simulation and Modeling Textbooks

• Law and Kelton Simulation Modeling and Analysis.
  3rd edition, 2000, McGraw-Hill
• Jerry Banks, Editor Handbook of Simulation.
  1998, Wiley Sons
• Hamdy A. Taha Operations Research An
  Introduction. 9th edition, 2011, Prentice Hall
• Hillier and Lieberman Introduction to Operations
  Research, 8th edition, 2005, McGraw-Hill

49
Operations Research Models

• Deterministic Models
• Linear Programming Models
• Integer Programming Models
• Network Flow Programming Models
• Nonlinear Programming Models
• Stochastic Models
• Inventory Models
• Queueing Models
• Queueing Networks and Decision Models
• Simulation Models
• Field Operating Models?