Title: Practical Implications of Finding Consistent Route Flows Hillel BarGera Purdue University, and BenGu
1Practical Implications of Finding Consistent
Route Flows Hillel Bar-GeraPurdue University,
and Ben-Gurion University of the Negev, Israel
Yu Nie and David BoyceNorthwestern
UniversityMay 20, 2009
- FHWA 2008 TRANSPORTATION PLANNING COOPERATIVE
RESEARCH (DTFH61-08-R-00011)
2Acknowledgements
Sponsor and study participants FHWA Office of
Environment and Planning and six collaborating
transportation planning organizations ____________
__________________________________________________
__________________________________________________
Other research team members Yang Liu, Yucong
Hu _______________________________________________
__________________________________________________
_______________ Volunteers Jeffrey Casello,
Birat Pandey, Robert Tung ________________________
__________________________________________________
______________________________________ Software
vendors Caliper, Citilabs, INRO,
PTV ______________________________________________
__________________________________________________
________________ The authors alone are
responsible for the content and views expressed
in this presentation.
3Travel forecasting challenges
- Influence decisions, be useful!
- Data, data and data, particularly travel times.
- Model choice and model assumptions.
- Calibration and validation.
- Computational quality challenges
- Sufficient precision (convergence) for scenario
comparisons - Reasonable and consistent route flows.
4Comparing total link flows and link costs between
FW (10 iterations) and TAPAS for the Chicago
regional network
5Distribution of deviations in link flowsfor the
Chicago regional network
6Distribution of deviations in link flowsfor the
Chicago regional network
7Solutions evaluated in this study
Six solutions for Chicago are evaluated in this
study, produced by six tools CUBE, EMME-LA,
TransCAD-FW, TransCAD-OUE, VISUM (RB), and
TAPAS. Commercial tools that are not evaluated
are EMME-PG (RB) ESTRAUS (FW, OB) SATURN
(FW, OB, RB) VISUM-Lohse (FW), VISUM-LUCE
(OB).
8Precision of evaluated solutions
- All evaluated solutions are converged to 1E-4.
- A very precise solution (1e-12) produced by
TAPAS is used as a reference, when needed. - Definitions of convergence are not identical,
but are within the same order of magnitude. - Convergence of 1e-4 is chosen because
- Scenario comparisons require at least this
- level of convergence.
- All commercial tools can achieve it.
- This is the current best practice.
9Precision of total link flows
Comparison of total link flows to precise link
flows
10Distribution of residuals of total link flow
11Comparison of convergence of research tools for
the Chicago regional network
12Precision of solutions used in this study
- It is important to separate the discussion of
precision of specific solutions from the
discussion of precision of methods. - All solutions have similar level of precision,
with residuals less than 10 vph on the majority
of links. - Proper precision comparison between methods is
beyond the scope of the current project. - Precision performance of FW-type tools (CUBE,
EMME-LA, TransCAD-FW) is very different from
quick-precision tools (TransCAD-OUE, VISUM).
13Multiple UE route flow solutions
A
2
100
40
40
160
4
D
1
120
60
120
B
3
14Who needs route flows?
- Multi-class models
- Select Link Analysis determine the
distribution of link flows by their origins and
destinations - Estimation of OD flows from link flows
- Derivation of OD flows for a subarea of a region,
e.g. for micro-simulation - License plates surveys
- Validate model results against survey data
- Design a survey to capture travelers at least
twice, or as much travel as possible.
15OD flows through North Ave. Bridge WB
Each point represents vehicle flow per hour for
one OD pair. X - reference solution (TAPAS-3057
ODs) Y - evaluated solution (RG 1e-4)
16OD flows through North Ave. Bridge EB
Each point represents vehicle flow per hour for
one OD pair. X - reference solution (TAPAS-3,376
ODs) Y - evaluated solution (RG 1e-4)
17OD flows through Harlem Ave. SB
Each point represents vehicle flow per hour for
one OD pair. X - reference solution (TAPAS-4,752
ODs) Y - evaluated solution (RG 1e-4)
18OD flows through Harlem Ave. NB
Each point represents vehicle flow per hour for
one OD pair. X - reference solution (TAPAS-5,034
ODs) Y - evaluated solution (RG 1e-4)
19How to choose a single route flow solution?
The condition of Proportionality Same proportions
apply to all travelers facing a choice between a
pair of alternative segments.
Consider the pair of segments 1,2,4 and
1,3,4. First segment proportion is
40/(40120)1/4.
20How to choose a single route flow solution?
The condition of Proportionality Same proportions
apply to all travelers facing a choice between a
pair of alternative segments.
Consider the pair of segments 1,2,4 and
1,3,4. First segment proportion is
40/(40120)1/4.
For travelers from A to D the proportion is
25/(2575)1/4.
21How to choose a single route flow solution?
The condition of Proportionality Same proportions
apply to all travelers facing a choice between a
pair of alternative segments.
Consider the pair of segments 1,2,4 and
1,3,4. First segment proportion is
40/(40120)1/4.
For travelers from B to D the proportion is
15/(1545)1/4.
22The condition of proportionality
Same proportions for the two segments. Origin and
destination do not matter. Previous or subsequent
decisions do not matter.
By proportionality, flow on designated route is
200 (150/200) (160/200) (180/200) 108
23The condition of proportionality
- Reasons
- Simple, reasonable, consistent, stable, and
therefore useful. - Proportionality is testable.
- Are there any other practical suggestions?
Implications The set of routes should be
consistent, meaning that any route that can be
used while keeping the same total link flows,
should be used. No route is left behind
(without a reason).
24(No Transcript)
25Paired Alternative Segments near Lake Shore Drive
Each point represents vehicle flow per hour for
one OD pair. X - Segment 1 Y - Segment 2 all
solutions converged to RG 1e-4.
26Paired Alternative Segments near Lake Shore Drive
Each point represents vehicle flow per hour for
one OD pair. X - seg. 1 flow seg. 2 flow Y
Log (seg. 1 flow / seg. 2 flow)
27(No Transcript)
28Paired Alternative Segments near North Ave.
Bridge
Each point represents vehicle flow per hour for
one OD pair. X - Segment 1 Y - Segment 2 all
solutions converged to RG 1e-4.
29Paired Alternative Segments near North Ave.
Bridge
Each point represents vehicle flow per hour for
one OD pair. X - seg. 1 flow seg. 2 flow Y
Log (seg. 1 flow / seg. 2 flow)
30Effect of convergence on consistency in TAPAS
Each point represents vehicle flow per hour for
one OD pair. X - seg. 1 flow seg. 2 flow Y
Log (seg. 1 flow / seg. 2 flow)
31Conclusions FW-type methods
- Solutions tend to satisfy the condition of
proportionality, although deviations do occur. - Route set consistency is problematic due to small
flows on non-optimal routes. - To achieve the precision needed for scenario
comparisons (1E-4 or better), hundreds of
iterations may be necessary, implying relatively
long computation times.
32Commercial quick precision methods
- Available in VISUM and TransCAD soon in EMME and
CUBE - Important for scenario comparisons of total link
flows - Do not satisfy proportionality and route
consistency, which could be problematic in select
link and similar analyses.
33Research progress
- A new method, TAPAS, has been developed
- Quick-precision assignment
- Reasonably consistent set of routes, mainly at
higher levels of convergence - Satisfaction of the condition of proportionality
- Functionality is limited to research purposes
- Additional experiments are on-going
34Summary
- Route flows are used often in practical
applications, at various levels of aggregation. - Results of select link and similar analyses are
quite different from software to software. - The set of routes should be consistent No
route is left behind. - The assumption of proportionality ensures unique,
consistent and stable route flows.
35Software vendor reactions
- Quick precision is more important than
proportionality. Select flows are more meaningful
at aggregate levels e.g., flow through a
selected link on other nearby links, rather than
by OD. (PTV) - Lack of proportionality for a tiny amount of
traffic is insignificant. (Citilabs) - Proportionality is a potentially useful mechanism
for rendering path flows unique applications for
multi-class assignments as well as behavioral or
empirical validation would lend it credibility.
(INRO)
36Discussion What makes a model useful?
- Proper sensitivity to policy decisions
- Reasonably accurate (i.e. realistic) predictions
- Ability to obtain needed data for inputs,
as well as for calibration and validation - Stability, repeatability, and consistency
- Computational efficiency
- Insights, understanding and accessibility
- Convincing
37Slide Notes
38- Slide 1
- This research is about route flows in the static
deterministic UE model. The research is funded by
FHWA. It began in September 2008, and is
scheduled for one year. The research is conducted
by Marco Nie, David Boyce, and Hillel Bar-Gera. - Slide 2
- Our purpose in this research is to support
decisions about future improvements to travel
forecasting practices. I am glad to say that the
software vendors, who are important leaders of
progress in this field, took a similar point of
view. They offered us help in various ways,
including many useful and productive comments,
which we highly appreciate. - Of course, this does not mean that they
necessarily agree with the content of this
presentation. - We also want to acknowledge the contribution of
several people that worked very hard with us to
prepare the results presented here - Yang Liu and Yucong Hu, Northwestern University
- Jeffrey Casello, Assistant Professor of Planning
and Civil Engineering, University of Waterloo,
Ontario - Birat Pandey, Senior Engineer, PBSJ, Austin,
Texas - Robert S. Tung, RST International, Inc.
-
39- Slide 3
- The main focus of this research is finding route
flows, which is a computational challenge. We
realize that you, as travel forecasting
practitioners, devote most of your time and
efforts to address other important challenges.
Among the many difficult decisions you need to
make, you need to choose which assignment method
to use and for how long to let it run. This is
why practitioners should be aware of assignment
computational challenges. - The main computational challenges in the static,
deterministic, user equilibrium (UE) model are
precision and route flows. In some ways these two
issues are quite intertwined, while in other ways
they are completely orthogonal. - In particular, as far as we know, not much has
been done in practice regarding route flows. On
the other hand, there has been a remarkable
change, almost a revolution, regarding precision
over the last five years or so. - From our point of view, when we asked
practitioners five years ago how many iterations
they use, the answers were 10, or 20, or
sometimes 5. If we suggested that more iterations
might be helpful, the response was that this
would be a complete waste of computer time. - This presentation starts with evaluation of the
precision of a solution obtained by the FW
algorithm in 10 iterations. During the last year
we showed this evaluation to several
practitioners, and they immediately jumped and
said that 10 FW iterations do not provide
sufficient precision for any analysis. - We think that in order to put in context the
issue of route flows, it should be discussed
together with precision. This is why nearly half
of this presentation will be devoted to
precision, and only then we will discuss route
flows. -
40- Slide 4
- This is a comparison of a solution obtained by
the FW algorithm in 10 iterations with a very
precise solution obtained by TAPAS, which is
converged to a Relative Gap of 1e-12. - In theory, total link flows and link costs are
uniquely determined by a UE assignment. Indeed we
see a good match between the link flow results,
but it is not perfect the link cost results are
more problematic. - Slide 5
- We can examine the comparison under the
microscope by considering the distribution of
the differences between the two solutions. Notice
that the difference in flow on the horizontal
axis is in log scale. We see that a difference of
10vph or more, which is not trivial, occurs for
40 of the links. In many applications this
precision is not enough, so more iterations are
needed. -
- Slide 6
- As the number of iterations increases, the
precision increases, and the differences become
smaller, as expected. - The needed level of precision depends on the
application. One way to choose is to pick a
threshold and choose a solution with sufficiently
small tail beyond that threshold. If the
threshold is 10 vph, then 10-iterations solution
is clearly not good enough, but 1000-iterations
solution probably is. - Slide 7
- FW The Frank-Wolfe or Linear Approximation (LA)
method - RB route-based method
- OB origin-based method
-
- Slide 9
- As you can see, the match in total link flows,
with the reference TAPAS solution (1e-12), is
quite good in all six evaluated solutions.
41- Slide 8
- The level of convergence is defined in term of
the Relative Gap (RG). - Slide 9
- As you can see, the match in total link flows,
with the reference TAPAS solution (RG 1E-12),
is quite good in all six evaluated solutions. - Slide 10
- Examination of the differences in total link
flows from the reference solution shows that all
six solutions are fairly precise, with only a
small tail of differences above 10 vph. This
evaluation gives us the confidence that the
conversion of inputs to all the software was done
properly, which is not a trivial thing, and that
the subsequent comparison of select link analyses
results are valid. - According to this figure the precision of all six
solutions is in the same order of magnitude. This
does not mean that the methods have similar
precision performance, because in order to
compare methods we need to consider CPU time.
Performing such a comparison between commercial
software in a proper manner is far beyond the
scope of this project. To give you an idea about
the possible differences between methods we show
here a comparison of convergence vs. CPU time for
several research tools. - Slide 11
- We can see here that modest levels of precision
can be obtained fairly quickly by several
different methods, including FW. When higher
precision is needed, the computation time for FW
increases dramatically, while other methods can
achieve high precision fairly quickly. We refer
to such methods as quick-precision methods. - Slide 12
- To summarize our discussion about precision, here
are the main conclusions.
42- Slide 12
- To summarize our discussion about precision, here
are the main conclusions. - Slide 13
- It is quite well known that under the UE
assumption route flows are not unique. Here is a
simple example to explain why. Suppose that the
total link flows indicated here represent perfect
UE solution, for which the two segments from 1 to
4 have exactly the same cost. If we switch one
vehicle from A that uses the segment through 2
with a vehicle from B that uses the segment
through 3 the total link flows remain the same,
so link costs remain the same and the perfect
equilibrium situation also remains. In this table
we can see 3 different route flow solutions and
all of them correspond exactly to the same total
link flows shown above. - Slide 14
- In many practical applications, for example in
most cost-benefit analyses, we are interested
only in the full aggregation of route flows to
total link flows. It is quite rare to find
practical application where fully disaggregate
route flows are needed. But there are quite a few
applications where various different intermediate
levels of aggregation are needed. A few of them
are listed here. The important point is that
different route flow solutions may lead to
different answers in each of these partially
aggregated analyses. - Slide 15
- One of the most typical partially aggregated
analyses is select link analysis. In this
analysis we want to know the breakdown of a flow
on a single link by OD pair. We can see here a
comparison for one link in the Chicago network
between the six evaluated solutions and the
reference solution. OD flows on both axes are in
log scale. Points along the axes represent values
below 1E-4, including zeros. On the left you see
the three FW-type methods, and on the right you
see the quick-precision methods.
43- Slide 15 (continued)
- If we compare the number of ODs identified by
the various solutions, these numbers are quite
different from each other. (In the reference
solution there are 3,057 ODs that use this
link.) So clearly the sets of ODs using this
link are quite different in all the solutions. - If we focus on the comparison of OD flows through
this link, and particularly the larger flows,
evaluated solutions from the three FW-type
methods as well as from TransCAD OUE and TAPAS
are quite similar to the reference solution,
while the Visum solution is slightly different. -
- Slide 16
- Considering the same link in the other direction
we see that a match with the FW type methods and
a mismatch with the quick-precision methods. (In
the reference solution there are 3376 ODs that
use this link.) - Slide 17
- For a completely different link on Harlem Ave. we
get fairly similar patterns. (In the reference
solution there are 4752 ODs that use this link.) - Slide 18
- Considering the same link on Harlem Ave. in the
opposite direction we find a mismatch with all
the methods. (In the reference solution there are
5034 ODs that use this link.) - This small sample of 4 links out of 40,000 was
chosen fairly arbitrarily, and is not necessarily
statistically representative. Even so, it is
enough to conclude that differences between
solutions at the select link analysis level do
occur.
44- Slides 19-22
- It would be useful to find a way to choose one
specific solution out of all the many options. - One way to do that is by the condition of
proportionality, which is explained in these four
slides. - Slide 23
- The main reasons to adopt proportionality are 1)
a reasonable condition that is easy to
understand, implying consistent treatment which
may be important when equity issues are present,
and 2) provision of stable solutions with respect
to model inputs. All of these properties make the
resulting model quite useful. - As we will see soon, it is possible to test
whether any particular method satisfies
proportionality or not. - And the only other existing alternative is to
make a completely arbitrary choice. - An important implication of proportionality is
that any route that can be used under the UE
condition, should be used. For example, in the
previous slide there are 8 routes under
proportionality all of them are used. So a
precondition to satisfying proportionality is to
make sure that no route is left behind, unless
of course it is not a minimum cost route. We
refer to this property of the set of routes as
consistency. - Slide 24
- The assumption of proportionality is based on
pairs of alternative segments. In the Chicago
model there are 5000 basic pairs of alternative
segments, which can be used to construct all
other pairs of alternative segments. Here is one
of them.
45- Slide 25
- In each evaluated solution we found the breakdown
of the flow on the two segments by OD. So each
point here represents a single OD, and the
horizontal and vertical axes represent the flows
on segments 1 and 2 respectively. If the same
proportions apply to all OD pairs, all the points
should fall on a straight line (with a slope of
45 degrees). Both axes are in log scale. Points
along the axes represent values below 1E-4,
including zeros. - A fairly straight line is observed for all
FW-type solutions, especially for higher flows,
as well as the evaluated TAPAS solution (RG
1E-4). - For TransCAD OUE we see three main lines, each
line corresponds to a different origin. This
means that within each origin proportionality is
maintained, but between origins proportions are
not the same. The Visum solution in this case is
quite extreme, where only one OD pair uses both
segments. - Slide 26
- Another way to look at the same data is shown
here, where the horizontal axis shows the sum of
flow on both segments in log scale, and the
vertical axis shows the log of the flow ratio.
Under proportionality the flow ratio should be
constant, so the log of the flow ratio should be
constant, so all OD pairs should be on a
horizontal line. This is pretty much the case for
TAPAS. It is more or less the case in the FW-type
solutions for the higher flow values. - For the VISUM solution most OD pairs have all
their flow either on segment 1, with log ratio of
infinity, or on segment 2, with log ratio of
minus infinity. So clearly proportionality does
not hold. In the TransCAD solution we see a group
of ODs that use only segment 1, and three other
groups corresponding to three origins, each
having its own ratio.
46- Slide 27
- Here is another pair of alternative segments we
examined. - Slide 28
- Again we see linear lines for all FW-type
methods, but not for the three quick-precision
methods, including the evaluated TAPAS solution
(1E-4). - Slide 29
- Using the log ratio plots further enhance the
same conclusions regarding FW type solutions. All
three quick-precision methods suffer from
substantial inconsistency, as many ODs use only
one segment out of the two. When ODs use both
segment, in the TAPAS solution the proportions
are the same, while in the commercial
quick-precision methods each OD has its own
proportion. - The two pairs of alternative segments do not
necessarily represent all 5000 other pairs in
this model. They do offer an idea for what might
be expected in other cases. - Slide 30
- Consistency in TAPAS solutions improves
considerably with convergence. Nearly perfect
consistency is shown here at relative gap around
1E-9, and a noticeable progress is shown already
at relative gap around 1E-7. Notice that reaching
these higher levels of precision does not require
too much computation time. - Preliminary experiments with commercial
quick-precision tools did not demonstrate
improvement in consistency or proportionality at
higher levels of precision, but additional
exploration is needed to verify these
observations.
47- Slide 33
- At present, TAPAS exists only as a research code.
As such its functionality is limited to research
needs. It did not go through the extensive
testing expected from commercial products. The
results from TAPAS demonstrate the potential of
incorporating proportionality into assignment
methods. The results are not perfect, so there
are possibilities for future improvements. - Slide 35
- We agree that quick precision is more important
than proportionality, but proportionality is also
important. - We plan to study other levels of aggregation in
the future. - We agree that small flow values are less
important, the problem in practice is how to tell
whether the flows are small and should be
attributed to solution imprecision, or whether
they are not so small and represent something
else. - We certainly appreciate the positive reaction
from INRO. - Slide 36
- We think that the main consideration when
choosing a model is its usefulness. The main
criterion for usefulness is the model ability to
support decision processes. All other criteria
are derived from this one. Reasonable realism is
obviously important. All else being equal, a more
realistic model should be preferred. However,
there are many other considerations, so in most
cases not all else is equal. As a result, in some
cases a more realistic model is not more useful.
More important to our discussion, if several
methods produce solutions that are equally
realistic, other criteria should be considered to
choose the most useful method. We believe that
all other considerations, and particularly the
ability to understand why the method chose a
specific solution, and the ability to explain
that to others, indicate that solutions that
follow the proportionality condition are more
useful. Thank you.