Results of the evaluation and prioritization of projects Strengths and weaknesses

1
Results of the evaluation and prioritization of
projectsStrengths and weaknesses
4th Expert Group Meeting on Developing Euro-Asian
Transport Linkages 21 24 November 2006,
Thessaloniki, Greece

• By the External Consultant
• Dimitrios A. Tsamboulas
• Associate Professor
• Department of Transportation Planning and
  Engineering
• School of Civil Engineering
• National Technical University of Athens (NTUA)

2
Brief Presentation Outline

• Part A Methodology
• Part B Projects database
• Part C Results
• Part D Conclusions

3
Part A Methodology

• Objectives
• Phases
• Analysis of Phases Definitions
• Data Needs, Processes Templates

4
Objective

• Identify projects prioritization/
  categorization,
• support elaboration of a medium and long-term
  investment strategy
• encourage the realization of projects that have
  good chances of implementation
• all projects that are on the EATL routes or being
  extensions of those routes will be considered

5
Phases of Methodology

• Three consequent phases
• PHASE A Identification
• PHASE B Evaluation
• PHASE C Prioritisation

6

• Identification the initial screening process
  that grouped projects in two groups, those with
  committed funding and those without committed
  funding.
• Evaluation of projects without committed funding
  with respect to more specific evaluation
  criteria.
• Prioritisation of the projects -based on the
  screening process and the evaluation results- in
  order to classify them into four specific
  Priority Categories (I, II, III, IV).
• It has to be noted that projects with no
  sufficient data/information were not possible to
  pass the identification phase and were directly
  placed in to a Reserve Priority Category.
• The whole exercise was based on the countries
  reports.

7
Identification Phase

• Within the identification phase, projects were
  grouped according to whether they have committed
  funding or not.
• If a project has already secured necessary
  funding was directly prioritised as Priority
  Category I.
• The rest would pass the evaluation phase
• In the identification phase and based on the
  country reports, the consultants completed
  TEMPLATE 1, which contained the list of projects
  proposed in their country reports, indicating the
  securitization of funds or not.

8
TEMPLATE 1 Identified Projects
9
Data to be collected - between Identification and
Evaluation Phases

• Regardless the case of a project having secured
  funds or not the countries were requested to
  further elaborate this list of projects in
  TEMPLATE 1 and also complete the respective
  TEMPLATES 2, in the following manner
• For projects with funding committed, only some
  additional technical information should be
  completed (Section 1 of TEMPLATE 2).
• For projects without funding committed,
  additional technical information and evaluation
  criteria questionnaire should be completed
  (Section 1 and Section 2, respectively, of
  TEMPLATE 2).
• For newly proposed projects, complete all
  necessary information in TEMPLATE 2.

10
TEMPLATE 2 Road Project Fiche / Section 1
11
(No Transcript)
12
TEMPLATE 2 Rail Project Fiche / Section 1
13
(No Transcript)
14
TEMPLATE 2 Inland waterway Project Fiche /
Section 1
15
(No Transcript)
16
TEMPLATE 2 Ports/Depots/Terminals etc. Project
Fiche / Section 1
17
(No Transcript)
18
TEMPLATE 2 Project Fiche (for all project
types) / Section 2
19
(No Transcript)
20
(No Transcript)
21
(No Transcript)
22
Evaluation Phase

• Selection of Criteria two dimensions of
  criteria
• horizontal dimension or CLUSTER A
• Functionality/ Coherence expresses the role of
  the project in the functionality and coherence of
  the Euro-Asian Transport Linkages (CA)
• vertical dimension or CLUSTER B
• Socio-economic Efficiency/ Sustainability
  expresses the socio-economic return on investment
  (CB)
• Measurement of criteria physical performances
• Quantification of Criteria - Scores
• Weighting/ Hierarchy of Criteria Delphi/Paired
  Comparison
• Total Performance of Project

23
Selection of Criteria

• 1. Serve international connectivity (reaching a
  border crossing point or provide connection with
  a link that is border crossing) (CA1)
• A Greatly improves connectivity, B
  Significantly improves connectivity, C Somewhat
  improves connectivity, D Slightly improves
  connectivity, E Does not improve connectivity.
• 2. Promote solutions to the particular transit
  transport needs of the landlocked developing
  countries (CA2)
• The projects provides solution..
• A Greatly, B Significantly, C Somewhat, D
  Slightly, E Does not

24

• 3. Connect low income and/or least developed
  countries to major European and Asian markets
  (CA3)
• The projects connects..
• A Greatly, B Significantly, C Somewhat, D
  Slightly, E Does not
• 4. The project crosses natural barriers, removes
  bottlenecks, raises substandard sections to meet
  international standards, or fills missing links
  in the EATL (CA4)
• The project crosses natural barriers or removes
  bottlenecks and/ or missing links in EATL..
• A Greatly, B Significantly, C Somewhat, D
  Slightly, E Does not

25

• 5. Have high degree of urgency due to importance
  attributed by the national authorities and/or
  social interest (CB1)
• The project is..
• A In the national plan and immediately required
  (for implementation up to 2008), B In the
  national plan and very urgent (for implementation
  up to 2010), C In the national plan and urgent
  (for implementation up to 2015), D In the
  national plan but may be postponed until after
  2015, E Not in the national plan.
• 6. Pass socio-economic viability test (CB2)
• The project is expected to increase traffic
  (both existing and generated)
• A More than 15, B 10-15, C 5- 10, D less
  than 5, E Will not affect traffic

26

• 7. Have a high degree of maturity, in order to
  be carried out quickly (i.e. project stage)
  (CB3)
• Projects is at stage of
• A Tendering, B Feasibility study, C
  Pre-feasibility study, D Planning, E
  Identification
• 8. Financing feasibility (CB4)
• Projects financing feasibility is..
• A Excellent, B Very Good, C Good, D Medium,
  E Low
• 9. Environmental and social impacts (CB5)
• The project has potentially has negative
  environmental or social impacts (pollution,
  safety, etc).
• A No impact, B Slight impact, C Moderate
  impact, D Significant impact, E Great impact.

27
Measurement of Criteria

• The physical scale of criteria measurement was
  derived by the consultant based on his experience
  from similar studies.
• The physical scale ranges from A to E as
  presented before
• Criteria were measured by direct classification
  (from A to E) performed by the countries (the
  national representatives in the EATL project) by
  completing the evaluation criteria questionnaire
  (Section 2 of TEMPLATE 2 as already presented).

28
Quantification of Criteria

• Then the transformation of criterion measurement
  to the artificial scale (derivation of scores)
  took place.
• According to the quantification of criteria the A
  value is 5 (the highest) in terms of score and
  respectively for value E, is 1 (the lowest).
• Therefore
• where
• J A or B and
• i 1,.,5

29
Criterion Scores from Country Experts

• It has to be noted here, that the good
  communication between the externals and the
  country experts is necessary in order to quantify
  as good as possible all the criteria.
• Nonetheless, for unfunded projects that no
  answers were provided in the evaluation
  questionnaire, the lowest scores were assigned.

30
Weighting/ Hierarchy of Criteria

• By using Paired Comparison Method weights were
  derived. According to policy priorities set out
  from the interviewed experts (the consultants,
  UNECE and UNESCAP) pair wise comparisons of all
  criteria were made
• A standard axiom of most of multicriteria methods
  is that the sum of criteria weights should be 1. 
• Therefore and
• where
• J A or B and
• i 1,.,5

31
Paired Comparison Method

• Paired comparison approach is a scaling approach.
  
• Only one question to be answered is is this
  criterion more important than the other?.
• This means that the paired comparison matrix (see
  Table I next) can be filled with zeros and ones,
  where one represents is more important.
• By adding these values over the column, a measure
  is obtained for the degree to which a criterion
  is important compared to all other criteria, if
  finally these measures are standardised (see
  Formula I next), a set of criteria weights is
  created.

32
Table I An example of Paired Comparison matrix
Standardised score wi (I)
33
Criteria Weights from the Country Experts

• It has to be noted here, that countries (though
  national representatives) may provide their own
  weights, with the proper justification of course.

34
Total Performance of Project

• To derive the projects total score we use the
  following relationship
• T.S.Project
• where
• CJi ? 1,5
• WJi ? 0,1
• J A or B and
• i 1,.,5
• TSProject ? 1,5

35
Prioritization Phase

• The combination of the criterions scores and
  priorities puts each project in one of the four
  priority categories or reserve category.
• If the project already has committed funding, it
  belongs to priority category I.
• If the project scores between 4-5 then it belongs
  to priority category II.
• If the project scores 3 -4 then it belongs to
  priority category III.
• If the project scores 1 -3 then it belongs to
  priority category IV.
• If the project has not pass the pre-selection
  phase then it belongs to reserve category.

36
Priority Categories

• I projects, which have funding secured and are
  ongoing or planned and are expected to be
  completed in the near future (up to2010).
• II projects which may be funded and implemented
  rapidly (up to 2015).
• III projects requiring some additional
  investigations for final definition before likely
  financing (up to 2020).
• IV projects requiring further investigations for
  final definition and scheduling before possible
  financing
• Reserve projects to be implemented in the long
  run, including the projects where insufficient
  data existed.

37
Part B Projects database

• Countries participation
• Coding/Abbreviation
• Number, types of infrastructure and cost of
  projects
• Statistics on projects types and costs

38
Countries participation

• Out of the 18 countries participating in this
  project, 15 countries have submitted data on the
  projects under evaluation.
• Countries that submitted data
• Armenia, Azerbaijan, Belarus, Bulgaria, China,
  Georgia, Islamic Republic of Iran, Kazakhstan,
  Kyrgyzstan, Moldova, Romania, Tajikistan, Turkey,
  Ukraine, Uzbekistan.
• Countries not submitting data
• Afghanistan, Russian Federation ,Turkmenistan

39
Coding/Abbreviations

• Each project is identified with a unique Project
  ID specifying country, transport mode and a
  specific number.
• Abbreviations for country Afghanistan (AFT),
  Armenia (ARM), Azerbaijan (AZT), Belarus (BL),
  Bulgaria (BG), China (CH), Georgia (GE), Islamic
  Republic of Iran (IR), Kazakhstan (KZ),
  Kyrgyzstan (KG), Moldova (MD), Romania (RO),
  Russian Federation (RU), Tajikistan (TJK), Turkey
  (TU), Turkmenistan (TM), Ukraine (UKR),
  Uzbekistan (UZB).
• Abbreviations for type of infrastructure Road
  projects (ROD), Railway project (RLW), Maritime
  projects (MAR), Inland waterway project (INL).
  Inland/border crossing and other projects (INM).
• So a project with an ID such as AZT-RLW-1 is thus
  railway project number 1 in Azerbaijan.

40
Projects submitted in raw numbers

• In total 230 projects were included in this phase
  for a total value of 42.02 bill. of which
• 112 road projects for a total value of 11.9
  bill.
• 68 railway projects for a total value of 22.7
  bill.
• 37 maritime projects for a total value of 5.7
  bill.
• 11 inland waterway projects for a total value of
  1.6 bill and
• 2 inland/border crossing etc. projects for a
  total value of 0,003 bill.
• Table next slide

41
Table 1 Data submitted by countries for all
projects and per type of infrastructure (costs in
mio. )
Table includes only the countries that sent data.
42
Projects submitted statistics

• 48,7 of the Projects are Road projects, for a
  total value of 11.956,07 mio, representing
  28,45 of the total investment cost.
• 29,6 of the Projects are Railway projects, for a
  total value of 22.755,82 mio, representing
  54,15 of the total investment cost.
• 16,1 of the Projects are Maritime projects, for
  a total value of 5.735,45 mio, representing
  13,65 of the total investment cost.
• 4,8 of the Projects are Inland waterway
  projects, for a total value of 1.570,65 mio,
  representing 3,74 of the total investment cost.
• 0,9 of the Projects are Inland/Cross border
  (etc.) projects, for a total value of 3,12 mio,
  representing 0,01 of the total investment cost.

43
No Project type/No All projects

• 230 projects
• 112 road projects
• 68 railway projects
• 37 maritime projects
• 11 inland waterway projects
• 2 inland/border crossing etc. projects

Project type cost/Cost of all projects

• total value 42.02 bill.
• road projects 11.9 bill.
• railway projects 22.7 bill.
• maritime projects 5.7 bill.
• inland waterway projects 1.6 bill
• inland/border crossing etc. projects 0,003
  bill.

44
Part C Results

• Combined analysis - in raw numbers and statistics
  -of
• Prioritization results all and per
  infrastructure type
• Cost analysis all and per infrastructure type

45
Prioritization results and cost analysis (for all
and per type of infrastructure) raw numbers
All costs in mio Projects received priority
categorisation IV, due to lack of data
46
Prioritization results and cost analysis (for
all) statistics

• 56,5 of the Projects belong to Priority Category
  I, for a total value of 20.602,79 mio,
  representing 49,03 of the total investment cost.
• These projects have secured funding
• 7 of the Projects belong to Priority Category
  II, for a total value of 13.244,02 mio,
  representing 31,52 of the total investment cost.
• For these projects funding was not secured but
  the national representatives have sent sufficient
  data/answers on criteria valuation and thus it
  was possible to perform multi-criteria evaluation
  method and derive priority
• 5,7 of the Projects belong to Priority Category
  III, for a total value of 3.058,35 mio,
  representing 7,28 of the total investment cost.
• For these projects funding was not secured but
  the national representatives have sent sufficient
  data/answers on criteria valuation and thus it
  was possible to perform multi-criteria evaluation
  method and derive priority
• 31,3 of the Projects belong to Priority Category
  IV, for a total value of 5.115,95 mio,
  representing 12,17 of the total investment cost.
• For these projects funding was not secured and
  the national representatives have not sent
  sufficient data/answers on criteria valuation and
  thus the consultant assigned directly the lowest
  score and derived the lowest priority

47
No Project per Priority Category/No All projects

• 230 projects
• 130 in Priority Category I
• 16 in Priority Category II
• 13 in Priority Category III
• 72 in Priority Category IV

Cost of Projects per Priority Category/ Cost of
all projects

• All Priorities - total value 43.4 bill.
• Priority Category I, 20,6 bill.
• Priority Category II, 13,2 bill.
• Priority Category III, 3,05 bill.
• Priority Category IV, 5,1 bill.

48
Statistics concerning Road Projects priorities
and cost

• 80,4 of the Road projects belong to Priority
  Category I, for a total value of 10.175,47mio,
  representing 85,11 of the total investment cost
  for Road projects.
• 1,8 of the Road projects belong to Priority
  Category II, for a total value of 640 mio,
  representing 5,35 of the total investment cost
  for Road projects.
• 8 of the Road projects belong to Priority
  Category III, for a total value of 678,05 mio,
  representing 5,66 of the total investment cost
  for Road projects.
• 10,7 of the Road projects belong to Priority
  Category IV, for a total value of 462,55 mio,
  representing 3,87 of the total investment cost
  for Road projects.

49
No of Road projects per Priority Category/No All
Road projects

• 112 Road projects
• 90 in Priority Category I
• 2 in Priority Category II
• 9 in Priority Category III
• 12 in Priority Category IV

Cost of Road projects per Priority Category/Cost
of All Road projects

• All Priorities for Road projects - total value
  11.95 bill.
• Priority Category I, 10,2 bill.
• Priority Category II, 0,64 bill.
• Priority Category III, 0,68 bill.
• Priority Category IV, 0,46 bill.

50
Statistics concerning Rail Projects priorities
and cost

• 38,2 of the Railway projects belong to Priority
  Category I, for a total value of 9.581,2 mio,
  representing 42,1 of the total investment cost
  for Railway projects.
• 16,2 of the Railway projects belong to Priority
  Category II, for a total value of 8.212,42 mio,
  representing 36,09 of the total investment cost
  for Railway projects.
• 5,9 of the Railway projects belong to Priority
  Category III, for a total value of 2.380,3 mio,
  representing 10,46 of the total investment cost
  for Railway projects.
• 39,7 of the Railway projects belong to Priority
  Category IV, for a total value of 2.581,9 mio,
  representing 11,35 of the total investment cost
  for Railway projects.

51
No of Rail projects per Priority Category/No All
Rail projects

• 68 Rail projects
• 26 in Priority Category I
• 11 in Priority Category II
• 4 in Priority Category III
• 27 in Priority Category IV

Cost of Rail projects per Priority Category/Cost
of All Rail projects

• All Priorities for Rail projects - total value
  23,4 bill.
• Priority Category I, 9,5 bill.
• Priority Category II, 8,2 bill.
• Priority Category III, 2,4 bill.
• Priority Category IV, 2,6 bill.

52
Statistics concerning Maritime Projects
priorities and cost

• 16,1 of the Maritime projects belong to Priority
  Category I, for a total value of 253,75 mio,
  representing 4,42 of the total investment cost
  for Maritime projects.
• 5,4 of the Maritime projects belong to Priority
  Category II, for a total value of 4.190 mio,
  representing 73,05 of the total investment cost
  for Maritime projects.
• 78,4 of the Maritime projects belong to Priority
  Category IV, for a total value of 1.291,7 mio,
  representing 22,52 of the total investment cost
  for Maritime projects.

53
No of Maritime projects per Priority Category/No
All Maritime projects

• 37 Maritime projects
• 6 in Priority Category I
• 2 in Priority Category II
• 29 in Priority Category IV

Cost of Maritime projects per Priority
Category/Cost of All Maritime projects

• All Priorities for Maritime projects - total
  value 5,7 bill.
• Priority Category I, 0,2 bill.
• Priority Category II, 4,2 bill.
• Priority Category IV, 1,3 bill.

III
54
Statistics concerning Inland waterway Projects
priorities and cost

• 54,5 of the Inland waterway projects belong to
  Priority Category I, for a total value of 589,25
  mio, representing 37,52 of the total investment
  cost for Inland waterway projects.
• 9,1 of the Inland waterway projects belong to
  Priority Category II, for a total value of 201,6
  mio, representing 12,84 of the total investment
  cost for Inland waterway projects.
• 36,4 of the Inland waterway projects belong to
  Priority Category IV, for a total value of 779,8
  mio, representing 49,65 of the total investment
  cost for Inland waterway projects.

55
No of Inland waterway projects per Priority
Category/No All Inland waterway projects

• 11 Maritime projects
• 6 in Priority Category I
• 1 in Priority Category II
• 4 in Priority Category IV

Cost of Inland waterway projects per Priority
Category/Cost of All Inland waterway projects

• All Priorities for Maritime projects - total
  value 1,6 bill.
• Priority Category I, 0,6 bill.
• Priority Category II, 0,2 bill.
• Priority Category IV, 0,8 bill.

56
Statistics concerning Inland/Border crossing
(etc.) Projects priorities and cost

• 100 of the Inland/Border crossing (etc.)
  projects belong to Priority Category I, for a
  total value of 3,12 mio.

57
Part D Conclusions

• Missing data next steps
• Strengths and weaknesses

58
Data missing

• Three countries have not sent data at all
• Afghanistan, Russian Federation and Turkmenistan
• Half of the countries that submitted projects,
  have not provided all necessary data
  (specifically Armenia, Azerbaijan, Bulgaria,
  Georgia, Kyrgyzstan, Moldova, Romania,
  Tajikistan, Ukraine)
• either for their unfunded projects they have not
  completed Section 2 of TEMPLATE 2 with the
  evaluation criteria (in order to ease the
  evaluation exercise for these projects)
• so for these unfunded projects that no answers
  were provided in the evaluation questionnaire,
  the lowest scores were assigned
• or in some cases they have not provided projects
  costs or if the project is funded or unfunded

59
Strengths and Weaknesses in the EATL exercise

• More than 50 of projects have secured funding
  (Priority I)
• These secured projects represent almost 50 of
  total cost
• In each type of infrastructure (except maritime),
  Priority I projects is the
  majority
• Overall
• Good chances of quick implementation of EATL
  network

S

• 30 of projects belong to Priority IV due to
  lack of data
• The above is very obvious especially for Maritime
  projects
• Too many road projects (48,7), enough railway
  (29,6), some maritime (16,1) and very few inland
  waterway (4,8 ) etc. no balance among
  infrastructure types
• Overall
• serious lack of data (reversible weakness)
  
• unbalance of infrastructure types

W