KARNATAKA SYSTEM OF EVALUATION HISTORY

1
KARNATAKA SYSTEM OF EVALUATIONHISTORY

• The idea of setting up of an agency for
  evaluation was originated
• with the suggestion of the Planning
  Commission in 1963.
• Evaluations were generally taken up at the
  initiative of
• Senior Govt.Officials, Departmental
  Secretaries and Heads of
• Departments.
• Evaluations were also proposed at the KDP
  meeting at the
• Secretary level held every month by the
  Chief Secretary to
• review departmental programs.
• Till 2000 two departments viz., Evaluation
  Division of the
• Planning Department the Directorate of
  Economics Statistics
• (DES) were called upon to undertake program
  evaluation.

2
PROBLEMS FACED

• Evaluation is generally neglected and results
  of evaluation
• studies are not widely used for improving
  the
• implementation of schemes.
• Evaluation and monitoring have been by and
  large under-funded.
• Lack of involvement of the implementing
  departments in the
• finalization of the study design.
• Lack of interest in providing required
  information for the
• study.
• Lack of interest in providing feedback on the
  evaluation reports to
• evaluating agency.
• Lack of interest in taking follow-up actions on
  the
• recommendations of the evaluation report.

3
INDEPENDENT EVALUATION INITIATIVE (IEI)

• Evaluation holds good for improving qualitative
  aspects of
• various programs, therefore should be given
  adequate
• priority.
• In all the Externally Aided Projects(EAPs)
  about 10 of the
• provision is earmarked for evaluation and
  monitoring.
• It was considered necessary to adopt a similar
  approach for
• all other schemes.
• In November, 2000, after detailed examination
  GOK has
• established an IEI through a Government
  Order.

4
GOVERNMENT ORDER DATED 17TH NOV. 2000

• All Govt. schemes/programs whose Five Year Plan
  outlay is Rs.100
• lakhs and above shall be evaluated before
  the completion of the Five Year
• Plan period through external agencies.
• To earmark after the third year of
  implementation 1 of the FYP outlay of
• the schemes in the annual budget, subject to
  a minimum of Rs.1 lakh and
• max. of Rs. 5 lakhs for evaluation studies
  and monitoring of the
• implementation of recommendations of
  evaluation studies.
• Re-appropriation of this amount shall not be
  permitted.
• To constitute Departmental Evaluation
  Committees(DECs) Evaluation
• Co-ordination Committee(ECC) in place of
  the existing State Evaluation
• Committee.
• Not to continue the schemes not evaluated in
  the next Five Year Plan.
• Continuation of schemes not evaluated have to
  be approved by the ECC.

5
SYSTEM
ECC (Chaired by ACS to Govt.)
DECs (headed by Deptl./Secretaries)
Evaluation Division, PD
External Agencies
Pilot Departments
6
COMPOSITION OF DECs ECC
DECs 1. Secretary of the Dept.
Chairman 2. Heads of the Depts.
Members 3. Senior Dir. Evn.Divn.
Member 4. Internal financial Adviser
Member 5. Rep. of IT Dept.
Member 6. Deputy Sec. of the Dept.
Member-Secy
ECC 1. Addl. Chief Secy. Chairman 2. ACS
DEV Comm Member 3. Secy. of the Dept.
Member 4. Pr. Secy. FD
Member 5. Pr. Secy. PD Member 6.
Sr.Dir. Evn.Divn. Member-Secy
7
FUNCTIONS OF DECs
1. To decide on the plan schemes/ programmes for
evaluation study. 2. To assign the evaluation to
an external agency. 3. To consider and finalize
the evaluation study report prepared by the
external agency. 4. To finalize recommendations
contained in the evaluation study and suggest
follow-up action on the recommendations. 5. To
review the follow-up action taken on the
implementation of the recommendations finalised
in 4 above.
8
FUNCTIONS OF ECC
1. To co-ordinate the annual programme of
evaluation as finalised by DECs. 2. To assign
specific proposals of schemes/programs for
evaluation to the Evaluation Division. 3. To
consider the draft evaluation reports approved by
the DECs and give its approval. 4. To review the
follow-up actions based on recommendations made
in the evaluation reports. 5. To consider and
approve the reports of Evaluation Division. 6. To
examine the list of schemes not taken up for
evaluation by the DECs and decide about their
continuance or otherwise in the next Five Year
Plan with or without getting them evaluated.
9
EVALUATION DIVISION

• Consequent to changes introduced, the functions
  of the Evaluation Division have undergone a
  change.
• It is the servicing agency for the ECC.
• It has assumed the role of coordinating the
  activities under IEI.
• It has to issue guidelines periodically for the
  evaluation process.
• It has identified 20 major departments as pilot
  for IEI in phases.
• It has to assist the DECs in the discharge of
  functions of identifying and entrusting
  evaluation studies to external agencies,
  finalization of the evaluation reports and the
  recommendations of the study.
• It has to update the evaluation database
  created in the website of the PD
  URLhttp//nitpu3.kar.nic.in/planning/evdb/evdbind
  ex0.asp
• It has to update the list of external agencies
  engaged in evaluation and circulate among the
  departments.

10
PILOT DEPARTMENTS

• Prepare concept paper and submit to the DEC for
  
• finalization.
• Circulate the concept paper among the selected
  external
• agencies in order to get their proposal.
• Prepare a comparative statement of the
  proposals received from the external agencies
  and submit to the DEC for selecting a suitable
  agency.
• Write the TOR and sign with the external
  agency.
• Provide required information to the agency and
  feedback on the draft report of the agency.
• Present the findings and recommendations of the
  draft
• report before the DEC for finalization of the
  report and
• recommendations.

11
CONCEPT PAPER

• Concept papers are critical since at this stage
  that the department articulates what its
  hypotheses and main questions are.
• Apart from providing the details of the scheme,
  the concept paper should include.
• Why the scheme is being picked for
  valuation(hypotheses).
• What is hoped will be learnt from
  evaluation(evaluation questions /objectives of
  evaluation).
• How it is planned that this will be
  learnt(Methodology for evaluation/evaluation
  design-case study, survey, cost benefit analysis
  etc.)
• How the information will be used by the
  department concerned.

12
List of Pilot Departments (Secretariat
Departments)
PHASE-1 1. Public Works 2.
Rural Dev. Panchayat Raj 3. Water Resources -
Minor Irrigation 4. Water Resources - Major
Medium Irrigation

• 7. Housing
• 8. Primary Secondary Education
• 9. Women Child Welfare
• 10. Social Welfare
• 11. Urban Development
• PHASE-3
• 1. Commerce Industries
• 2. Revenue
• 3. Co-operation
• 4. Labour
• 5. Information Tourism
• Youth Service

PHASE-2 1. Agriculture Horticulture 2.
Animal Husbandry Fisheries 3. Energy 4. Food,
Civil Supplies Consumer Affairs 5. Forest,
Ecology Environment 6. Health Family Welfare
13
EXPERIENCE PROBLEMS

• From a modest beginning in the early months,
  there is now considerable momentum under the
  initiative.
• The various departments have completed 61
  evaluations within 4 years.
• In terms of expenditure incurred on these
  studies, per study average expenditure works out
  to Rs.3.55 lakhs.
• Most departments voice support for the need for
  conducting evaluations.
• There is a widespread problem of poor quality in
  both concept papers and evaluation reports.
• Departments feel under-qualified to prepare
  concept papers particularly with regard to
  specifying the evaluation design.
• External Agencies find the concept papers
  inadequate.
• Evaluation reports are of variable quality
  because of
• Difficulties with identifying suitable agency
• Insufficient clarity on objectives and
  inadequate information provided by the
  departments.
• Lack of refereeing of reports by the experts.

14
EXPERIENCE PROBLEMS (Contd.)

• There is inadequate follow-up, which threatens
  to make the
• entire exercise of conducting evaluation
  studies futile.
• The ECC has played a critical role by reviewing
  and
• encouraging the departments to take actions
  to improve
• program design.
• Few departments have held a series of internal
  discussions on
• policy recommendations and follow-up with
  local level
• (districts)Officials which have let to changes
  in program
• implementation.
• Departments have used the results of the
  evaluation in
• improving the program design, Ex. RDPR,
  Horticulture.