Developmental Reviews at Qassim University

1
Developmental Reviews at Qassim University

• National Commission for Academic Accreditation
  Assessment
• Briefing Meeting, April 13, 2009

2
Reminder

• To achieve its purposes the processes followed in
  the developmental review must be as close as
  possible to what will be required for
  accreditation.
• Main Elements
• Self Study Reportsbased on evidence, not just
  opinions.
• Supporting documents and related reports.
• Independent review involving analysis of
  documents, and site visits.

3

• Role of the external review panels is to verify
  conclusions, make suggestions and recommendations
  and (in a real accreditation, recommend on
  accreditation.
• To do this they need evidence, and will carry out
  their own investigations by examining documents,
  visits to facilities, consultations with faculty,
  students, graduates, employers, etc.

4
Key Dates for Developmental Reviews

• April 30, 2009 Final Self study reports and key
  documents sent to NCAAA.
• May 15, 2009 Self study reports sent to external
  reviewers.
• June 30, 2009 Responses to queries
• October 17 to 29, 2009 Site visits by
  reviewers.
• October 17 to 22 (Institution)
• November 24 to 29 (Programs)
• After that
• Completion of draft reports.
• Check on factual accuracy.
• Final Reports.
• Response to recommendations

5
Activities of External Reviewers after Receiving
Material

• Study reports, evaluate processes and evidence of
  quality that is provided, and identify gaps in
  evidence or issues requiring investigation.
• Teams exchange views, form preliminary opinions,
  and agree on responsibilities for detailed
  investigation.
• Additional information may be requested.
• Teams meet or consult to develop strategy for
  investigation.
• Material provided must include sufficient
  reliable information for these tasks to be
  carried out.

6
Evaluation Criteria

• Achievement of appropriate goals and objectives
• Consistency of all processes and learning
  outcomes with generally accepted standards of
  good practice. (NCAAA Standards)
• Programs in professional fields will need to be
  consistent with what is normally expected in
  those fields, with appropriate adaptations for
  the Saudi context.
• For any conclusions, expect to be asked How do
  you know? Objective evidence required.

7
Documents Required

• For the Institutional Review
• Self Study Report
• Completed self evaluation scales
• Strategic Plan and if a separate document,
  Strategic plan for quality
• Current student handbook
• Available at the institution for consideration
  during the site visit
• Faculty handbook or equivalent with details of
  policies, procedures, PD, etc.
• Administrative / financial policies and
  procedures
• QA manual or equivalent document describing QA
  processes
• Data on faculty qualifications and experience,
  staff student ratios (total and by sections),
  research data (optional for a college), staff CVs

8
Documents Required

• For the Program Reviews
• Self Study Report
• Completed self evaluation scales
• Program Specification and annual program report
• Program description from handbook
• Available at the institution for consideration
  during the site visit
• Course specifications and course and program
  reports
• College/Department handbook or equivalent
• CVs for staff and summary of staff qualifications
  
• Survey responses from students, and other
  stakeholders (if available)
• Statistical data on evaluations by students,
  employers (if available)
• Data on employment outcomes
• Samples of student work and assessments

9
Site Visit Process

• A schedule for meetings and consultations will be
  worked out in advance by the NCAAA in
  consultation with the teams, and with the
  institution.
• The schedule provides for meetings dealing with
  all standards and service providers, involving
  teaching staff, students, visits to facilities,
  meetings with employers and graduates, etc.
• Strict rules for conduct of meetings. (timing,
  no phones, etc)
• Follow up meetings and visits for detailed
  further investigations and informal contacts,
  assessment of student work, faculty research
  reports or ?????.

10
Documents of Self Study Reports

• Agreed process was for earlier draft documents to
  be revised taking account of initial NCAAA
  feedback and provided by beginning of April.
• Further comments provided by NCAAA
• Revised final documents provided by 30 April for
  distribution to reviewers.

11
Some General Problems

• Individual issues will be mentioned in later
  discussions but
• Complete self evaluation scales must be provided.
• Over-reliance on general evaluative opinions
  without detailed descriptions of processes or
  specific data. Opinions are useful in many (but
  not all cases) but dont replace evidence. (Were
  details of standards considered or just headings
  and sub-headings?)
• In many cases statistics not provided or
  analyzed. (Completion and progression data very
  poorly done)
• Explanations of processes usually not given.
  These must be described and evaluated or there is
  no basis for the external reviewers to study them
  and form their opinion about them. A general
  conclusion in the self study without a
  description of what is done is of very little
  value and gives external reviewers nothing to
  evaluate.

12
Some General Problems

• There are no descriptions of how relationships
  between male and female sections are managed or
  how effective they are. When both sections are
  involved specific evaluative data should be
  provided for each section, and then comments made
  about overall performance and significance of any
  differences.
• There were general references to indicators and
  occasionally to benchmarks, but these have often
  not been used as a basis for evaluation.
• In the starring system three stars is regarded as
  the minimum acceptable level of performance.

13
Presentation of Documents

• Provide each of the required documents as a
  separate document in a folder dealing with the
  review concerned.
• Name of folder and of each document should
  indicate the document and the program concerned
  (or the institution)
• Present materials in word format.
• Self study (periodic report) should be a complete
  self contained document. Use attachments or
  appendices if necessary (including any large
  documents) but wherever possible include tables,
  charts etc in the main document with whatever
  comment or discussion is needed.
• In templates, retain headings but delete
  instructions to you unless they are necessary to
  explain what you have done.
• Front cover of each document should clearly
  indicate the program concerned.

14
What Next? (by April 30)

• Make sure complete self evaluation scales have
  been completed. (Use the correct scalesmost
  recent version of program or institutional
  scales)
• Review self study reports taking account of
  comments made here and in later discussions.
• Check format.
• Provide to Quality Center for checking and
  forwarding to the NCAAA.