NES Induction Module for New Supervisors: Pilot Evaluation DROSS Ambleside July 27th 2006

1
NES Induction Module for New SupervisorsPilot
EvaluationDROSS AmblesideJuly 27th 2006

• Dr Gellisse Bagnall
• Educational Development Manager
• NHS Education for Scotland

2
NES Induction Module for New Supervisors

• Overview
• - rationale for module
• - content and delivery
• - current status
• Pilot Evaluation
• - Method
• - Results
• - Discussion

3
Background

• Clinical Psychology training in Scotland
• SCPMDE - - - - - 1999
• DClinPsy
• NES MSc (2005)
• (2002) (Psychological Therapy in Primary
  Care)
• MSc (2006)
• (Applied Psychology for Children and
• Young People)

4
Rationale for Module Devpt

• Recommendations from education and training needs
  analysis of all clinical psychologists in NHS
  Scotland (Allen, Bagnall and Campbell, 2003)
• Inc. Pressures on trainee supervision
• Inc. n on training courses
• Introduction of flexible training (Edinburgh
  course)

5
Module Development

• 2004-2005
• Development of module content, in
  collaboration with project Steering Group
  (incl. HOPS, Clinical Tutors)
• Definition of Learning Outcomes (c/f DROSS)
• Development of framework for Supervisory
  Competencies

6
Educational Philosophy Delivery

• Principles of adult learning and constructivism
• Active and participative learning
• Relevance experiential learning/theory to
  practice links
• Blended learning Face-to-face online (VLE)
  delivery
• Flexible learning
• Collaborative learning (on-line small group
  activity)
• Reflective learning reflective journal and
  personal learning portfolio

7
Supervisory Competencies External Accreditation

• Competency framework developed for project
  linked to learning portfolio
• Pilot participants submitted portfolio of
  evidence for external review
• Ongoing dialogue re external
• accreditation (portfolio?)

8
Overview of Module

• Structure
• 2-day face-to-face workshop
• Virtual Learning Environment (VLE) 5 months
  of flexible online learning supported at a
  distance
• 1-day face-to-face workshop

9
Overall Aim of Module

• To prepare new supervisors in clinical psychology
  for their role with trainees by developing their
  understanding of effective supervisory practice

10
Module ContentOverview of 2-day F2F

• What is supervision definitions and
  responsibilities
• Approaches to supervision
• Facilitating trainee learning
• Setting educational objectives
• Promoting reflective learning
• Giving constructive feedback
• Familiarisation with course paperwork
• Supervision role play (case study/scenarios)
• Identifying and managing trainee issues (video)
• Introduction to online learning

11
Overview of Online Component
12
Module Pilot

• Target Group DClin Psy graduates from Oct
  2003 (N24)
• Pilot 1 Apr-Nov 05 (N11) Pilot 2
  Apr-Nov 06 (N13)
• Minimum prerequisites
• 18 months post-qualification experience(c/f BPS)
• Protected time for study (40-50 hours over 6
  months)
• Access to PC at work (for online component)
• Responsibility for supervising Clin. Psych.
  trainee

13
Module Evaluation Method(P1)
Cohort 1 (11) Cohort 2 (13) Apr 2005
(t1) Questionnaire (10) Questionnaire
(8) Apr-Nov 2005 Module Feedback - (anon.
Forms n10) Nov 2005 (t2) Questionnaire
(7) Questionnaire (7) (adapted from
Hall-Marley, S.(2001)Supervisor Feedback Form -
www.cfalendar.com(2006) Questionnaire Aim to
collect data on s/a competence in ability to
deliver supervision to trainee
14
Pilot 1 EvaluationResults

• 12 participants recruited (50 of identified
  cohort)
•  
• 8 completed course and submitted completed
  portfolios
•  
• Non-completers (no course certificate awarded)
• 1 dropped out after approx 6 weeks no trainee
• 1 did not engage with VLE, limited activity
  logged online did not attend final F2F
• 1 opted out of Final day at last minute -
  clinical commitments
• 1 completed module but postponed portfolio
  submission problems with access to trainee, and
  thus inadequate evidence for portfolio. (NB
  Also registered for additional PG course)

15
Pilot 1 Evaluation Results(contd)

• Module Content and delivery
• Two-day F2F
• Overall positive feedback
• Specific Action points to consider included
• More small group work on day 1
• Need to make all module requirements more
  explicit PC access, online learning time.
•  

16
Pilot 1 Evaluation Results(contd)

• 2. Online Component / VLE
• Specific action points to be addressed
• gt Clarity re time required
• Encourage gt use of Discussion Forums clinical
  psychologist as facilitator?
• Access speed to website

17
Pilot 1 Evaluation Results(contd)

• 3. F2F Day3 Module Review
• Key messages learned
• Importance of understanding and setting
  boundaries for supervision negotiating
  relationship between supervisor and trainee
• Importance of giving feedback and how to do it
  constructively
• Importance of clarifying and setting learning
  objectives for supervision with trainee, as
  two-way process
• Need to recognise power relationship between
  supervisor and trainee
• Need to allow a shift in boundaries as trainee
  gains in confidence and becomes more independent

18
Pilot 1 Evaluation Results(contd)

• 3. Module Review Key messages (contd)
• Importance of being willing to share personal
  experience as a practitioner with trainee
• Importance of having formal models or a framework
  for supervision
• Recognising importance of getting supervision
  environment right
• This feedback
• Validates the relevance of the course content
• Suggests that learning outcomes were achieved
•  
•         

19
Pre-Post Module Evaluation

• Questionnaire to all cohort, pre and post module
• Aim of Questionnaire To collect data on
  self-assessed competence in ability to deliver
  supervision to trainee (adapted from Hall-Marley,
  2001)
• Competency statements clustered under 4 headings
• Atmosphere for learning
• Supervision style
• Supervision conduct
• Supervision impact
• Each statement self-rated on 3 point scale (no
  confidence, some confidence confident)

20
Pre-Post Module Evaluation (contd)

• Results
• Questionnaire returns at t1 and t2
• 7 participants and 7 comparison group
• Charts show n who changed confidence rating t1 to
  t2
• Inc P n Participants with increased confidence
  rating
• Inc NP n comparison group/Non-Participants
  with increased confidence rating

21
Atmosphere for Learning items of
interest (Items where shift in self-assessed
competence differs between Module participants
and non-participants by at least 2) Item 2
Establishing clear boundaries for
supervision Item 3 Ability to recognise
trainees strengths
22
Item 1 Openly discuss and respect difference in
style, orientation and case conceptualisationIte
m 2 Encourage trainee to question or challenge
supervisors opinionItem 3 Allow trainee to
structure sessionsItem 5 Make supervision a
collaborative exerciseItem 7 Admit errors or
limitations without undue defensivenessItem 8
Enable relationships to evolve from advisory to
consultative to collegial 
Supervision Style - items of interest
23
Supervision Conduct - items of interestItem 1
Make decisions and take responsibility when
appropriateItem 2 Make concrete and specific
suggestions when neededItem 6 Raise ethical
and legal considerationsItem 8 Offer
suggestions appropriate to trainee level of
trainingItem 9 Present theoretical rationale
for suggestionsItem 10 Assist trainee to
integrate different techniques
24
Item 1 Provide teaching that generalises or
transcends individual cases to strengthen
trainees general skill level Item 4
Facilitate trainees self-confidence to accept
new challenges
Supervision Impact items of interest
25
Evaluation Conclusions

• Small number no statistical comparisons and
  thus cannot generalise
• BUT
• Descriptive data suggest
• Participation in Module contributed to increased
  confidence in tasks of supervision relating to
  module content and learning outcomes
• Module achieved its overall aim of preparing new
  supervisors for role with first trainee