Design, implementation and evaluation of a - PowerPoint PPT Presentation

1 / 36
About This Presentation
Title:

Design, implementation and evaluation of a

Description:

... (before and after using the ePortfolio) ... educational and career development Quality Assurance Agency for Higher Education, ... (Year 4 SSC) Encouraged ... – PowerPoint PPT presentation

Number of Views:77
Avg rating:3.0/5.0
Slides: 37
Provided by: PaulD205
Category:

less

Transcript and Presenter's Notes

Title: Design, implementation and evaluation of a


1
Design, implementation and evaluation of a
'generic' ePortfolio the Newcastle experience.
Simon Cotterill, Tony McDonald, Paul Drummond,
Geoff Hammond School of Medical Education
Development University of Newcastle, UK
2
Background
SST (Student Support Tutoring)
1st Newcastle-Nottingham PARs project, 1998-2000
  • Internet-PARs
  • DfES funded
  • PDP focus
  • FDTL-4 project
  • ePortfolios in Medicine
  • Evidencing outcomes
  • PDP

2nd Newcastle-Nottingham PARs project, 2000-02
3
Background
  • An FDTL-4 project developing ePortfolios for
    undergraduate Medicine (2002-2005)
  • Newcastle University (lead site)
  • Leeds University
  • Sheffield University
  • Dundee University (for consultancy)

Project funded by
4
Aims
1. Personal Development Planning (PDP) "a
structured and supported process undertaken by an
individual to reflect upon their own learning,
performance and/or achievement and to plan for
their personal, educational and career
development Quality Assurance Agency for
Higher Education, 2001.
5
Aims
2. Supporting Life-Long Learning (LLL)
  • b) Encouraging the development of LLL Skills
  • Evidencing Learning Outcomes
  • Transition to Independent learning
  • Preparation for professional requirements
  • - Appraisals
  • - Assessment
  • - Revalidation

6
Aims (technical)
3. Design a generic ePortfolio -transferable /
useful in different contexts
4. Integration with VLEs
7
Architecture
Generic framework
Developed with Open Source products Zope, MySQL
8
Generic framework Customisaton
Selecting tools by Course and Year
Course Admin view
Student view
  • create context-specific
  • tools via simple Web forms

9
Generic framework Customisaton
Configuring learning outcomes / skills sets
Course Admin view
10
Generic framework Sharing
11
Piloting Implementation The ePortfolio
developed at Newcastle has been applied in the
following contexts
2003/4 Undergraduate Medicine Contract Research
Staff (CPD) Generic Portfolio (project
demonstrator)
2004/5 Dentistry (Undergrad Vocational
Training) Undergraduate Biosciences St Andrews
University (Medicine)
12
Generic ePortfolio Public Demonstrator
httpwww.eportfolios.ac.uk
13
Medicine at Newcastle Integration with a bespoke
VLE
14
Common Tools Learning Diary (Medicine
Newcastle)
15
Context-specific Tools PDP for Year 4 Student
Selected Components
16
Common Tools Record courses (CRS)
17
Common Tools Learning Outcome / Skills (CRS)
18
Common Tools SWOT (Bioscience prototype)
19
Common Tools CV (St Andrews University)
20
Common Tools Tutor meetings (St Andrews
University)
21
Context-specific Tools Dentistry
Project involves 6 Dental schools and 2
Postgraduate Deaneries
22
Context-specific Tools Dentistry
Promoting Continuity LLL in Dentistry
23
Integration 2 Blackboard (via a tab)
24
Integration 2 Blackboard (via the tool bar)
25
Evaluation 1 Medicine (Years 12) Cotterill
SJ, Sarma S, McDonald AM, Bradley P
  • Ethical approval / informed consent
  • Focus groups Questionnaires
  • Key issues
  • Generally well received but what motivation ?
  • not assessed
  • not viewed by tutors
  • (staff interface completed later)
  • Need better clarity of purpose
  • Wanted a more quirky / fun design !
  • Learning diary - useful at first, less so over
    time
  • Good to browse the Learning Outcomes

26
  • Evaluation 2 Medicine (Year 4)
  • PDP for student selected components
  • Cotterill SJ, McDonald AM, Bradley P, Robinson R,
    Hammond GR
  • Ethical approval / informed consent
  • Questionnaires (before and after using the
    ePortfolio)
  • Change in awareness of intended learning outcomes
    ?
  • Factors influencing use of the ePortfolio
  • Attitudes and perceptions of using the ePortfolio
  • Student perception of impact of using the
    ePortfolio
  • Evaluation of technical features and usability
  • 186 students completed the portfolio (100
    compliance)
  • 105 questionnaires completed (56 of cohort) at
    30/06/2004

27
Evaluation - positive
It encouraged me to really give thought to what
I wanted to achieve during the option, which was
especially useful as this was my first option. As
a result of the portfolio I think I got much more
out of the option than I would have otherwise.
It made me concentrate on creating aims at the
start of the option and allowed me to plan the
option with my supervisor in a defined way.
Overall it made my learning for the option more
organised and focused.
28
Evaluation - barriers problems
Most of the things I learned couldn't be
'measured/quantified' so I felt it was of little
value to try and invent a way in which they
could.
29
Building the ePortfolio was a useful learning
experience
80 thought it was a useful learning experience
30
At the end of the option I spent time considering
what I had learned from this option.
93 reflected on their learning after their SSC
31
Conclusions 1
  • The generic ePortfolio
  • Highly configurable
  • Can be readily adapted for different contexts
  • Existing tools have an emphasis on PDP
  • -but the portfolio framework can serve other
    purposes
  • Integration in VLEs
  • Fully integrated into a VLE for Medicine
  • Initial work to integrate with Blackboard has
    been successful

32
Conclusions 2
  • Evaluation (in Medicine)
  • ePortfolios implemented / piloted in 2003/4
  • Demonstrated Feasibility Acceptability
  • Most students perceive these as beneficial
  • However, unless assessed / mandatory the
    motivation to complete is reduced.

33
Conclusions 3
  • Educational Objectives (including PDP)
  • Some evidence that PDP had a positive impact on
    planning and approach to learning (Year 4 SSC)
  • Encouraged reflection in some students
  • Good preparation (and practice) for LLL CPD
  • However, long-term impact on learning is unknown
  • Further research is essential
  • Some (limited) evidence that PDP improves
    learning

A systematic map and synthesis review of the
effectiveness of personal development planning
for improving student learning. EPPI-Centre
(2002) http//eppi.ioe.ac.uk
34
  • Additional Developments
  • ePortfolios Extension Toolkit (ePET)
  • -A JISC funded project to develop a Web
    services interface for the portfolio (by
    04/2005) http//www.eportfolios.ac.uk/ePET
  • IAMSECT Shibboleth Single Sign-On project
  • JISC funded project
  • http//iamsect.ncl.ac.uk
  • Further information on the ePortfolio
  • http//www.eportfolios.ac.uk
  • Simon Cotterill (S.J.Cotterill_at_ncl.ac.uk)

35
Implementation of ePortfolios - Humans are
important too !
  • Buy-in of key stakeholders
  • Learners
  • Clarity of purpose ?
  • Sense of Ownership ?
  • Tutors Admin staff
  • Extra work ?
  • Senior Curriculum staff
  • Integration with existing processes
  • Training (all)
  • Monitoring and evaluation

36
Selecting sub-sets of portfolio data for
different purposes
Portfolio for Presentation
Portfolio for Assessment
Portfolio for Application (job / promotion)
Portfolio for Accreditation/ Revalidation
Learners repository
Institutional Data
Portfolio for Appraisal
PDP (shared)
PDP / Reflective (private)
Write a Comment
User Comments (0)
About PowerShow.com