Strengthening Feedback in LearnerCentred Education through the Use of ICT

1
Strengthening Feedback in Learner-Centred
Education through the Use of ICT

• Martin Brown, Tim Dornan, Dan Powley, Mohammed
  Islam, Gary Murray, Phil Boulton
• University of Manchester Salford Royal Hospital
  Trust
• Horus/Horus-FP projects
• http//www.csc.umist.ac.uk/horus
• martin.brown_at_manchester.ac.uk

2
Presentation Outline

• Educational rationale
• Feedback driven, learner-centred education
  processes
• Data storage
• Portfolio dissemination
• Resource discovery
• Quality assurance
• Software architecture
• Validation

3
Educational Pain Points Drivers
1 Rationale

• Commercial VMLEs do not typically support
  medical/work-based learning
• Learning events that comprise of both timetabled
  and individual events, which may require
  registration (sign-ups)
• Competency-based assessment
• Reflective learning, recording both quantitative
  and qualitative comments
• Quality assuring the learning process, auditing
  event occurrence, learner attendance and
  competency/skill progression
• This research is aimed at
• Ensuring an equality of learning opportunities
• Supporting portfolio management
• Quality assuring the educational processes
• This functionality has been developed as a set of
  services that can be embedded in existing VMLEs

4
Feedback-driven Educational Processes
2 Processes
Peer knowledge dissemination selection
Evaluate peer feedback
Last experience
Learning group
Directed resource discovery
Choose
Compare
Report
Current experience
Teachers
Learner
Analyze
Give feedback on last experience
Reflect on experience
Course leads/ managers
Find/ register relevant learning opportunities
Next experience
Review module objectives
Quality assurance
Information gathering self-reflective learning
5
Competency-based Learning Cycle
1 Intro
2 Processes
Directed selection by experience relevance
Learner access
Provide feedback
Reflective learning Skills gap
analysis Deviation detection
Quantitative aggregation
Knowledge dissemination
Implemented as a VMLE called iSUS in 2003
6
Educational Feedback Transactions
3 Storage

• Typical feedback-driven educational processes
  include
• Each instance of a process will generate a set of
  quantitative and qualitative data
• Each piece of e-learning rating data is
  decomposed into an educational transactions, of
  the form
• Learner Event Teacher Date Question
  Rating

Measure
Dimensions
7
Star (OLAP) Cubes
3 Storage

• Each dimension is a hierarchical structure such
  as
• Learner Department School University
• Event Type Group
• The measure is a numeric quantity that needs to
  be analysed
• Competency/skill assessment (1-6)
• Teaching/event quality (1-5)
• Event attendance (0/1)

Data can be queried by/aggregated across any of
the dimensions
8
Dissemination via Portfolios
4 Portfolios

• A basic requirement is that a learner/teacher
  should be able to view any information that has
  been entered by them
• By selecting any other dimensions in the
  educational transaction (event, learner, group,
  teacher, ) feedback relating to that entity can
  be disseminated as appropriate

View
Create a competency profile for an event by
aggregating over all the learners ratings
9
Resource Discovery
5 Discovery

• To find/select relevant learning opportunities in
  a curriculum that has a core set of
  competencies/skills/learning outcomes
• Compare the learners competency profile with a
  target/peer group
• Match competency gap to learning opportunity
  profile and calculate relevance
• Rank learning opportunities

10
OLAP Quality Assurance
6 Quality Assurance

• Educational quality assurance is about
• Ensures minimum standards are met as well as
  identify areas of best practice
• Identify where problems (people, events, ) are
  occurring and investigate why
• OLAP (On-Line Analytic Processing), a database
  technology, is an enabler for Fast Analysis of
  Multi-Dimensional, Shared Information
• OLAP servers/clients provide an easy to use
  browser for data stored in cubes/star schemas

• Competency/skill attainment
• Weak/strong competencies
• Learners competency profiles
• Competency ratings by assessor
• Average and worst case analysis for higher level
  groups

11
Architecture and Services
2 Horus
7 Services
Educational

portal

Services/

Portlets

Web
.Net

Osiris Timetable management
service

remoting

Horus toolkit




Authorization


Gather feedback


Authentication


Manage learning
Sweet.Net Personnel management
ou
tcomes


Directed resource

Session

discovery


management


Creating reflective
learning portfolio


Quality

Skills/comp
System

assurance

management

events

analysis

C, .Net development Individual services of the
form string GetFeedbackForm(int ReturnType,
string LearnerID, string EventID) string
GetLearnerProfile(int ReturnType, string
LearnerID, int StarID)
12
Educational Validation
8 Validation

• iSUS intelligent Sign Up System
• Been in use since October 2003 for medical UGs at
  Manchester
• Learners learn reflectively, assemble a personal
  development record and discover relevant learning
  opportunities
• Quality-managed some 8K learning opportunities
  involving 140 learners
• Obtained 24K quantitative and 2K qualitative
  feedback items
• gt90 response rate
• Horus-FP Foundation Programme
• Will be deployed in December 2005 at 3 teaching
  hospitals in the NWN
• Used to build portfolios, record assessment and
  appraisals and quality assure the national
  programme
• Support analyst, administration, assessor etc
  roles
• JISC NWD sponsored

13
Conclusions

• Storing the reflective/assessment/portfolio/QA/
  data in star schemas allows many different roles
  to be supported

Analysts
Portfolios
Event occurrence
Competency attainment
Learners
Data entry
Quality assessment
Resource distribution
Competency profile
Assessment history