Bob Smart Managing Director Warning Signs of EBureaucracy In EAssessment EPortfolios

1
Bob SmartManaging DirectorWarning Signs of
E-BureaucracyIn E-Assessment / E-Portfolios
TM
Providing E-Vocational Technology Since 1996
2
What are the Dangers?

• Compliance for hundreds of Regulatory Bodies
• Common Accord Code of practice conflicts
  (borders) etc
• Speed (requests for shortcuts by users . . The
  customer!)
• Savings Driven (requests on price . .
  Compromise!)
• IT Just because it can (Perception of what can
  actually be done v reality?)
• Trying to make it Equality Compliant (Too much
  can render it unfriendly)
• Everyone wants a different report (all centres
  need different hybrids)
• How many suppliers are there?
• 59 last count E-portfolio 271 E-Learner / testing
  Content Providers . .?
• How many Vocational Awards, and do they
  appear the same on systems?
• Awarding Bodies, ALI, Ofsted, LSC, DfES etc etc
  Req bureaucracy (at mo)
• Open Source brings concerns for
  standardisation.
• Driven by Academic Research that blanket
  e-portfolios without understanding vocational
  awards! (they dont do them!)
• Web Based, Stand Alone, which is best?
• System Approvals, do colleges want 8 portfolio
  systems to cover A/B approvals etc etc.
• Emperors Clothes!

3
What standards are already in place?

• Examples
• BS8788 The UK Lifelong Learning Profile (confused
  with NVQs)
• RDCEO Re-useable Definition of Competency
  Educational Objective.
• VDEX IMS Vocabulary Definition Exchange
• SCORM Sharable Content Object reference model
• CORDRA Content Object Repository Discovery and
  Registration / Resolution Architecture.
• Open Source . . . !
• Etc. etc
• Lets look at some examples that we have
  encountered!
• Different compliance and understanding of the
  word itself . .what is an e-portfolio?
• European verses English, Welsh and Scottish
  requirements.
• Different standards and requirements exist even
  in our own borders!
• University interpretation of e-portoflio
  gathering steam based on repository model and
  academic achievement. This could bully the
  market place at the influential levels
  (inadvertently)
• Open Source, fine for certain awards, but main
  stream vocational assessment it is not fit for
  purpose. A/B need to approve systems, etc.
• Focus is on the learning and development element
  and not the practical and accreditation process
  required to gain the award.
• Oh its a computer, it can therefore give me a
  report like this. . . That has no material
  benefit, but it looks good!
• The actual level of competences required by the
  examiner / assessor (IT), is greater than is
  currently in place and no specific development
  and training plan is available, only generic
  awareness of systems unless they purchase them!

4
What is an e-portfolio and should different
regulations apply to different systems!

• There are many different systems
• Repository of evidence (scrapbook of
  Qualifications, CV etc)
• Formal Accreditation (NVQs etc) (Either Assessor
  or Candidate driven)
• Student lead, such as DIDA, OCR
• Working and Learning e-portfolio
• Possibly a private, working e-portfolio used as a
  staging post before entering into another system.
• So should different standards apply to different
  systems? Never mind Framework for achievement .

5
Whats the current thinking!

• Are the proposed action areas for aligning
  assessment feasible and appropriate?
• JISC feels that in this section of the strategy,
  clarity is needed on what is actually meant by
  e-assessment.  Does it refer to doing online
  assessment (eg. a live online interaction) or
  does it refer to supporting assessment through
  e-learning systems.
• Are the proposed action areas for assuring
  technical and quality standards feasible and
  appropriate?
• The JISC believes that technical and quality
  standards need to be treated separately as the
  two are quite different.  JISC is only concerned
  with technical standards and does not feel that
  it appropriately constituted to tackle pedagogic
  or quality standards of electronic resources
• But what is clear, is that no-one appears to have
  addressed the work of the electronic portfolio in
  vocational awards. It all appears to be academic
  based and focused on integrated e-learning. The
  terminology e-assessment has clouded the issues
  and as such creates a massive hole in the world
  of vocational assessment process / technology.
• To sum up, unless we work together to demonstrate
  to show the route forward, then many years of
  work could be undone or at best misrepresented.
  To this end, someone must lead the way in
  creating a standard for interoperability . . .
  E-skills? Catch up between developers being
  clever and deliverers trying to implement! (Gap
  getting bigger in some areas!, closer in others!)
• Previous Speaker . . Thanks Paul, (Stakeholder
  Tools!) . . I expect the phone call then ?
• What is the train of thought . . The mindset!

6
theyre not ready . . . which generation?
_at_
2004
1944
Examiners/Verifiers/SMT
Teachers/Lecturers/SOs/POs
NQTs/Graduates/Workforce of tomorrow
FE/Secondary student
Primary pupil
7
Radical innovation
Paper-based
Computer- based
No innovation
8
Radical innovation
Paper-based
Computer- based
No innovation
9
Bob SmartManaging Director
Blake's Parliamentary Listed
TM
Providing E-Vocational Technology Since 1996
10
What can we do?

• The largest e-portoflio providers in the UK (as
  researched and indicated by C3 education), have
  joined forces and are to set up their own kite
  mark system, Jointly managed but administered by
  PaperFree Systems Ltd.
• These companies are
• PaperFree Systems Ltd FE penetration 41
• ENVQ FE penetration 27
• Quickstep FE penetration 21
• Creation of own Technical Standards with the
  assistance of the Awarding Bodies and DfES / LSC.
• Development of the New E1 Assessor Award to train
  practitioners in best assessment practice.
• Share common data infrastructure to allow
  interoperability with systems like Maytas and
  Cogisoft (already done) for Funding
• Work with major training Providers to create
  common platform
• Already in Place with REMIT and Others.

11
Bob SmartManaging Director
Blake's Parliamentary Listed
TM
Providing E-Vocational Technology Since 1996