ePortfolio research: Select, Reflect, Connect

1
ePortfolio research Select, Reflect, Connect
Individual thoughts Additional ideas
after discussion

• On-line tracking/monitoring
• Questionnaires sampling of learner perceptions
• Interviews pre-, during and post- a phase of ePF
  use
• Focus groups real interaction / dialogue with
  learner
• Exemplar material
• A baseline of past practice
• Identify parallel behaviours in different
  cultures
• Action research
• Case study
• Linking to attainment, retention, career success

• Stop researching method of implementation of
  Portfolio
• Research the depth of learning through research
  on Learning Journals generally
• What is the ePortfolio helping learners to
  achieve? What difference does it make?
• Presentation gallery
• Identify the problems people have to which ePF
  is the answer
• e.g. How can ePF help make institutions systems
  work more easily?
• What would motivate learners at a higher level?
• Does ePF use by learners change pedagogy?
• Does semi-automation, pre-population of fields,
  make a real difference to student engagement, by
  providing a high starting point?

• Select
• What methods will help answer your questions?

2
ePortfolio research Select, Reflect, Connect
Individual thoughts Additional ideas
after discussion

• Impact on curriculum, practice, quality
  reflections / usage
• In terms of logical services, breaking down PDP
• In terms of how discrete services form flows
• Against predictive outcomes
• Cultural / human aspects
• Technology implications
• Quantitatively - of users etc
• Qualitatively feedback from those in categories
  1. and 3. above
• Collate and compare comments. Identify specific
  themes and common issues. Follow up with deeper
  research e.g. interview or with wider research
  e.g. questionnaire

• Cultural / human aspects
• Changes in practice
• Sufficient notion practice
• Kinds of service that technology needs to offer
• Educationalists vs Technologists
• ePF aggregates existing domains of practice
• The value of ePF is in integration of previously
  separate activities, e.g. ILP and IAG
• Literature reviews on Journals / use of
  electronic processes, etc
• Changes in practice

• Reflect
• How could the data be analysed?

3
ePortfolio research Select, Reflect, Connect
Individual thoughts Additional ideas
after discussion

• Exemplar material on-line for student reference
  articles?
• To the domains of practice for logical services
• Through implementation of example systems
• Workshops
• Networks
• At different levels
• - strategists
• - managers
• - potential users
• as well as academics
• To inform development and usage
• Need to disseminate to all who have
  responsibility for front-line implementation
  need to demonstrate relevance face to face best

• Avoid self-serving academic roundabout
• Audience?
• Policy makers in government?
• Managers in School
• ECM agenda
• Exemplars
• Do/Should ePortfolios need implementers?
• Inform future development audience

• Connect
• How should the findings be disseminated?

4
ePortfolio research Select, Reflect, Connect
General comments

• If is easy to provide answers to the wrong
  questions, in particular to demonstrate the
  benefit of learning a technique rather than the
  principles behind a technique. Peter Rees Jones
• If the question is How far does the use of
  ePortfolios by learners (this category needs
  breaking down into major groups /types) enhance
• a. Integrative Learning
• b. Employability Angela Smallwood

5
ePortfolio research Connect
Individual thoughts Additional ideas
after discussion

• Project reports, vendors claims
• Own institution case studies
• Conference presentation
• Discussion forums, standard body documents HESA
  stats, QA assumptions
• Number of ePF holders by type, institution,
  demography, etc.
• Number of institutions using ePF
• Type of organisation using ePF
• Demographic data of ePF holders
• Purposes of ePF
• Types of ePF hosts
• Size of ePF storage
• Software products used to support ePF
• Frequency of use of ePF
• Age of ePF
• General topics / sections within ePF
• Financial data
• Any that is interoperable
• A lot? A little?

• Unexpected outcomes
• Make researchers assumptions explicit
• UCAS / HESA
• Number of holders / users / institutions
  (Demographics Where are they coming from etc.)
• Purposes of ePF
• Technology types, hosts, storage, size
• Frequency of use maintainability
• Age of ePF
• Cost
• Different subject areas?

• What data are already available?

6
ePortfolio research Connect
Individual thoughts Additional ideas
after discussion

• Self-conception of Identity
• Usage stats
• Qualitative data for forming categories
• Problem of access personal data
• Prof competencies what profs want to be
  collected
• Take up rates of generically offered ePF, PDP
  within depts
• Employment skills required
• Users of social space sites numbers grouped by
  institution, type of study
• Reasoning and priorities of institution,
  implementing PDP eP
• Usability testing and feedback from implemented
  systems

• Transition, progression, retention, achievement
• Users and non-users
• Impact on employability
• Impact on users
• Depends on purpose of research
• Discipline dependencies
• Views of learners and teachers
• Project reports
• Case studies
• UCAS
• Is e relevant aside from just portfolio?

• What data need to be collected?

7
ePortfolio research Connect
Individual thoughts Additional ideas
after discussion

• School / uni data on marks, destinations
• Questionnaires and interviews
• Protocol analysis?
• Behaviour tracking (observation and analysis of
  output)
• Usable STD format (electronic)
• Quantitative

• Qualitative, quantitative assumptions of
  implementers
• Guides, intros, include explanations
• Reactions
• Any formats that are interoperable
• Survey, portfolios, interviews

• In what format?

8
ePortfolio research Connect
Individual thoughts Additional ideas
after discussion

• an ePortfolio
• Time
• Taxonomies of Portfolio practice
• Interpersonal / ethnographic skills
• Usage and take up rates
• Usage patterns just in time or developmental
  non-users
• Survey tools
• Co-operation
• Researchers / Framework Linking / usable across
  a network of all institutions
• Case study built-up

• Permission
• Access
• External
• School / college / uni data
• Questionnaires and interviews
• Survey tools
• Co-operation
• Reflection how is this identified, recognised,
  measured?
• Researchers? Across Universities

• What resources are needed for collection?

9
ePortfolio research Connect General comments

• Purpose Recipient Design Policy formation
  Evaluation Social Scientific research
  Institutional Practitioner

10
ePortfolio research Purpose and Audience
Individual thoughts Additional ideas
after discussion

• To inform the design of effective ePortfolios
  systems
• To understand the scope and purpose of
  ePortfolios
• To understand how ePortfolios fit into the space
  of learner support systems
• To share good practice
• To add to body of research
• To enable development of ePortfolios
• To give concrete, quantifiable evidence and avoid
  folksonomy and demonstrate what purpose / use
  is actually most helpful and useful.

• Social Network
• The importance of social interaction ? what is it
  around the take up of some social software that
  engenders success
• The importance of the human interface
• Traffic formal and informal learning
  processes

• What is the purpose?

11
ePortfolio research Purpose and Audience
Individual thoughts Additional ideas
after discussion

• Systems designers
• Policy makers
• Researchers
• Teachers and lecturers
• Developers (open source and commercial)
• Government bodies
• Institutions
• Learners
• Policy makers where does the government need to
  intervene and where not
• Educational practitioners and institutions

• Social Network
• The importance of social interaction ? what is it
  around the take up of some social software that
  engenders success
• The importance of the human interface
• Traffic formal and informal learning
  processes

• Who must the research inform (audience)?

12
ePortfolio research Purpose and Audience
Individual thoughts Additional ideas
after discussion

• How can ePortfolio assist a learner to cross the
  boundary between formal and informal learning?
• How can ePortfolios condition, frame and assist a
  lifetime of learning?
• How can ePortfolios be appreciated by the learner
  as a valued tool?
• How ePortfolios aid transition
• Impact on employment / admission to appropriate
  courses
• How will ePortfolio processes drive, support or
  hinder the modernisation of current industrial
  revolution style education?
• Does collection / showing of evidence actually
  support transition?

• Evidence from employers perspective
• www.h2g2.com
• What makes social software successful? (H2G2
  model vs Wikipedia)

• What specific research questions interest you?

13
ePortfolio research Purpose and Audience
Individual thoughts Additional ideas
after discussion

• A continuing set of interlinked projects, forming
  a community of researchers and practitioners
• 2 years - long term so can track
• Why would learners want to use eP processes if
  they were not required by education / employment
  etc. to do so?
• In other words, what is in it for them outside
  the general requirements of education and
  employment
• More than 3yrs for evidence of lifelong use,
  therefore need reasonably sized cohort to be able
  to collect results . gt50?

• Human / system interfaces

• What should be the scale of the research (scope
  and timeline)?

14
ePortfolio research Purpose and Audience
Individual thoughts Additional ideas
after discussion

• Researchers
• Practitioners / universities (experience
  know-how) CETIS and other pertinent bodies
• What happens with learners who do not like
  articulating themselves through ePortfolios in
• Required eP situations
• Voluntary use of eP
• X sectoral groups balance of pedagogy and
  users including lay people.

• Trying to make it personalised why is it worth
  doing anything at all
• Need to show benefits, need evidence based

• Who should do research into ePortfolios?

Digital (literacy), articulacy, divide