Research Methods in Distance Education: Design Based Research - PowerPoint PPT Presentation

1 / 46
About This Presentation

Research Methods in Distance Education: Design Based Research


Research Methods in Distance Education: Design Based Research Terry Anderson PhD Seminar Leicester UK Feb. 2012 * * * * * * * * * * Integrative Learning Design ... – PowerPoint PPT presentation

Number of Views:241
Avg rating:3.0/5.0
Slides: 47
Provided by: JoAn148


Transcript and Presenter's Notes

Title: Research Methods in Distance Education: Design Based Research

Research Methods in Distance Education Design
Based Research
  • Terry Anderson
  • PhD Seminar
  • Leicester UK
  • Feb. 2012

My Focus The context of Distance Education
  • Disruptive innovation (Christensen, 2008)
    simpler, not wanted by main stream customers
  • Rapid gains in functionality
  • Cheaper
  • Adaptive
  • Moving from peripheral to mainstream (blended and
    online for full time students)

Research Paradigms
Research Paradigms
Research Paradigms
  • Quantitative discovery of the laws that govern
  • Qualitative understandings from an insider
  • Critical Investigate and expose the power
  • Design-based interventions, interactions and
    their effect in multiple contexts

Paradigm 1Quantitative Research
  • employs a scientific discourse derived from the
    epistemologies of positivism and realism.

  • those who are seeking the strict way of truth
    should not trouble themselves about any object
    concerning which they cannot have a certainty
    equal to arithmetic or geometrical demonstration
  • (Rene Descartes)
  • Inordinate support and faith in randomized
    controlled studies

Quantitative Example 1 CMC Content Analysis
  • Anderson, Garrison, Rourke 1997-2003
  • http// - 9 papers
    reviewing results focusing on reliable ,
    quantitative analysis
  • Identified ways to measure teaching, social and
    cognitive presence
  • Most reliable methods are beyond current time
    constraints of busy teachers
  • Questions of validity
  • Serves as basic research as grounding for AI
    methods and major survey work of the future
  • Serves as qualitative heuristic for teachers and
    course designers

Quantitative Meta-Analysis
  • Aggregates many effect sizes creating large Ns
    more powerful results.
  • Ungerleider and Burns (2003)
  • Systematic review of effectiveness and efficiency
    of Online education versus Face to face
  • The type of interventions studied were
    extraordinary diverse only criteria was a
    comparison group
  • Only 10 of the 25 studies included in the
    in-depth review were not seriously flawed, a
    sobering statistic given the constraints that
    went into selecting them for the review.

  • 1928-2008
  • distance delivery modes from correspondence
    schools, radio, television, video, and now
  • when the course materials and teaching
    methodology are held constant, there are no
    significant differences (NSD) in learner outcomes

http// Slide
from Tom Reeves
Ungerleider, C., Burns, T. (2003). A systematic
review of the effectiveness and efficiency of
networked ICT in education. Ottawa Industry
Is DE Better than Classroom Instruction?Project
1 2000 2004
  • Question How does distance education compare to
    classroom instruction? (inclusive dates
  • Total number of effect sizes k 232
  • Measures Achievement, Attitudes and Retention
    (opposite of drop-out)
  • Divided into Asynchronous and Synchronous DE

Bernard, R. M., Abrami, P. C., Lou, Y.
Borokhovski, E., Wade, A., Wozney, L., Wallet,
P.A., Fiset, M., Huang, B. (2004). How does
distance education compare to classroom
instruction? A meta-analysis of the empirical
literature. Review of Educational Research,
74(3), 379-439.
Quantitative Research Summary
  • Can be useful especially when fine tuning well
    established practice
  • Provides incremental gains in knowledge, not
    revolutionary ones
  • The need to control context often makes results
    of little value to practicing professionals
  • In times of rapid change too early quantitative
    testing may mask beneficial positive capacity
  • Will we ever be able to afford blind reviewed,
    random assignment studies?

Paradigm 2 Qualitative Paradigm
  • Many different varieties
  • Generally answer the question why rather then
    what, when or how much?
  • Presents special challenges in distributed
    contexts due to distance between participants and
  • Currently most common type of DE research (Rourke
    Szabo, 2002)

Qualitative Example
  • Dearnley (2003) Student support in open learning
    Sustaining the Process
  • Practicing Nurses, weekly F2F tutorial sessions
  • Phenomenological study using grounded theory

Core category to emerge was Finding the
professional voice
Dearnley and Matthew (2003 and 2004)
Qualitative example 2
  • Mann, S. (2003) A personal inquiry into an
    experience of adult learning on-line.
    Instructional Science 31
  • Conclusions
  • The need to facilitate the presentation of
    learner and teacher identities in such a way that
    takes account of the loss of the normal channel
  • The need to make explicit the development of
    operating norms and conventions
  • reduced communicative media there is the
    potential for greater misunderstanding
  • The need to consider ways in which the developing
    learning community can be open to the other of
    uncertainty, ambiguity and difference

3rd ParadigmCritical Research
  • Asks who gains in power?
  • David Nobles critique of digital diploma Mills
    most prominent Canadian example
  • Are profits generated from user generated content
    or OERs exploitative?
  • Confronting the net changes everything mantra
    of many social software proponents.
  • Who is being excluded from online world?

See Norm Friesens
Friesen, N. (2009) Re-thinking e-learning
research foundations, methods, and practices.
Peter Lang Publishers
(No Transcript)
  • Why does Facebook own all the content that we
  • Does the power of the net further marginalize the
  • Why did the One Laptop Per Child project fail? Or
    did it?

Do Either Qualitative or Quantitative Methods
Meet Real Needs of Practicing Distance Educators?
But what type of research has most effect on
  • Kennedy (1999) - teachers rate relevance and
    value of results from each of major paradigms.
  • No consistent results teachers are not a
    homogeneous group of consumers but they do find
    research of value
  • The studies that teachers found to be most
    persuasive, most relevant, and most influential
    to their thinking were all studies that addressed
    the relationship between teaching and learning.

But what type of research has most effect on
  • The findings from this study cast doubt on
    virtually every argument for the superiority of
    any particular research genre, whether the
    criterion for superiority is persuasiveness,
    relevance, or ability to influence practitioners
    thinking. Kennedy, (1999)

Design-Based Research
  • Developed from frustration of the lack of impact
    of educational research on educational systems.

4th ParadigmDesign-Based Research
  • Related to engineering and architectural research
  • Focuses on the design, construction,
    implementation and adoption of a learning
    initiative in an authentic context
  • Related to Development Research
  • Closest educators have to a home grown research

Design-Based Research Model
Phase 2
Phase 4
Phase 1
Phase 3
Analysis of Practical Problems by Research and
Practitioners in Collaboration
Development of Solutions Informed by Existin
Design Principles and Technological Innovations
Reflection to Produce Design Principles and
Enhance Solutions Implementation
Iterative Cycles of Testing and Refinement of
Solutions in Practice
Refinement of Problems, Solutions, Methods, and
Design Principles (Reeves, 2006, p. 59)
Reeves, T. C. (2006). Design research from the
technology perspective. In J. V. Akker, K.
Gravemeijer, S. McKenney, N. Nieveen (Eds.),
Educational design research (pp. 86-109). London
Design-Based Research Studies
  • iterative,
  • process focused,
  • interventionist,
  • collaborative,
  • multileveled,
  • utility oriented,
  • theory driven and generative
  • (Shavelson et al, 2003)

Critical characteristics of design experiments
  • According to Reeves (20008), Ann Brown (1992)
    and Alan Collins (1992)
  • addressing complex problems in real contexts in
    collaboration with practitioners,
  • integrating known and hypothetical design
    principles with technological affordances to
    render plausible solutions to these complex
    problems, and
  • conducting rigorous and reflective inquiry to
    test and refine innovative learning environments
    as well as to define new design-principles.

Design-based research
  • Methodology developed by educators for educators
  • Developed from American pragmatism Dewey
    (Anderson, 2005)
  • Recent Theme Issues
  • The Journal of the Instructional Sciences, (13,
    1, 2004),
  • Educational Researcher (32, 1, 2003) and
  • Educational Psychologist (39, 4, 2004)
  • See bibliography at http//
  • My article at

Anderson, T., Shattuck, J. (2012). Design-Based
Research A Decade of Progress in Education
Research? Educational Researcher, 41(Jan/Feb.),
16-25. Retrieved from http//
Anderson, T., Shattuck, J. (2012). Design-Based
Research A Decade of Progress in Education
Research? Educational Researcher, 41(Jan/Feb.),
16-25. Retrieved from http//
Design Tradition
  • Learning and productivity are the results of the
    designs (the structures) of complex systems of
    people, environments, technology, beliefs and
    texts New London Group 2000
  • DBR opens the door for teachers, researchers and
    learners to become designers, not merely
    consumers, bosses or observers .

Integrative Learning Design(Bannan-Ritland, 2003)
  • design-based research enables the creation and
    study of learning conditions that are presumed
    productive but are not well understood in
    practice, and the generation of findings often
    overlooked or obscured when focusing exclusively
    on the summative effects of an intervention Wang
    Hannafin, 2003

  • Iterative because
  • Innovation is not restricted to the prior design
    of an artifact, but continues as artifacts are
    implemented and used
  • Implementations are inevitably unfinished
    (Stewart and Williams (2005)
  • intertwined goals of (1) designing learning
    environments and (2) developing theories of
    learning (DBRC, 2003)

Design Based research and the Science of
  • Complexity theory studies the emergence of order
    in multifaceted, changing and previously
    unordered contexts
  • This emerging order becomes the focus of iterate
    interventions and evaluations
  • Order emerges at the edge of chaos in response
    to rapid change, and failure of previous
    organization models

DBR ExamplesCall Centres At Athabasca

Answer 80 of student inquiries Savings of over
100,000 /year Anderson, T. (2005). Design-based
research and its application to a call center
innovation in distance education. Canadian
Journal of Learning and Technology, 31(2), 69-84
Design Based research in Action
  • Phase 1 Exploration surveys, talking to faculty
    and tutors, investigating open source tools,
    setting research questions
  • Phase 2. Building the intervention Elgg through
    two versions and 85 plugins (on going)

Design Based research in Practice
  • Athabasca Landing
  • Elgg based
  • Started in 2008
  • 3500 users
  • Unpaced
  • Paced Courses
  • Informal Learning
  • Staff and alumni networking
  • Problem of critical mass

  • Phase 3 Evaluation Before and after surveys
  • Anderson, T., Poelhuber, B., McKerlich, R.
    (2010). Self Paced Learners Meet Social Software.
    Online Journal of Distance Education
  • Dr students Use of past student archives
  • Ongoing iterations and development of tools
  • Phase 4 Design Principles
  • Development of design principles/patterns Nets
    and Sets Dron, J., Anderson, T. (2009). How the
    crowd can teach. In S. Hatzipanagos S.
    Warburton (Eds.), Handbook of Research on Social
    Software and Developing Community Ontologies.

Network Tool Set (example)
Stepanyan, Mather Payne, 2007
Access Controls in Elgg
  • Need to study usability, scalability and
    innovation adoption within bureaucratic systems
  • Allow knowledge tools to evolve in natural
    context through supportive nourishment of staff

  • Education research is grossly under-resourced to
    meet the magnitude of opportunity and demand
  • Paradigm wars are unproductive
  • Design-based research offers a promising new
    research design model
  • It can be used for Doctoral dissertations see
  • Herrington, J., McKenney, S., Reeves, T.,
    Oliver, R. (2007). Design-based research and
    doctoral students Guidelines for preparing a
    dissertation proposal.


Your comments and questions most welcomed!
  • Terry Anderson
  • Blog
  • Twitter terguy
Write a Comment
User Comments (0)