Design Based Research: A New Research Paradigm for Open and Distance Learning Feb, 2007 - PowerPoint PPT Presentation


PPT – Design Based Research: A New Research Paradigm for Open and Distance Learning Feb, 2007 PowerPoint presentation | free to download - id: 5b493-Y2I1N


The Adobe Flash plugin is needed to view this content

Get the plugin now

View by Category
About This Presentation

Design Based Research: A New Research Paradigm for Open and Distance Learning Feb, 2007


Practical experience (Rosemary) Interconnections (Sarah, Katherine, Cindy) ... Kennedy (1999) - teachers rate relevance and value of results from each of major ... – PowerPoint PPT presentation

Number of Views:1690
Avg rating:3.0/5.0
Slides: 86
Provided by: Joan210
Learn more at:


Write a Comment
User Comments (0)
Transcript and Presenter's Notes

Title: Design Based Research: A New Research Paradigm for Open and Distance Learning Feb, 2007

Design- Based Research A New Research Paradigm
for Open and Distance LearningFeb, 2007
  • Terry Anderson, Ph.D.
  • Canada Research Chair
  • in Distance Education

Presentation Overview
  • Traditional opening joke
  • Need for distance education research
  • Sorry state of current research
  • Methodological Orientations
  • Quantitative
  • Qualitative
  • Critical
  • Design-based
  • Dissemination and building a research culture

Why is Distance Education Better Than Sex?
  • If you get tired, you can stop, save your place
    and pick up where you left off.
  • You can finish early without feeling guilty.
  • You can get rid of any viruses you catch with a
    50 program from McAfee
  • With a little coffee you can do it all night.
  • You dont usually get divorced if your spouse
    interrupts you in the middle of it.
  • And If you're not sure what you are doing, you
    can always ask your tutor.

Athabasca University, Alberta, Canada
Fastest growing university in Canada 34,000
students 700 courses Graduate and
Undergraduate programs Largest Master of Distance
Education program Only USA Accredited University
in Canada
Athabasca University
  • Athabasca University

(No Transcript)
(No Transcript)
(No Transcript)
Distance Education Research
  • Can you think of at least one major contribution
    to practice made by distance education research
    in the last decade?
  • Why does this particular research result make a
  • Question
  • context
  • methodology
  • clarity of presentation?

Typical DE Research
  • Here is what we are doing at my university.
    Isnt it wonderful !!
  • Here is what we are doing , why dont you come
    and research it?
  • Variable quality by journal, by conference, by
    region, by practice.

Defining Research
  • the systematic study of materials and sources in
    order to establish facts and reach new
    conclusions (Oxford Compact Dictionary, 1991).
  • adjectives describing research as
  • disciplined, organized,
  • transparent, problem orientated,
  • public, creative,
  • scientific, systematic,
  • diligent, labourous
  • and accessible

Why Do Research in Distance Education?
  • Many unresolved questions of traditional distance
  • - attrition F2F tutorial value, paced vs
  • new forms of distance education provision.
  • what combinations of group based learning are
    worth the cost and inconvenience?
  • Do face to face tutorials really make a
    difference or is real time video or audio
    conferencing just as effective?
  • How much does expensive multimedia really enhance
    student learning?
  • how important are real time interactions compared
    to asynchronous ones ?
  • Do Web 2.0 and social software tools really
    encourage new forms of communities of inquiry?

Why Educational ResearchJust Dont Get No
  • Most research is not valued by funders, other
    academics or worse by practitioners
  • Not funded financially
  • Education 0.01 of expenditures
  • Health 3.0
  • High tech companies 10.0 - 15.0
  • Overall (Canada, 2002) 1.88

Rodney Dangerfield
Assessment of DE Research
  • Many experimental research projects do not
    display rigour in their design
  • Many generalize inappropriately
  • Cultural, linguistic and environmental factors
    often not taken into consideration
  • Few concerned with teacher and tutor support
  • Few studies based on current learning,
    pedagogical or psychological theories
  • Olugbemiro Jegede (1999)

A practitioner's perception of educational
  • answers are too narrow to be meaningful, too
    superficial to be instrumental, too artificial to
    be relevant, and, on top of that, they usually
    come too late to be of any use. van den Akker

Barriers to Educational Research
  • Its nobodys job
  • How many in this room have at least 50 of their
    job requirement to do research?
  • Negligible industry support
  • No large scale focus on particular problems
  • Nobody keeping score in meaningful terms
  • Pervasive lack of trust in research efficacy
  • In sum, lack of an effective research culture
  • (Burkhardt and Schoenfeld, 2003).

Who Should Do Research?
  • Action Research
  • Focused on an authentic problem
  • Designed and implemented by participants
  • Includes reflection ands with dissemination
  • Students as researchers
  • Why are students excluded from this rich learning
  • Professional researchers
  • DE or educational technology researchers
  • Faculty teaching DE within a different academic

How do we Build a Culture of Research in Distance
Research Paradigms
  • Quantitative discovery of the laws that govern
  • Qualitative understandings from an insider
  • Critical Investigate and expose the power
  • Design-based interventions, interactions and
    their effect in multiple contexts

Quantitative Paradigm
  • Key words like evidence based systematic
    review scientific research
  • employs a scientific discourse derived from the
    epistemologies of positivism and realism
  • Long tradition borrowed from the natural sciences
  • Since context is so pivotal in education, a great
    number of studies must be done to eliminate
    contextual variance and combined using meta
  • Inordinate support and faith in randomized
    controlled studies

  • those who are seeking the strict way of truth
    should not trouble themselves about any object
    concerning which they cannot have a certainty
    equal to arithmetic or geometrical demonstration
  • (Rene Descartes,). 1496-1650

The challenge of meeting criteria for
quantitative study
  • Control group assignment rarely possible
  • Blind assessment not practical
  • Contextual variables in natural contexts negate
    transfer and replicability
  • What works in one context, at one time does not
    guarantee it will work again
  • Interventions are never controlled nor identical
  • Educational results must always be interpreted

Is meta analysis the gold standard?
  • Canadian example

Quantitative Ex. Meta-Analysis
  • Ungerleider and Burns (2003)
  • Systematic review of effectiveness and efficiency
    of ICT
  • The type of interventions studied were
    extraordinary diverse only criteria was a
    comparison group and use of ICT
  • Only 10 of the 25 studies included in the
    in-depth review were not seriously flawed, a
    sobering statistic given the constraints that
    went into selecting them for the review.

(No Transcript)
Ungerleider, C., Burns, T. (2003) . A
systematic review of the effectiveness and
efficiency of networked ICT in education . P.38
Ottawa Industry Canada. Retireved Jan. 24, 2004
from http//
USA Department of Education (2003) guidelines for
Identifying and Implementing Educational
Practices Supported By Rigorous Evidence
Quantitative Summary
  • Can be useful especially when testing well
    established and consistent practice.
  • The need to control context often makes results
    of little value to practicing professionals
  • In times of rapid change too early quantitative
    testing may mask beneficial positive capacity
  • Will we ever be able to afford blind reviewed,
    random assignment studies?

Qualitative Paradigm
  • Many different varieties
  • Generally answer the question why rather then
    what or how much?
  • Presents special challenges in DE due to distance
    between participants and researchers
  • Currently most common type of DE research (Rourke
    and Szabo, 2002)

1st Qualitative Example
  • Dearnley (2003) Student support in open learning
    Sustaining the Process
  • Practicing Nurses, weekly F2F tutorial sessions
  • Phenomenological study using grounded theory

Dearnley (2003)
  • Support structures to facilitate personal and
    professional development within this context need
    to be in place and attention must be given to the
    provision of effective learner support.
    (Dearnley, 2003)

Johnson H. (2007). Dialogue and the Construction
of Knowledge in E-Learning Exploring Students'
Perceptions of Their Learning While Using
Blackboard's Asynchronous Discussion Board. EURODL
  • Four different ways of perceiving online learning
    were identified
  • Practical experience (Rosemary)
  • Interconnections (Sarah, Katherine, Cindy)
  • Expressing own thoughts (Anthony, David)
  • Flexible learning (Larry)

Qualitative Summary
  • Measure of quality is critical appraisal
    concerning plausibility, internal consistency and
    fit to prevailing wisdom
  • Burkhardt Schoenfeld (2003)
  • But what if the only producers and consumers are
    researchers not practitioners?
  • Often the only answer that makes it to the
    practice is it depends!

Critical Example Friesen, N. (submitted). The
Experience of Computer Use Expert Knowledge and
User Know-How. GLIMPSE Phenomenology and Media.
  • I will also show how the actual experience of
    computer use casts into doubt the educational
    efficacy of computers understood as instruments
    of cognitive amplification, or simply
  • Did my failure to save my work or to properly
    address my email message arise from a mismatch in
    "system" and "user" models?
  • user knowledge of the system appears as
    embodied, performative and emphatically
    provisional in nature.
  • http//

Do These Research Paradigms Meet the Real Needs
of Practicing Distance Educators?
But what type of research has most effect on
  • Kennedy (1999) - teachers rate relevance and
    value of results from each of major paradigms.
  • No consistent results teachers are not a
    homogeneous group of consumers but they do find
    research of value
  • The studies that teachers found to be most
    persuasive, most relevant, and most influential
    to their thinking were all studies that addressed
    the relationship between teaching and learning.

But what type of research has most effect on
  • The findings from this study cast doubt on
    virtually every argument for the superiority of
    any particular research genre, whether the
    criterion for superiority is persuasiveness,
    relevance, or ability to influence practitioners
    thinking. Kennedy, (1999)

  • In any dispute the intensity of feeling
  • is inversely proportional to the value
  • of the stakes at issue -- that is why
  • academic politics are so bitter.
  • Wallace S. Sayre, 1905-1972
  • quoted in,
  • "Issawi's Laws of Social Motion" (1973)

4th ParadigmDesign-Based Research
  • Related to engineering and architectural research
  • Focuses on the design, construction,
    implementation and adoption of a learning
  • Related to Development Research
  • Closest educators have to a home grown research
  • Interventionist within a real educational context
  • Action Research on Steroids!

The Contextual Turn
  • Postmodern assertion of variability pervasively
    induced by uniqueness of any particular context
  • Context of
  • Place virtual, home, classroom, institution
  • Actors many individual differences, temporary
    and long lasting
  • Culture including intra cultural heterogeneity
  • Learning, Cognition, Knowing and Context are
    irreducibly co-constituted and cannot be treated
    as isolated entities or processes (Barab
    Squire, 2004)

Context creates Content creates Context
deFigueiredo (2005) Learning Contexts a Blueprint
for Research,
The Complexity Turn
  • Increased interest in viewing educational
    contexts as complex environments
  • Not possible to precisely forecast or predict
    behaviour explain and promote - but not predict
  • Interventions from the outside (teacher
    interventions) are conditioned and recursively
    amplified or extinguished by contextual variables
  • Resulting in emergent behaviours of organisms
  • Complexity at the edge of chaos provides
    opportunity for creativity and necessary change
  • See Underwood (2000) complexity theory Pascale
    et. al (2000) Surfing the Edge of Chaos, Bennet,
    (2004) Organizational Survival

Design Intervention
Evaluation Assessment
Pervasive effect of assessment
(Bannan-Ritland, 2003)
4th Paradigm Design Studies
  • iterative,
  • process focused,
  • interventionist,
  • collaborative,
  • multileveled,
  • utility oriented,
  • theory driven and generative
  • (Shavelson et al, 2003)

Design Based Example 1 - Athabasca
  • Tutor Model
  • On call 2 hours/week
  • Part time, problem with knowledge of institution
  • knowledge of a single course
  • Personal relationship
  • Subject matter expertise
  • Personal knowledge base
  • Call Centre Model
  • Advisor on call 40 hours/week
  • Full time, steeped in university environment
  • All business curriculum
  • Customer relationship
  • Refers to academics for subject matter expertise
  • Formal FAQ and data collection


Call Centres Answer 80 of student
inquiries Saves over 100,000 /year
Stage 1 Informed Exploration
  • Literature review, theoretical extrapolation and
    expert and participant input
  • Often an ideal provides a vision and a guide as
    well as significant component of the measuring
    stick by which the ideal, as instantiated in
    actions within a real context, is measured.

Stage 1 Informed Exploration
  • Review of call centre literature
  • Interviews with current tutors and managers
  • Data collection on current processes and costs
  • Visit to other call centres, especially those in
    related but uncompetitive contexts

Stage 2 Enactment
  • Production phase highly visible
  • Need for project management, tracking and
  • Prototype articulation, design and construction
  • Designs should be widely circulated and critiqued

Stage 2 Enactment
  • Design and coded using Lotus Notes
  • Project management, data collection on
    development problems and costs,
  • Pilot testing
  • Multiple iterations

Stage 3 Local Evaluation
  • Interviews, focus groups with call centre staff
  • Student satisfaction surveys
  • Student interviews
  • Analysis of transaction logs and FAQ
  • Cost analysis
  • Interviews with tutors, union

Stage 3 Local Evaluation
  • Multi-methodological evaluation of the
  • Iterative moving from formative to summative

Stage 4 Broader Impact Evaluation
  • Generate and advance a particular set of
    theoretical constructs that transcends the
    ..contexts in which they were generated, selected
    or refined (Barab Squire, 2004)
  • Use of thick description and qualitative
  • Work to expand and develop theory
  • Tools and conceptual models to understand and
    adjust both the context and the intervention
  • Value of an intervention lies in its capacity to
    effect positive change not in the scientific
    significance of the results
  • Call for national and international clearinghouse
    of phase 4 evaluations (Collins et al. 2004)

Stage Four - Trials in Multiple Context
  • Currently four help desks operating at Athabasca
  • Continuing evaluation showing NSD between
    perceptions of value by students between tutor
    and call centre model
  • Increased use of web services decreasing need for
    either tutors or call centers ie Am I ready for
    AU adaptive testing
  • Further research analyzing institutional
    resistance to change

Design-Based Study 2
  • A work very much in progress
  • Social software solutions for continuous
    enrollment courses
  • But what type of interaction meets students
    needs, is cost effective and is least restrictive
    on freedom of both learners and teachers?

Stage 1 Informed Exploration
  • Review of literature on interaction and self
    directed study
  • Interviews with course developers, faculty in
    regard to experience with social interventions
  • Telephone interviews with others around the world
    involved in continuous enrollment DE programming
  • Survey of students in classes with social
  • Anderson, Annand and Wark 2005 Having your Cake
    and eating it, 2005. Austariasia Journal of
    Educational Technology

Two Solitudes of Distance Education
Collaborative, Distance education 3rd gen.
video, audio and computer conf
Type C Communications Technology
Independent Study 1st gen. correspondence 2nd
gen. telecourses Type I
Information Technology
AU Undergrad
AU Grad
AU Future ??
Anderson, in press
Type S Distance Education
Collaborative, Distance education 3rd gen.
video, audio and computer conf
Type C Communications Technology
Socially Enhanced Independent Learning Type S
Social Technology
Independent Study 1st gen. correspondence 2nd
gen. telecourses Type I
Information Technology
AU Undergrad
AU Grad
AU Future ??
Anderson, 2006
Learning Freedom
  • Paulsens (1993) theory of cooperative freedom
  • Freedom of space
  • Freedom of pace
  • Freedom of time
  • Freedom of media
  • Freedom of content
  • Freedom of access
  • Freedom of relationship (Anderson)

  • But these freedoms are compromised by the
    interactive, paced and collaborative form of 3rd
    generation DE that is based on Type C
  • Is the value of the social, paced and
    collaborative learning of sufficient magnitude to
    justify these restrictions?

Educational Social Technology (EST)
  • Networked tools that support and encourage
    learning through face-to-face and virtual social
    interactions while retaining individual control
    over time, space, presence, activity, identity
    and community. (Anderson, 2005)
  • Social software are tools that support
    communication using the five devices of
    identity, presence, relationships, conversations
    and groups. (Butterfield, 2003)

Stage 1 Research questions
  • What tools, learning activities and incentives
    are needed to allow students in learner-paced,
    continuous enrollment courses to provide social,
    cognitive and teaching presence to each other?

Survey results
  • 78 of self-paced student respondents indicated
    that they would interact with other students as
    long as they were able to proceed through the
    course at their own pace.
  • 95 of student respondents reported a desire to
    access the work of other students either
    currently or previously enrolled in the courses.
  • Preferred Mode of that interaction
  • 70 preferred asynchronous media like email and
    computer conferencing,
  • 27 preferred a combination of synchronous and
    asynchronous technologies
  • only 3 preferred synchronous interaction alone
    (for example, audio conferences or face-to-face

Stage 1 summary
  • Interviews and survey of students
  • Review of the interaction literature
  • Investigate social software tools
  • Develop research questions and methodology

Stage 2 Enactment
  • Select and install an instance of elgg
  • Develop support strategies and documentation
  • Pilot testing (faculty, undergraduate and
    graduate classes)
  • Develop new instructional designs
  • Successive iterations

Stage 2 Enactment
  • Social software typically supports
  • Individual and group Wiki, Blog, Events, Tasks,
  • User profiles.
  • Friend of friend
  • Interest sharing, group formation and access
  • RSS
  • Connections on and off line

Technologies of MDE 663 Fall 2006
Blogging Connections
Content Admin Asynchronous Int.
Portal Products
Learning Objects
Real Time Pacing Social Presence
Dissemination Knowledge Polling
Stage 3 Local Evaluation
  • Ethics clearance and resolving privacy issues
  • Interviews, focus groups with developers and
  • Student satisfaction surveys
  • Student interviews
  • Analysis of transaction logs
  • Cost analysis using completion rate data

Usefulness over 8 Educ Functions
N 9 of 13
Supporting Social and Feeling Connected
N 9 of 13
Mastering Knowledge Objectives
N 9 of 13
Stage 1-3 Iterations
  • Adding functionality
  • Testing new designs and learning activities
  • Reacting to Organizational Change
  • Think staff development and redefinition of
    learning roles
  • Think decentralization students developing
    unanticipated use of system

Stage 4 Broader Impact Evaluation Theorizing
  • Publishing of results
  • Trials in different contexts
  • Use in paced and unpaced courses
  • Use in different disciplines
  • Collaboration with international ELGG groups
  • Synthesis of application in different contexts
  • Adoption of Innovation framework
  • Theorizing Does Educational Social Software
    allow scaleable, high quality, learning while
    maximizing learner freedoms?

Design-based Research Conclusion
  • Methodology developed by educators for educators
  • Developed from American pragmatism Dewey
    (Anderson, 2005)
  • Recent Theme Issues
  • The Journal of the Instructional Sciences, (13,
    1, 2004),
  • Educational Researcher (32, 1, 2003) and
  • Educational Psychologist (39, 4, 2004)
  • See bibliography at http//
  • My article at

Building the Research Culture
  • Better dissemination within a combined
    research/practitioner communities
  • Better tools
  • More funding less fighting

(No Transcript)
Online Journals
  • International Review of Research on Open and
    Distance Learning

A Tale of 3 books
Open Access 72,000 downloads plus indiv.
chapters 400 hardcopies sold _at_ 50.00 Free at
Commercial publisher 934 copies sold at
52.00 Buy at Amazon!!
E-Learning for the 21st Century Commercial
Pub. 1200 sold _at_ 135.00 2,000 copies in Arabic
Translation _at_ 8.
  • DE Research is grossly under-resourced to meet
    the magnitude of opportunity and demand.
  • Paradigm wars are unproductive.
  • Design-based research offers a promising new
    research design model.
  • Web 2.0 enhanced interaction (multiformat, synch,
    asynch, immersive environments) may offer key to
    more effective knowledge growth and exchange.

  • Wisdom is fed equally from practical,
    theoretical, and ethical knowledge and from a
    large quantity of reflected experience
  • Otto Peters, 2003 p. 137

Your Comments or QuestionsMost Welcomed !
Terry Anderson
Research philosophy for professionals Ulrich
  • reflective competence is
  • self-critical the effort of systematically
    examining ones own premises through
    self-reflection and dialogue, with a view to
    carefully qualifying the meaning and validity of
    ones claims
  • emancipatory working actively to help others in
    emancipating themselves from ones claims, as
    well as from theirs and
  • ethically alert making transparent to oneself
    and to others the value implications of ones
    claims, and limiting these claims accordingly.