Rapid Research in the Educational Technology Development Process - PowerPoint PPT Presentation

Loading...

PPT – Rapid Research in the Educational Technology Development Process PowerPoint presentation | free to download - id: 8f322-MmY0Y



Loading


The Adobe Flash plugin is needed to view this content

Get the plugin now

View by Category
About This Presentation
Title:

Rapid Research in the Educational Technology Development Process

Description:

Rapid Research in the Educational Technology Development Process. Cara Lane, PhD ... Student computer labs. Computer classrooms. Advanced facilities. Knowledge ... – PowerPoint PPT presentation

Number of Views:42
Avg rating:3.0/5.0
Slides: 35
Provided by: ctlt2
Category:

less

Write a Comment
User Comments (0)
Transcript and Presenter's Notes

Title: Rapid Research in the Educational Technology Development Process


1
Rapid Research in the Educational Technology
Development Process
  • Cara Lane, PhD
  • Research Scientist
  • Catalyst Research Development
  • Office of Learning Scholarly Technologies
  • UW Technology
  • University of Washington

2
Session Outline
  • Part I Research overview
  • Part II Needs assessment usability
  • Part III Best practices institutional data

3
  • http//catalyst.washington.edu

4
Our Work
  • Web Tools
  • 13 modular tools
  • Free to UW community
  • Learning Spaces
  • Student computer labs
  • Computer classrooms
  • Advanced facilities
  • Knowledge Services
  • Center for Teaching, Learning, Technology
  • Faculty and student workshops
  • Research

5
Types of Research
  • Assessing user needs
  • Conducting usability studies
  • Identifying best practices
  • Gathering institutional data

6
Approaching Research as Service
  • Our research team
  • 1.5 staff members and 1 graduate assistant
  • Collaboration with designers, outreach staff, and
    anyone else we can find
  • Process
  • Unlearning what we learned about research in
    graduate school
  • Overcoming unique challenges
  • Developing new practices

7
Unlearning Research Myths
  • Myth 1
  • All research should be grounded in theory
  • Myth 2
  • All research must present your best thinking
  • Myth 3
  • Research is an independent or small team endeavor

8
(No Transcript)
9
Research Challenges
  • Frequent organizational change
  • Fitting research into development timeline
  • Does UW really want to learn best practices from
    the IT shop?
  • Perception of bias

10
Organizational Change
11
Discussion
  • Where does this type of research fit within your
    institution?
  • How do you currently gather information about
    technology use and needs?
  • Are there projects you are currently working on
    where it would be useful to add a research
    component?

12
  • Part II
  • Needs Assessment Usability Studies
  • or
  • How fast do you need that?

13
Needs Assessment
  • CommonView
  • Course or project workspaces
  • Integrate multiple Web tools maintain
    flexibility of use
  • Questions
  • How do Catalyst users integrate multiple tools?
  • How do users of courseware use features of those
    tools?
  • What are the unmet needs of both groups?

14
Challenge Time
  • Pace of development cycle
  • 1 month to develop study, recruit participants,
    and gather data
  • Our response
  • Collected internal evaluation data only
  • Expanded team designers outreach staff
  • Developed new focus group procedures
  • Streamlined data analysis

15
Focus Group Format
  • Traditional model
  • IRB approval for research study
  • Formal, written consent procedures
  • Audio-recording and transcription of session
  • New procedures
  • Evaluation data for UW internal use
  • No identifying information recorded
  • 1-2 note-takers per session
  • Observations added to notes/gaps filled

16
Data Analysis Process
  • By population
  • Meeting of leaders and note-takers
  • Review of notes
  • Documentation of findings
  • Most students reported accessing their course Web
    sites using bookmarks. They had difficulty
    finding the sites from computers other than their
    own.

17
(No Transcript)
18
Challenge Bias
  • UW committees want to know user needs
  • We have the most information
  • Can others trust our information?
  • Our response
  • Share overall summary of findings (incl. those
    unflattering to our group)
  • Detail purpose of study and limitations of
    findings
  • Explain research and development process

19
Usability Studies
  • Conducted by design team
  • Several times during design and development cycle
  • Challenge
  • Quickly finding sufficient numbers of diverse
    users to test tools
  • Our response
  • Creation of Catalyst User Experience (CUE)
    project

20
CUE
  • Began in 2007
  • 241 members
  • Initial membership survey
  • 11 targeted usability studies and surveys
  • CUE 60-90 response to invitations
  • Old methods 5-20 response
  • Biannual newsletter

21
(No Transcript)
22
(No Transcript)
23
Discussion
  • Examples of other fast-paced research techniques?
  • What strategies do you use to share information
    about user needs among IT groups and with other
    segments of university of college community?

24
  • Part III
  • Best Practices Institutional Data
  • or
  • Who wants to learn this stuff from the IT shop?

25
Identifying Best Practices
  • Inter/National Coalition for Electronic Portfolio
    Research (cohort 1)
  • 3-year study of ePortfolio use
  • UW contests
  • Identify spectrum of student use
  • Spring 2005 2007
  • Case study Expository Writing Program
  • Gather knowledge about implementation path
  • 2005/6 pilot 2006/7 option

26
Our Process
  • IRB Minimal risk research
  • Data Gathered
  • Interview transcripts for students, TAs, and
    administrators
  • Surveys of contest applications, EWP students,
    and EWP TAs
  • Review of student portfolios and instructional
    materials

27
(No Transcript)
28
Challenge Getting campus to listen
  • Are we the right people to be discussing best
    practices?
  • Is pedagogy our focus right now?
  • Our response
  • Practical reports for UW audience
  • Breadth of data can tell multiple stories
  • Academic publications for larger audience

29
Gathering Institutional Data
  • 2008 Faculty, Teaching Assistant, and Student
    Surveys on Learning and Scholarly Technologies
  • Campus-wide surveys
  • All instructors of record (approx. 4000)
  • Stratified sample (by broad disciplinary group)
    of 1000 TAs
  • Stratified sample of 5000 students

30
Challenge Time
  • How can we gather enough information to write
    up-to-date surveys when the survey itself is our
    main vehicle for gathering information on current
    needs?
  • Our Response
  • Autumn focus groups with faculty, TAs, students
  • IRB Exempt research
  • Utilized needs assessment focus group process
  • Expanded team staff from UW Technology,
    Libraries, and Office of Educational Assessment
  • Provided focus group training sessions for staff

31
(No Transcript)
32
Challenge Getting campus to listen
  • How can the survey results best inform
    technology-related decisions on campus?
  • Our response
  • Survey focus on goals and opinions, rather than
    simple snapshot of use
  • Extensive collaboration
  • Units who can implement changes are part of team
  • Practical reports and presentations for UW
    audience
  • Academic publications for larger audience

33
Discussion
  • Examples of other similar projects or challenges?
  • What strategies can we try for establishing
    research partnerships across campus?
  • What strategies can we use to share research
    processes and findings between institutions?

34
Contact Information
  • Cara Lane cgiacomi_at_u.washington.edu
  • Catalyst http//catalyst.washington.edu
About PowerShow.com