Title: Impact of A Websupported Course Delivery Model on Approaches to Teaching and Learning from the Instr
1- Impact of A Web-supported Course Delivery Model
on Approaches to Teaching and Learning from the
Instructor Perspective
Norm Vaughan Jim Zimmer Mount Royal
CollegeWinter 2001
2Overview
- Study Background
- Research Purpose
- Conceptual Framework
- Methodology
- Results
- Conclusions
- Discussion
3Study Background
- assessment of web-based course development tools,
Fall 1998 - pilot test of Blackboards CourseInfo, Fall
1999 - research project funded by MRCs Scholarly
Pursuits Committee
4Number of MRC CourseInfo Sites
Currently 310 course sites
5Research Purpose
- To study the impact of web-supported course
delivery - on teaching-learning environments from the
instructor - perspective. Specifically
- 1. Training and support needs
- 2. Most and least useful features of the course
development - tool
- 3. Impact on teaching-learning methods/behaviours
- 4. Perceived impact on student learning
- 5. Instructor satisfaction
6Conceptual Framework Flashlight Model
Conditions Favorable to Learning
Learning and Other Outcomes
Learner goals, needs and characteristics
Organization of teaching and learning
Selection of Appropriate Technology
7Conditions Favourable to Learning
- Seven Principles of Effective Practice
- frequent student-teacher contact
- cooperation and collaboration among students
- active learning
- rich and rapid feedback
- time on task
- high expectations
- respect for different learning styles
- Enabling Conditions
- access to technology
- pre-requisite skills, training, ongoing support
8(No Transcript)
9Study Population and Methodology
- Instructors using CourseInfo during Fall 99 and
Winter 2000 - 63 item survey
- 28 faculty respondents
- Focus Groups x 2
- Items drawn from TLT Groups Flashlight Current
Student Inventory
10 Study Results
11Demographics
- Instructors from following departments
- Arts - Humanities Behavioural Science English
- SciTech - Computer Science, ChemBio
- Community Studies - Community Rehab, Social Work,
Sports Management, Recreation Studies - Center for Communications - ACOM
- Center for Health Studies CCNP Advanced
Specialty Health Studies
12Computer Skills and Access
- Computer Skills - Self Rating
- Novice Intermediate Advanced No Answer
- 15 77 8 0
- Home Internet Access
- Yes - 77 No - 23 No Answer - 0
- CourseInfo Access
- Off-campus On-campus Both,equally
- 15 54 31
-
-
13Research Question 1
Instructor training and support needs
14Mastering CourseInfo
- Hours to learn how to begin using it...
- 0 to 3h 4 to 6h 7 to 9h gt10h No Answer
- 76 8 8 8 0
- Time until comfortable with it...
- 0 to 2wk 3 to 5wk gt6wk No
Answer - 70 15 15 0
-
15Training and Support NeedsComments
-
- Im an historian. I learned it by printing off
and reading the online manual. - I learned how to use it through conversations
with ADC and by trial and error. When I did have
a technical problem I used the listserv. - Im a self confessed workshop junky like to
learn in a social setting where I can view other
instructors sites and discuss teaching ideas
with other faculty. - I learned most effectively from another faculty
member who had used it before.
16Training and Support NeedsSuggestions
- Have ADC workshops focus on teaching as well
technical issues of CourseInfo - Continue ADC drop-in support
- Continue listserv - posting common problems
- Develop FAQ page--for students and instructors
linked to the CourseInfo home page
17Training and Support NeedsConcerns
-
- Need more time to learn and plan. Feel like Im
learning on a need to know basis right now,
rather than planning an overall concept and
proceeding. - Biggest frustration is the time to learn all the
functionality--want to learn more but how much
time can you put into it? - I was frustrated at not being able to get a lab
for training. I had to get the students to come
in outside of class time. Most of them did but we
shouldn't have to do that. - Problems with lab access are going to get worse
as more and more faculty and students are using
CourseInfo
18Research Question 2
Most and least useful features of CourseInfo
19Features of CourseInfo
- VU or Somewhat Not No
- Useful Useful Useful Answer
- Online grade book 87.5 0 0 12.5
- Course documents 84.6 7.7 0 0
- Links to external web sites 70.0 20.0 0
10.0 - Statistics about student usage 63.7 27.3
0 9.1 - E-mail links 55.5 22.2 11.1 11.1
- Instructor help manual 57.0 14.3 14.3
14.3 - Discussion board 50.0 12.5 25.0
12.5 - Digital drop box 42.9 42.9 0
14.3 - Student home pages 33.3 33.3
16.7 16.7 - Online quizzes 20.0 60.0 0
20.0 - Group tools 14.3 42.9 28.6 14.3
- Student calendar 14.3 28.5 42.9
14.3 - Virtual chat 0 25.0 50.0 25.0
20Features of CourseInfo Course Documents
- Students really appreciated access to lecture
notes, summaries. - Having access to notes enabled students to listen
more carefully, participate in the discussion. - Less need to disseminate information - allowed
for activities like group work and skill
development. - Changed the dynamics of my class - allowed the
class to be more action-oriented. - Allowed me to be more organized - clearer focus
of where I was going with my course. - Didnt deter attendance in my experience.
-
21Features of CourseInfo Discussion Forums
- I used the discussion forum to introduce ideas
that dont come up during class time, or that we
didnt have time for - Often used comments from the discussion forum to
spark discussion during class time - Seemed to encourage studentstudent mentoring
- Allowed students to hear more voices and make
connections and friends with other students - Helped to link theory to practice in my class -
bridge classroom to practicum - I think online discussion is valuable but many
students are not participating (even for 5). I
don t know why.
22Features of CourseInfo - Concerns
- Interface and Navigation - would like more
flexibility and control - Course documents - posting documents in native
format vs. converting to html format - Virtual chat tool -slow unstable inconvenient
- Digital drop box - doesn't work with older
versions of Netscape or IE - Assessment tool - in the construction of a quiz -
if you accidentally hit the back button - you
lose all of your quiz data and have to start
again - Gradebook tool - not very flexible (e.g. cant
weight grades)
23Research Question 3
Teaching-Learning Methods and Behaviours
24Student-Instructor Contact Student-Student
Interaction
- In comparison with other More About
Less Not - courses not using Likely
Same Likely Sure - CourseInfo, how likely
() () () () - were your students in this
- course to . . .
- . . . interact with instructor 54
31 0 8 - . . .participate effectively in
46 23 0
23 - in-class discussions
- . . . discuss ideas and concept
39 15 0
39 - with their classmates
- . . . work on assignments
23 39 0
31 - with other students
-
25Related Comments
- Brought out the reserved students, online and
in-class too - Discussion board enabled students to see issues
from multiple perspectives, and encouraged
connections - Used the group tools to support collaborative
writing projects generally worked well but
students without home Internet access felt
disadvantaged - I put an anonymous CAT up and they could fill it
out as often as they wanted I think the students
felt they had a constant channel to me at
points in the course they would express certain
tensions that we could talk about in class
26Related Concerns
- Getting students to post messages of depth was a
challenge - Made me a little too accessible--lots of junk
e-mail - Evening virtual office hours appreciated by
students, but did increase my workload, my sense
that even at home my work wasn't ever done - CourseInfo has some great e-mail tools, but
unless the college supplies the student with
e-mail access theyre not useful
27Active Learning
- In comparison with other More About
Less Not - courses not using
Likely Same Likely Sure - CourseInfo, how likely
() () () () - were your students in this
- course to . . .
- . . . use the Internet as a 62
0 0 23 - source of course-related
- information
- . . . Independently search for
- answers to your questions 39
15 0 39 - rather than ask other students
- or instructor
-
-
28Related Comments
- Students came better prepared--they would go to
the site and read the outline of what I thought
was important. - Student access to course documents meant that I
could hit the highlights of the theory and use
class time for interactive exercises and
discussion. There was a real difference in what I
could do in class. - I can cover more ground in the course because the
students cover more material independently
outside of class time. - Mandatory or optional for students? Issues of
student workload and unequal access to computers
and Internet. -
-
-
29Research Question 4
Perceived Impact on Learning, and Other Outcomes
30Perceived Impact on Learning
- Because of the way I used Agree
Disagree Not Sure CourseInfo,
students . . . ()
() () - . . . had access to a greater variety of 92
0 8 - learning materials
- . . . were better able to understand the 62
0 38 - ideas and concepts taught in
- this course
- . . . acquired skills that will be
62 8 30 - useful in their chosen profession
- . . . spent more time on course related 54
8 38 - work
31Related Comments Concerns
- I sensed students giving more time to their
learning and taking more responsibility - Enhanced the generic skills, especially computers
and information research. - Students commented that using the site forced
them to learn basic computer skills. This is
positive. I had students who wouldn't know how
to turn on a computer. - External web links opened up another world to my
students forced them to move beyond the
classroom - CourseInfo was only one of the computer tools
used. I cant say that it produced changes
either way. The computer lab situation and
CourseInfo proved distracting for some, drawing
them away from course focus at times.
32Access, Flexibility and Convenience
- Increased convenience for both myself and the
students we knew where all the course documents
were - Easily modifiable from anywhere with web access
great convenience for instructor - Students really appreciate access to course
materials from anywhere at anytime, and so do I - One of the best things was that it was easily
modifiable by the instructor--dont have to rely
on I-S for simple updates - Liked the practice tests being marked for me so I
could just focus on patterns of how they're
doing. - Concerned about the student computer hardware and
software requirements to run CourseInfo - certain
students are loathe to use the student labs.
33Research Question 5
Instructor Satisfaction
34Instructor Satisfaction
- Agree Disagree Not
Sure - () ()
() - ...CourseInfo offered advantages
92 0 8 - over the way I previously taught
- my course.
- ...Students seemed to enjoy using 69
23 0 - CourseInfo
- ...I would use CourseInfo with 85
8 0 - another class or course
- ...I would recommend the use of 85
8 0 - CourseInfo to other instructors
- ...Overall, I thought the use of 85
8 0 - CourseInfo enhanced my course
35Related Comments
- Overall, really happy with it and look forward to
using it again. - Very positive experience. Am using it in all of
my courses. - The simplicity was great--took a fragmented group
of technologies and assembled them in one place. - CourseInfo definitely added to my workload but
it's worth it Im building my skills and
growing my web site with time.
36Related Concerns
- Increased workload and time issue - developing
the site keeping external links up to date
increased availability to students via phone and
e-mail - technical trouble shooting student
issues - With 210 students I get so many e-mails, delete
lots and sometimes miss the really important ones - Student expectations of rapid response
- Not a great synchronous tool works very well
asynchronously - Too early to know whether all the extra hours are
worth the effort. If it makes the student
experience more positive, then its worth it.
But if Im just adding to their workload, and
mine, its not worth it. - Student accessibility a concern off-campus
Internet, dated HW and SW
37Conclusions and Study Limitations
38Conclusions
- Majority of instructors felt that their use of
CourseInfo had enhanced the teaching-learning
experience in a number of ways - more student-student and student-instructor
contact - students more actively involved in the learning
process - increased flexibility and convenience for faculty
and students - development of generic skills
39Conclusions (contd)
- Instructors generally found CourseInfo easy to
learn and easy to use, but noted that - variety of learning styles necessitates variety
of training and support mechanisms for faculty - limitations with various aspects of the tool they
would like addressed - frustration with limited computer lab access for
student training/orientation sessions - overall effectiveness of the tool is compromised
by lack of college e-mail system
40Conclusions (contd)
- Results suggested that use of CourseInfo was
associated with teaching-learning approaches
conducive to student learning - increased student-instructor interaction
- increased student-student cooperation and
collaboration - increased active learning
- increased student time-on-task
41Conclusions (contd)
- Majority of instructors perceived student
learning gains in - understanding of course materials
- skills useful in their chosen profession
- generic skills, notably computing, knowledge of
the Internet, and information research skills - Additional research is needed to verify and
further describe impact of web-supported approach
on student learning outcomes
42Conclusions (contd)
- Majority of instructors were satisfied with their
experience, would recommend it to their peers,
and would engage in the experience again - Significant concerns about
- Workload implications
- training
- development of materials
- delivery
- maintenance
- Student access to computers and Internet, on and
off campus
43Study Limitations
- Small sample size
- Sample weighted with early adopters
- Reliance on self-report measures
- Halo effect?
- Cross-sectional design no cause-effect
relationships
44Follow Up
- Instructor Support
- ADC drop in, Monday-Friday, 9am to 2pm in T110
- One hour workshops combining technical and
pedagogical aspects of Courseinfo - Instructor listserv - sharing teaching ideas and
resources, trouble shooting technical issues and
concerns - MRC's Virtual Centre for Online Learning and
Teaching - www.mtroyal.ab.ca/adc
- CourseInfo's online instructor manual and
resources
45Follow Up
- Instructor Support
- CourseInfo tutorial modules on the web (ADC web
site) - One Day Workshops (Reading Week, May, August)
- CourseInfo's online instructor manual and
resources - Instructor FAQ in development
- ITS pilot of web-based email system
46Follow Up
- Student Support
- START Program (T106, phone 7214)
- Available to instructors to provide lab-based
orientation to CourseInfo - Drop in support for students, Mon-Fri 10am-2pm in
the START area, T106 - One hour scheduled workshops, led by START
students on various topics related to CourseInfo - CourseInfo's online student manual and resources
- Student FAQs now available (courseinfo.mtroyal.ab.
ca)
47Future Study
- The Effect of Online Discussion Forums on the
Teaching-Learning Process from the Student and
Faculty Perpective - Study to begin in winter 2001 semester
- Looking for interested instructors and their
students to participate in this study
48Questions, Comments, Discussion
49Key Contacts
- Faculty Training Support
- Andrew Reil (7203)
- Student Training Support (START)
- RuthAnne Scaddan (7214)
- New Course Creation and Accounts
- Brenda Quantz (7231)
- Technical Problems Support
- Help Desk (6034)
50Reference
- Chickering, A. Ehrmann, S. (1997).
Implementing the seven principles technology as
a lever. - www.aahe.org/technology/ehrmann.htm
- MRC CourseInfo Study Student Perspective
- www2.mtroyal.ab.ca/nvaughan/norm/courseinfostuden
t/v3_document.htm - MRC CourseInfo Study Instructor Perspective
- www2.mtroyal.ab.ca/nvaughan/norm/courseinfoinstru
ctor/v3_document.htm