Paradox in a Graduate Program Delivered On-Line in a Human Science Curriculum

1
Paradox in a Graduate Program Delivered On-Line
in a Human Science Curriculum

• Janet Jeffrey RN, PhD Mina Singh RN, PhD
• School of Nursing
• York University
• Gail Lindsay RN, PhD
• School of Nursing UOIT
• Toronto

May 6th - 2009 CASN Nursing Research Conference
Funded by the Office of the Dean Faculty of
Health
2
PurposeMScN Program Evaluation Project

• To conduct program evaluation of the new on-line
  MScN program that is participatory with both
  formative (improve the quality of the program)
  and summative (determine the worth of the
  program) components, to ensure accountability to
  all stakeholders.
• To examine the design and delivery (process) of
  the MScN program as well as its outcomes ensuring
  depth as well as breadth given the proposed
  diverse data collection methods over time.
• This presentation focuses on the first cohort of
  students in the MScN program (2005-2007) and the
  faculty who taught in the first year of the
  program.

3
Background

• Masters in Science in Nursing (MScN) program was
  the first on-line graduate program at York
  University and in Ontario.
• The first cohort of students was admitted May
  2005 (after program approval by the Ontario
  Council on Graduate Studies in March 2005). So
  there was little time between application,
  admission, and starting the program for both
  faculty and students.
• Although distance learning has been available for
  almost 80 years, research has focused on
  singular issues in online education such as
  examining and/or measuring students experiences
  in terms of what matters to their learning

4
Context

• Philosophy of the MScN program is human science
  that values
• Lived experience and
• Faculty in relationship with students.
• Students are engaged in learning in an
  environment that is technology-based and at a
  distance.
• We were concerned that the online method
  of program delivery was somewhat
  contradictory to the human
  science perspective which could
  be more easily delivered
  face-to-face and wondered how
  students would get connected and establish
  relationships with faculty.

5
Research QuestionsProcess

• To what extent was the online MScN program
  implemented as developed/planned?

• What strengths and weaknesses of the program are
  identified by students and faculty?
• How is the online delivery congruent with
  students approaches to learning?
• In what ways are faculty-student relationships
  developed and maintained in the online learning
  environment?
• How well do resources support student learning
  and faculty development (Library, Centre for the
  Support of Teaching, Computer Help Desk
  Computer Services)?
• In what ways can implementation of the MScN
  program be improved?

6
Program Evaluation Design

• Qualitative approach
• focus groups and journal writing by both students
  and faculty during the program
• Quantitative approach
• - quasi-experimental design with questionnaire
  data collected at the time of students entry to
  the program and after graduation by both students
  and faculty (students and faculty serve as their
  own controls)

7
Sample

• Sample comprised of three groups
• Students admitted to the MScN Program to begin in
  May 2005.
• Faculty who are teaching the courses offered to
  this first cohort of students.
• Other stakeholders.

8
Data Collection - Students
Entry During Courses End
of Each Course After

Graduation
Journals (1 minute papers)

• Questionnaires
• Study Processes Questionnaire
• Edmonton Research Orientation Survey (EROS)

Course Assignments
Interviews

• Questionnaires
• Study Processes Questionnaire
• EROS

Focus group meeting beginning of each semester
Course Evaluation
9
Students Sample Size
2005 2006 2007
Full-time In program Participated in evaluation 8 5 4 3
Part-time Admitted Participated in evaluation 20 16 13 8 13 8

• graduated Fall 2006 7 graduated
  Fall 2007
• All students admitted to begin the program in May
  2005 were invited to participate in the study
• Some students did not really start the program
  others left for academic and/or personal reasons

10
Students Description

• 21 students originally agreed to participate in
  the evaluation
• 11 completed the evaluation of 28 of the first
  student cohort who completed their MScN (10
  females and 1 male)
• average age 45 years (SD 6 years range 35-55)
• All but two completed their baccalaureate
  nursing degrees from 2003 to 2005 the other two
  in 1978, 1998
• Average number of years of full-time work
  experience 19.45 (SD 9.4, range 8-32 years)
  primarily as clinicians
• 3 working part-time worked average of 9.3 years
  (SD 6.1, range 4-16 years)

11
Students Questionnaires

• Study Processes Questionnaire examines preferred,
  ongoing and contextual approaches to learning
  deep and surface strategies describe the way
  students engage in tasks
• Deep approach M 40.7 (SD 5.4) increased
  slightly (M 42.7, SD 3.1 NSS)
• Did the program attract these type of learners?
• Surface approach M 16.0 (SD 3.5) increased
  slightly (M 18.0, SC 7.8 NSS)

Biggs, J., Kember, D., Leung, D. Y. P. (2001).
The revised two-factor Study Process
Questionnaire R-SPQ-2F. British Journal of
Educational Psychology, 71, 133-149.
12
Students Questionnaires

• Edmonton Research Orientation Survey measures
  overall research orientation, how research is
  valued and strategies undertaken to promote/use
  research
• For 5 students who completed EROS at both times,
  Mean of 38 items prior to starting the program M
  2.78 (SD 1.3) and at graduation M 2.75 (SD
  1.0)
• For all 11 who completed EROS at baseline M
  1.61

Pain, K., Hagler, P. Warren, S. (1996).
Development of an instrument to evaluate the
research orientation of clinical professionals.
Canadian Journal of Rehabilitation, 9(2), 93-100.
13
Students Questionnaires

• Course evaluation provided information that was
  similar to what we learned from students in focus
  groups
• Online Learner Support Instrument (Atack,
  2001)
• Interaction with peers and teachers
• Course design and resources
• Technology and environment
• Perceptions of Learning Environments
  Questionnaire (QUT, 1994)
• What helped and hindered learning formatted for
  each course

Attack, L. (2001). Web-based continuing education
for registered nurses Clinical application and
learners' experiences, unpublished doctoral
dissertation, University of Calgary, Alberta,
2001. Queensland University of Technology (QUT).
(1994). The teaching and learning in tertiary
education (TLiTE) project. A report prepared for
the Teaching and Learning Committee, Queensland
University of Technology, Australia.
14
Data Collection - Faculty
Entry During Courses/Program
Delivery First Graduation
Journals (1 minute papers)
Questionnaire Approaches to Teaching Inventory
Questionnaire Approaches to Teaching Inventory
Focus group meeting monthly during semester

• All faculty teaching the first full-time cohort
  of students were invited to participate in the
  study - teachers of the required courses,
  electives, and major research projects.
• Almost 90 are participating.

15
Faculty Questionnaires

• Approaches to Teaching Inventory) measures ways
  in which teachers approach their teaching
• Conceptual change -- student-focused
• Prior to teaching M 32.0 (SD 3.6) stayed
  stable over the first year of the program
• Information transmission -- teacher-focused
• Prior to teaching M 13.5 (SD 3.1)
  unexpectedly increased M 15.8 (SD 2.9 p
  .018)

Prosser M., Trigwell K. (1999). Understanding
learning and teaching the experience in higher
education. Buckingham Open University Press.
16
Student Perspectives on Achievements, Challenges,
and Recommendations

• Analysis of journal writing and focus group
  meetings lead to list of achievements,
  challenges, and recommendations.

17
Student PerspectivesAchievements Beginning the
MScN Program

• Getting started (getting course readings/text,
  familiarity with WebCT, powerpoint)
• Getting organized
• Completing academic work on-time, meeting
  deadlines (work and school)
• Developing sense of community with other students
  in the course to provide support
• Balancing demands of work, family, and student
  roles
• Remaining in the program over time, Hung in
  there until the end
• Obtaining good grade(s)

18
Student PerspectivesChallenges Beginning the
MScN Program

• Time management, keeping up with multiple
  responsibilities
• Navigating the system at York inconsistent
  access to virtual lab and on-line library
• Silence of peers teachers
• Mastering group conflict
• Learning the language of human science
• Fitting in and being seen as worthy within the
  program
• Sacrificing others areas of life to complete
  academic work
• one more paper and Mommy will be done
• Fear of posting on-line
• Emotional toll depression, anxiety, fear
• Embodied toll (somatic complaints) e.g., carpal
  tunnel syndrome, arm tingling

19
Student PerspectivesAchievements Continuing
into2nd Semester

• Logistics of starting courses getting readings,
  texts etc
• Pacing, keeping up in course work
• Staying on top of timeline/deadlines
• Balance home/work/school
• Staying motivated to participate
  in the course

20
Student AchievementsDifferences from Entry to
Program

• Made it through the summer,
• Survived,
• Still going strong
• Persevering

21
Student PerspectivesChallenges Continuing
into2nd Semester

• Expectations for participation on on-line
  discussions (different for each course)
• Lack of support and feedback
• Timeliness of posting from peers in response to
  assignments

22
Student Challenges Differences from Entry to
Program

• Content of second term like learning new language
  (in new course)
• Keeping peace and harmony within group,
  competition and hostility within group
• Balance

23
Student Achievements1 Year Later

• Students supporting and helping each other
• Building of student network
• - Online
• - Phone
• - Workplace
• Persevering through the storm
• - New love for research is probably my greatest
  achievement..
• - Always finishing the course
• - Teaching sessional position

24
Student Challenges 1 Year Later

• Not enough time between courses
• Very tired, back to back semesters intense
• Workload heavy
• Accessing virtual software problematic
• Course selection restrictive
• Course objectives and assignments
  still disjointed
• Issues re use of preceptors

25
Lessons Learned1 Year Later

• Liked the philosophy of the program
• Convenient, accessible
• Review of courses
• PLEQ
• Helped
• Presented on online symposium
• Feedback obtained
• Colleagues experiences
• Hinder
• Too much theory
• Online discussions were repetitious
• Reading others notes
• Others not contributing

26
The First Graduates Look Back

• More time, more time..
• Student input was valuable
• Human to human close relationship is valuable
• Workload phenomenal
• Learned.

27
Faculty Perspectives on Achievements, Challenges,
and Recommendations
Analysis of journal writing and focus group
meetings lead to list of achievements,
challenges, and recommendations.
28
Faculty Achievements

• Program activated in 6 weeks
• Finding resources for learning online and course
  development
• Establishing congruence between course processes,
  content and learning outcomes
• Responding to students formative evaluations
  flexibly
• Surviving the workload
• Bringing our own research (content and method) to
  curriculum

29
Faculty Challenges

• No dedicated, timely, expert online education
  expertise designated
• Courses not designed with web expertise,
  multi-media rare
• Lack of peer consultation feedback available
• Overall workload finding uninterrupted time to
  work on course
• Context of MScN Program in School of Nursing
• How to see student engagement with the course and
  their learning
• Facilitating on-line small group conflict
  resolution
• Feeling disconnected, ambiguity about learning,
  lost in space

30
Lessons Learned

• Preparing an online class is like slow motion
• Be prepared to draft, redraft, edit, rewrite,
• Many faculty did arrange face to face meetings
  with students at least once during the course
• Formalize CST, ACS, FSC secondment/assignment
  to MScN Program
• Obtain work study students to be web/media
  designers
• Concept map the program using the first course
  outlines to evaluate coherence, development, etc.
  
• Explore shared formats for participation and
  other evaluation processes
• Finding a balance between students expressed
  wish for faculty participation
• Professional development for faculty

31
Mirroring . Paradox

• Faculty complain of feeling disconnected from
  students Students complain about faculty
  availability.
• Technology-enhanced learning is its own subject
  matter Need for ongoing professional/faculty
  development and support.
• At the start-up of each semester, some of the
  teachers are novice in the online environment
  Students have cumulative expertise over time as
  their experience is continuous through the
  program.
• Technology, while at a distance, is not
  disembodied Faculty report tendonitis, eye
  strain, and fatigue as well as emotional stress.

32
Actions Taken Moving Forward

• Students contribution to formative evaluation
  resulted in changes to
• orientation for incoming cohorts of students (now
  online)
• way in which WebCT platform is being used for
  course delivery since changed to Moodle
• Subsequent curriculum development undertaken
• Concept map the program using the first course
  outlines to evaluate coherence, relationship to
  program goals
• Explore shared formats between courses for
  participation and other evaluation processes
• Consideration of how students work in groups
  on-line

33
Actions Taken Moving Forward

• Improved access to York systems (virtual lab,
  library resources, learning how to use new
  software)
• New option by university bookstore to order texts
  online for free delivery prior to courses
  starting (coincidental)
• Development of two delivery options to match to
  students preferred way of learning
• on-campus (face-to-face) option to start Fall
  2009
• distance option fully online no presence on
  campus

34
Conclusions Implications

• It is important to act on lessons learned
  formatively and summatively and to ensure
  continuous program evaluation. Evaluation has
  resulted ongoing shaping of the curriculum and
  how it is being delivered.
• Ongoing evaluation of the purpose of technology
  philosophically pedagogically must be
  undertaken to explore the paradoxes of
  implementing a human science curriculum in an
  online environment.

35
For Further InformationContact

• Janet Jeffrey
• jjeffrey_at_yorku.ca
• [phone] ext 66696
• School of Nursing, HNES
• York University
• 4700 Keele St.,
• Toronto, Ontario M3J 1P3
• Canada