Online%20Learning%20Tools%20(workshop) - PowerPoint PPT Presentation

View by Category
About This Presentation
Title:

Online%20Learning%20Tools%20(workshop)

Description:

Geoff Swan (ECU) WebCT ... Geoff Swan. Physics Program. School of Engineering ... Swan, G.I. (2003). Regular motion. In G.Crisp, D.Thiele, I.Scholten, S.Barker ... – PowerPoint PPT presentation

Number of Views:70
Avg rating:3.0/5.0
Slides: 35
Provided by: SOE1
Category:

less

Write a Comment
User Comments (0)
Transcript and Presenter's Notes

Title: Online%20Learning%20Tools%20(workshop)


1
Online Learning Tools (workshop)
  • John OByrne (Sydney)
  • Mastering physics software 40 min
  • Alex Merchant (RMIT)
  • Online assignments 20 min
  • Geoff Swan (ECU)
  • WebCT Quizzes 20 min

Key Issues in Learning and Teaching in
Undergraduate Physics A National Workshop
(1600 to 1730 Sept 28, 2005, University of
Sydney).
2
WebCT Quizzes Points of view
  • Student viewpoint ACTIVITY
  • Do the autc05 quiz at least twice checking grades
    and individual question feedback
  • Use student logins passwords (autc1 to autc12)
  • http//thomson4.webct.com/public/swanserwaycowan/i
    ndex.html
  • Staff viewpoint Show
  • Quizzes individual questions (eg ECU example)
  • Creating new questions and quizzes?
  • Course management
  • Results (graph) download
  • Other?

3
THE END
4
Key to Participation
  • Geoff Swan
  • Physics Program

School of Engineering and Mathematics Faculty of
Computing, Health and Science Edith Cowan
University Perth, AUSTRALIA
AIP Congress, Canberra ACT (Feb 1, 2005)
5
Outline
  • PART 1 Setting the scene
  • Background
  • Online quizzing environment
  • 2003 trial
  • PART 2 Online quizzes in 2004
  • Major Adjustments from trial
  • Results
  • Participation rates
  • Unit grades
  • Student evaluation
  • Open responses issues
  • PART 3 Discussion

6
Background
  • Setting Physics of Motion first year first sem
    physics unit
  • Engineering, Aviation, Physics major Education
    students
  • Assessment 60 Exam, 20 Labs, 20 test and/or
    assignments?
  • Homework selected end of chapter problems
    (Serway Beichner/Jewitt). Not assessed.
    Students have worked solutions
  • Problem (anecdotal) Struggling students do
    insufficient physics at home.
  • Time Competing demands study vs work vs family
    etc
  • Motivation (affected by many factors)
  • A Solution? Provide regular online quizzes
  • Engage students in regular problem solving
  • Help students keep up with the curriculum
  • Aim at struggling students increase confidence
  • Encourage good study practices early (ie sem 1,
    year 1)
  • Provide early continuous feedback (positive and
    negative)
  • Improve students overall performances

7
Motion Quizzes Trial. Sem 1, 2003
  • Quiz No. Topic(s) Set Week Set
    Assessment
  • 0 ECUGUEST check out ID Password quiz
    site 1 NONE
  • 1 Motion in one dimension, Vectors 2 Highest
  • 2 Motion in two dimensions 3 Average
  • 3 Newton's Laws
    4 Average
  • 4 Circular motion dynamics,
    6 Average
  • Work Kinetic Energy
  • 5 Conservation of Energy, Momentum 7 Average
  • -------------------------------------mid-semester
    break-----------------------------------------
  • 6 Rotational Motion 9 Average
  • 7 Static Equilibrium, Elasticity, 11 Average
  • Oscillatory Motion
  • 8 Gravitation, Fluids 12 Average
  • Notes 12 week term. Max 70 minutes for each quiz
    (4-6 questions). Available after topic material
    covered. Allowed about 8 days to complete up to 2
    attempts.

8
Student orientated 1
  • Flexible delivery
  • Attempt any place (home or ECU or ?) any time
  • Fit in with other commitments
  • Guest quiz
  • Rewards
  • Intrinsic better understanding
  • Assessment use to improve final grade (if better
    than mid-semester test 20)
  • Revision have extra revision questions and
    answers for exam
  • No access at all if online quiz not attempted
    within timeframe

9
Student orientated 2 (Feedback)
  • Feedback
  • General (how am I going in this physics unit?) -
    Grades
  • Specific (how do I answer this question?) -
    Detailed Feedback!
  • NO detailed feedback provided by publishers test
    bank
  • Immediate Feedback (within seconds!)
  • Grades and detailed feedback
  • Any time and any place
  • Can act straight away on detailed feedback
  • Attempt a similar but different quiz (2003 wait
    of 30 minutes)
  • strike while the iron is hot!
  • Can improve grade
  • Formative assessment
  • Significant advantages exist with an online
    environment

10
Online advantages for the Lecturer
  • Save TIME? (for large classes)
  • existing publisher supplied WebCT test bank (1000
    questions but no detailed feedback!) and server
  • automatic marking
  • results easily transferred to excel mark sheet
  • Reliable assessment
  • Individual tests (similar but different)
  • Alternate multiple choice questions
  • Calculated questions (truth tables - usually 20
    variations)
  • individual quizzes harder to cheat plus 2nd
    attempt option
  • Flexible settings
  • questions, assessment, access and timing options
  • Belief Both pedagogical and practical advantages
    to using the online environment

11
WebCT Question types
  • Multiple choice
  • 1000 question test bank (no detailed feedback)
  • Calculated
  • Use formula to create truth table.
  • ie same question but (say) 20 alternate sets of
    numbers
  • Supported math operators
  • ( ), , -, , /, , sqrt, sin(x), cos(x), exp,
    log (base e ie ln)
  • Also available (but not used)
  • Paragraph (tutor marked)
  • Matching
  • Short answer

12
Example Masses on strings 1
13
Example Masses on strings 2
14
FLAG Your opinion??
  • RATE the detailed/general feedback
  • Consider a borderline first year first semester
    physics student
  • to SOLVE this question FOR THEMSELVES in a second
    attempt
  • Too much detail
  • About right
  • Not enough detail
  • Compare with the person next to you.
  • How does this compare with a worked solution?

15
2003 online quizzes trial - RECAP
  • Online quizzes Physics of Motion trial in sem 1,
    2003
  • Engage students with regular problem solving
  • 8 quizzes. Two attempts allowed (average mark
    normally used)
  • Optional assessment (if better than mid-sem test)
  • WebCT server (supplied by publisher)
  • Two types of questions
  • Calculated
  • Use formula to create truth table.
  • ie same question but (say) 20 alternate sets of
    numbers
  • Supported math operators
  • ( ), , -, , /, , sqrt, sin(x), cos(x), exp,
    log (base e ie ln)
  • Multiple choice
  • Based on 1000 multiple choice question testbank
    (no detailed feedback)
  • Detailed Feedback written for each question

16
2003 Results - SUMMARY
  • Students LIKED the Quizzes
  • 85 Quizzes easy to access, relevant, improved
    my understanding of physics, helped me develop
    problem solving skills etc (N29 with at least
    one quiz)
  • 80 Detailed feedback about right (20 not
    detailed enough)
  • Participation was POOR! (Of N63 students who
    attempted the exam)
  • 17 attempted 6,7 or 8 quizzes
  • No fails in this qroup! One student passed
    directly due to quizzes!!
  • 21 attempted 4 or 5 quizzes , 32 attempted 2 or
    3 quizzes , 6 attempted 1 quiz
  • 24 attempted ZERO quizzes.
  • Four times more likely to fail?
  • the 19 students who completed zero or one quiz
    accounted for two thirds of the units fail
    borderline pass grades (ie 63 of these students
    compared with 14 of students who completed more
    than one quiz)
  • Why zero quiz attempts? Lack of TIME!
  • Main Factors given were (N7) bloody hard,
    laziness, problems with internet connect -
    difficult to complete quizzes, time, dont have
    time/hate physics anyway, hard to allocate solid
    90 (sic) minutes in first few weeks and never got
    into it, work.
  • MUST increase participation rate in 2004!

17
Changes in 2004
  • ASSESSMENT
  • Compulsory assessment 20
  • Replaced mid-semester test (now just for
    practice)
  • Highest mark rather than average of two attempts
  • From 1 of 8 quizzes in 2003 to 6 of 8 quizzes in
    2004
  • Big incentive high marks possible for
    struggling students
  • OTHER
  • Orientation quiz in week 1 (not assessed)
  • Quiz time decreased (from 70 to 60 minutes)
  • New questions topics rearranged
  • Research additional survey items on
  • The quiz environment for students how they went
    about it
  • The use of detailed feedback on individual
    questions

18
Quizzes in sem 1, 2004
19
Participation Rates
  • Participation how many attempted at least 6 of
    8 quizzes?
  • Consider only students who sat end of semester
    exam
  • 2003 N63
  • 2004 N84
  • 2003 Just 17
  • 2004 85
  • HUGE increase in participation rates

20
Quiz and Unit Results
  • Consider only students (N84. sem 1, 2004) who
    sat exam
  • Pass rates
  • Quizzes Pass rate 70 (Distinction rate 43)
  • Note 15 students with 5 or less quizzes all
    failed here
  • Unit Pass rate 81 (Distinction rate 27)
  • 27 students received a distinction for the unit
  • 87 also received quiz distinction (ie 20 of 23
    students)
  • 19 students failed unit
  • 75 also failed quizzes (ie 12 of 16 students)
  • Highest quiz mark for failing student was 58
  • Below median average for quizzes
  • Quizzes as a good indicator for success and
    failure
  • Use for early intervention!?

21
Student evaluation of quizzes
  • Students were extremely positive ( agreement)
    Respondents (N34) with at least 6 or 7 quizzes.
    Survey in penultimate teaching week.)
  • Were easy to access (97)
  • Were relevant to the unit content (97)
  • Improved my understanding of physics (94)
  • Provided me with necessary practice in solving
    physics problems (91)
  • Helped me develop problem solving skills (94)
  • Detailed feedback was helpful (91)
  • Overall, helped me learn physics (94)
  • Students normally accessed the quizzes from home
    (70) and 91 of students thought that the number
    of quizzes (eight) was about right.

22
Assessment
  • Assessment by online quizzes was fair and
    reasonable (71)
  • Future assessment
  • 71 online quizzes only
  • 29 combination of online quizzes and mid-sem
    test
  • 62 prepared more thoroughly for the two average
    mark quizzes

23
Detailed Feedback
  • For my learning needs, the amount of feedback
    provided to assist me in solving the problem for
    myself was
  • About right 67
  • Not detailed enough 33
  • Consider small increase for 2005?
  • 81 found detailed feedback sometimes useful for
    correctly answered questions! Interesting
  • Not expected by author, but interesting!
    Subsequent conversations reveal students like to
    compare the lecturers method (concepts used and
    application) with their own regardless of whether
    they answered the question correctly or not.

24
Open response questions
  • Provided additional information on
  • Preparation
  • Detailed feedback
  • Collaboration
  • Study habits

25
Preparation?
  • Students listed a variety of ways in which they
    prepared for highest mark quizzes including
  • attending lectures
  • reading through lecture notes
  • reading chapters or chapter summaries in the
    textbook
  • doing set problems and even
  • reading the formula sheet.
  • A significant minority were not well prepared,
    including one student whose preparation was
  • Nothing, I would waste the first go work
    backwards from the solution
  • unlikely to result in any meaningful learning
  • unlikely to improve their exam score.

26
Detailed feedback?
  • How did students use detailed feedback? Three
    examples
  • To change the way I look at the questions
  • To try and understand the concepts, but
    sometimes it wasnt enough
  • I used it as a guide to lookup certain
    information in the book and notes. I found it
    very good
  • Student usage went beyond just getting the
    correct answer.
  • Author was surprised and pleased!

27
Collaboration?
  • 44 provided help to other students
  • 36 received help from other students
  • some students seemed somewhat insulted with these
    questions!?
  • Collaboration ranged form nothing or verbal
    to substantial. Consider two substantial
    examples
  • 1. We all worked out an answer, compared the
    way we did it the answers we got. It was
    through this that we were able to come to grips
    with theories
  • 2. Group work. Do the quizzes together and work
    together to a solution
  • FLAG Should the collaboration described above
    (considering example 1 example 2 separately)
    be
  • Encouraged (peer learning)
  • Neutral
  • Discouraged (its cheating!)

28
Further thoughts on collaboration
  • Students already collaborate in class time
  • Labs experiments done in small groups
  • Lectures short small group discussions on
    (mainly conceptual) questions.
  • Collaboration/group work/peer learning
  • Necessary part of the curriculum
  • Supported by large body of literature resources
    (eg Mazur, McDermott etc)
  • Attitudes changed over time
  • Individual with group support now OK
  • Original object was to replace mid-semester test
    (20 assessment)
  • As long as individual learning occurs and each
    actual quiz attempt is done by the student logged
    on to the computer
  • Delete second part of condition?? Students
    spending time interacting with and discussing
    physics is always a good thing!?

29
Study habits?
  • Students fairly positive here
  • Helped me work more consistently over the
    semester (85)
  • The quizzes made sure that we were studying
    without forcing us to study
  • The quizzes encouraged and influenced both study
    at and away from the computer.
  • Consider preparation for quizzes reported by
    students.

30
Conclusions
  • Provide opportunities to learn through
  • Detailed feedback
  • Formative assessment
  • Collaboration
  • Influence study habits
  • help students work consistently over the semester
  • Flexible time and place
  • Overall help students learn physics (94)
  • ECU evaluation (UTEI). BIG increase in overall
    satisfaction for unit. Partly due to quizzes?
  • From 64 (N28) in 2003 to 91 (N45) in 2004
  • Agree or strongly agree on 5 point scale (ie
    neutral counts as not satisfied)
  • Online quizzes have made a difference
  • Adapted for second semester unit Waves and
    Electricity 2004
  • Continue for 2005

31
FURTHER INFORMATION?
  • Contact
  • Dr Geoff Swan
  • Physics Program, SOEM
  • Edith Cowan University
  • 100 Joondalup Drive
  • Joondalup WA 6027
  • Tel 61 08 6304 5447
  • Email g.swan_at_ecu.edu.au
  • Quiz site at http//thomson4.webct.com/public/swan
    serwaycowan/index.html (login and
    passwordecuguest)

32
Discussion?
  • How might these online quizzes be improved?
  • Group work vs individual
  • Context Questions in the quizzes are mostly
    de-contextualised. Is there a pedagogical
    justification? (eg can reality be a distracter to
    learning physics?)
  • How might resources like this be shared between
    institutions? (AUTC report recommendation)
  • Students study habits and online quizzes
  • Is it our business to design curriculum to
    encourage good study habits and/or teach study
    techniques?
  • Are we responsive enough to competing demands on
    students who often combine uni work family?
    (AUTC report recommendation)

33
Paper references
  • References
  • Gordon, J.R., McGrew, R., Serway, R.A. (2004).
    Student Solutions Manual Study Guide to
    accompany Volume 1 Physics for Scientists and
    Engineers Sixth Edition. Belmont Brooks/Cole.
  • Honey, M., Marshall, D. (2003). The impact of
    on-line multi-choice questions on undergraduate
    student learning. In G.Crisp, D.Thiele,
    I.Scholten, S.Barker and J.Baron (Eds.),
    Interact, Integrate, Impact Proceedings of the
    20th Annual Conference of the Australasian
    Society for Computers in Learning in Tertiary
    Education. Adelaide, 7-10 December 2003.
  • McInnis, C, James, R, McNaught, C (1995). First
    year on campus Diversity in the initial
    experiences of Australian undergraduates.
    Canberra Australian Government Publishing
    Service.
  • Serway, R.A., Jewett, J.W. (2004). Physics for
    Scientists and Engineers with Modern Physics.
    (6th ed.). Belmont Brooks/Cole.
  • Swan, G.I. (2003). Regular motion. In G.Crisp,
    D.Thiele, I.Scholten, S.Barker and J.Baron
    (Eds.), Interact, Integrate, Impact Proceedings
    of the 20th Annual Conference of the Australasian
    Society for Computers in Learning in Tertiary
    Education. Adelaide, 7-10 December 2003.
  • Thelwall, M. (2000). Computer-based assessment a
    versatile educational tool. Computers
    Education, 34, 37-49.
  • Thomson Learning (2004). WebCT server login page.
    Thomson Learning. Available http//thomson4.webc
    t.com/public/swanserwaycowan/index.html July
    2004
  • Additional Peer instruction by Mazur. Tutorials
    in introductory physics by McDermott Shafer.

34
SCP1112 Students usage and feedback
  • SCP1112 Waves and Electricity Semester 2, 2001
  • Usage (5 full 1 half quiz)
  • Of 73 students who attempted the SCP1112 exam
  • 32 (44) attempted 3 or more full quizzes
  • 25 (34) attempted at 1 or 2 full quizzes
  • 16 (22) did not attempt any quiz (for these
    students unit failure rate gt 4 times that of
    other students)
  • Feedback
  • Large majority extremely positive (mid-sem and
    end-sem)
  • easy to use, relevant, improved understanding etc
  • feedback detail about right (a few students
    wanted more detail!)
  • More divided on
  • future compulsory assessment No. of quizzes
  • Helping to work more consistently over the
    semester
About PowerShow.com