A Sample of Best Practices in University Lower Division Science Education

1
A Sample of Best Practices in University Lower
Division Science Education

• CSE Brownbag
• 18 October
• Dan Bernstein, University of Kansas
• djb_at_ku.edu

2
Overview of session

• Uses of class time
• Capturing out of class time
• Alternative course designs
• Discussion of implementation
• costs and benefits
• cultural context
• recommendations
• Disclaimer

3
Using class time

• Pause for problems / interaction
• Mazur is the poster child
• Survey of KU clicker users
• Attendance and pop quizzes
• Check for understanding 3rd
• no plans for how to proceed
• Pollock argued that main benefit is collaboration
  during breaks
• Stop learning and listen to me
• Could be routine feature of classes

4
Group work tricky in large classes

• Managing groups requires planning
• One KU professor takes on Budig 120
• Tim Shaftel (Business) inserts group days
• Random assignment to fours w/ warmup
• Works a problem on the big screen
• Breaks out for comments and suggestions
• Roams the room for solutions
• Consider the video

5
Tutorials - U of Washington PEG

• Lillian McDermott and colleagues
• Crafted generative problem tutorials
• Intended to replace lecture/problem sessions TA
  doing the problem
• Active engagement in figuring conceptual features
  of physics
• Consider these examples

6
Collisions in 2-D
7
Progressive questions per set upFocus on
explanations
8
More challenging particulars
9
Magnetism -- with magnets
10
Progressively complex
11
Steve Pollock, PhysicsUniversity of Colorado

• Replaced typical discussion for Intro to Physics
  with Tutorials.
• Result very high learning gains, by national
  standards. (The final score matches what our
  junior physics majors get on this hard exam!)

12
Colorado -- BEMA pretest
BEMA Brief EM Assessment, validated
research-based survey of Conceptual elements of
EM. Blue data above is F04 (N319) Pretest ave
26
13
BEMA post -- Comparable to Grad Students
F04 (N319) 26 -gt 59, S05
(N232) 27 -gt 59 Posttest results yield an
impressive replication for two semesters High by
natl standards (typical trad courses, post score
30-40 !)
14
Pre/post FMCE (Sp04)
15
This is their research area
16
Inquiry laboratories

• Related to the tutorials -- constructivist model
  of understanding
• Taken to full hands on laboratory
• Joe Heppert, Jim Ellis, Jan Robinson
• Engage students in process
• Embedded, inductive, open-ended

17
High End -- Studio Physics

• Hands on discovery in place of lecture
• Reorganize even very large classes
• Two hour blocks of time
• Measure conventional and conceptual skills
• Taking inquiry lab, constructivist model to the
  whole experience
• Robert Beichner, NC State, one example

18
Teaching space very different
19
Conventional exam questions
20
MC items - Studio v. three lecturers
21
Studio gt comparable problem sets
22
Failure ratio Lecture/Studio
23
FCI gain - Highly replicable
24
Semester gain by class rank
25
Outside of Class Time

• Some use technology
• Center for Academic Transformation
• Others based on peers
• Community building
• Meta cognitive coaching

26
Carol Twigg invested Pew funds

• Re-gifted the money for course redesign
• Focused on technology as tool
• Emphasized saving money through efficient
  non-human or lower cost human delivery
• Committed to evaluation by learning and
  completion rates
• Increased success and/or difficulty of course
• Tracked learning downstream in curriculum
• Decreased rates of D, F, and Withdrawal

27
Carnegie Mellon -- Statistics

• Created StatTutor program
• Open-ended intelligent tutoring software
• Gives feedback on individual paths
• Focuses on decision making en route
• Aimed for high levels of skill not previously
  attainable
• 22 increase in scores
• Critical skill is selecting appropriate
  statistics to use

28
High rates of success

• Replicated in two course offerings (Ngt400)
• Selection error rates dropped from 9 to lt1
• Two skills not attempted before reached 70
  correct

29
Ohio State University - Statistics

• Buffet of options for gt3000 students / year
• Discovery laboratories, small groups, small
  lectures, training modules, video reviews
• All take common examinations
• Learning was greater than comparable daytime
  lecture based course
• Greatly enhance retention of students
• Fewer Ws, Fs, and Is
• Modular credit (1-5), reducing full retakes

30
Tutorial out performed day class

• Large class equaled smaller night class
• Fewest failures
• Maintained large enrollment

31
Penn State - Statistics

• Reduced lectures from 3 to 1 per week
• Replaced with computer lab time
• Computer mediated workshops
• Extended practice in computerized testing
• Lecture Exam pre-post was 50 gt 60
• Redesign Exam pre-post was 50 gt 68
• Selection of correct tool 78 gt 87
• DFW rate 12 gt 10
• 2200 students per year

32
University of Iowa - Chemistry

• 1300 students / year
• Pressure from Engineering and Pharmacy
• Fewer lectures, modular content, active
  participation, computer simulations
• Inquiry based laboratories
• No difference on common exam items
• Am Chem Soc exams 58 gt 65, 52 gt 61
• DFW 24-30 gt 13
• DF 16 gt 9 W 9 gt 4

33
U Mass - General Biology

• 700 students / semester
• Lectures 3 gt 2, add review session
• Inquiry lab already in place
• Interactive class technology, online quizzes
• Peer tutoring and supplemental instruction
• Use ClassTalk network for students
• Exams 61 gt 73 correct
• Questions 23 gt 67 required reasoning
• DF 37 gt 32

34
Peer led workshop groups

• Northwestern University Biology course
• Based on legendary work of Uri Treisman
• Peers prepared as facilitators at UT Austin
• Led group problem solving 2 hr / week
• Majority students outperformed controls
• Steady improvement across exams
• Minority students outperformed controls
• Improvement noted on last exam
• Historic controls show decline over course
• 3rd exam exceeds majority controls

35
Wendi Born _at_ CTE 7 November
36
Supplemental Instruction

• Peer led sessions with trained facilitators
• Part content and part meta cognition
• Study skills
• Learning about learning
• Designed at UMKC by Deanna Martin
• Address high failure rate by minorities in
  professional programs
• Identifies at risk courses, not at risk students
• Lani Guinier on the canary

37
Key Characteristics

• All students invited, not targeting weakest
• Always with faculty cooperation
• Sessions begin right away
• Not associated with having problems
• Minority students
• SI participants have 2.02 GPA in courses
• Non SI participants have 1.55 in same courses
• DFW rate
• Non SI at 43, SI at 36

38
Huge international following
39
Nebraska - Intro to Chemistry
Non SI had more than double the failure
rate 83 passed with SI, 57 without
40
Universal aid, like Studio Physics
41
Universal Design for Success

• Presume students can learn
• Discount need to sort or differentiate
• Maximize overall course performance

42
Benjamin Bloom promoted mastery

• Based on practice and feedback
• Divide course into many smaller units
• Take examinations and get results
• Require taking exam again until high score
• IFF 95 correct gt study next unit

43
Fred Keller promoted mastery

• IFF 95 correct gt study next unit
• Course grade is number of units passed
• No penalty for repeating and learning
• All who pass 12 units gt grade of A
• Do A work on 10 units gt grade of C

44
Also taught conventional lecture

• Lecture class
• Same exam questions
• Two attempts per test
• In class feedback
• No contingency

• Mastery Class
• 95 contingency
• No penalty for learning
• Immediate feedback
• No lecture required

45
Total amount learned

• Nearly twice as many at the high end of learning
• Virtually no one failed to learn
• Maximized learning for many students

46
Showed in amount and accuracy

• Many more questions answered
• Took 12 15-item tests
• Lecture was three tests of 20 items each
• Certify more learning
• Overall percent correct also higher

47
No magic -- students studied better

• They put in more time to their learning
• There was more work asked for by the course
• Note that they report doing the reading more
• Preparation for class is key issue (later also)
• Guideline in US -- 2 hours outside for every 1
  hour in class

48
Major meta analysis of 100 studies

• Kulik, Kulik, Bangert-Drowns
• Consistently more learning
• More time on task
• Greater retention over time
• Lower completion rates when used without
  deadlines and incentives

49
Placement and Prerequisites

• Variation on the same theme
• Languages require competence
• Tracking skill downstream in the curriculum
• Using mastery criteria for foundation courses
• Requires some coordination within and between
  units
• Could benefit from tutorials and SI

50
Marginal gains not clear

• Are these effects additive?
• Maybe they all help the same students in the same
  way

51
Ernest Boyer

• The work of the scholar
• remains incomplete
• until it is understood and used
• by others.

52
Challenges on teaching science

• Do we really want success? Grade inflation?
• How do we handle the coverage/depth issue?
• What about the resources?
• Space, funds, faculty time
• Students should also be responsible
• Are these technologies transferable/robust?
• What about bureaucratic hurdles?
• Remedial courses/tutorials, Undergrad TAs
• Semester based credit and tuition

53
Your Insights?
http//www.cte.ku.edu
54
Three functions of grading

• Certification of learning
• Motivation for learning
• Differentiation among learners
• Each has a legitimate purpose
• No one system does all well

55
Variability in conventional course

• Students learn at different rates
• When course ends, fast learners get best results
• Very good at identifying fast learners
  (differentiate)
• Less good at motivating for more work

56
Variability in a mastery course

• Everyone learns until material is mastered
• Reward is for work subjective probability of
  success
• Very good at certification of learning
• Provides incentive for studying, no penalty if
  slow

57
When is mastery the right approach?

• Foundation courses -- want knowledge
• Programs in which rate of learning is not a
  criterion for success
• Situations in which performance will not be timed
• Professions in which high skill is expected
• Why tolerate ineffective teaching?
• If we dont care or think it can not be learned
  by all

58
How much can a student learn?

• Boundaries are time, effort, and capacity
• Time and capacity are fixed
• Your leverage into learning is effort
• Organize a system that allows extra work
• Honor that work when it succeeds
• May lose some identification of capacity
• Greatly expand the amount of learning

59
Scholarship Assessed (1997)

• All forms of scholarship include
• Clear goals
• Adequate preparation
• Appropriate methods
• Significant results
• Reflective critique
• Effective presentation
• Glassick, Huber, Maeroff

60
Communities of inquiry on learning

• Being very public with teaching in same sense
  of a center for research
• Faculty need another lens to complement student
  voice, converging measures
• Have an external community that values this work
• Stresses our existing skills at intellectual
  inquiry as basis of exploration

61
Building a community to discuss ideas about
teaching

• Workshops and seminars for faculty members
• Straightforward process of peer interaction
• Exchange ideas around three themes
• Provide resources for exploration
• Written product of thinking and planning

62
Collaborative Working Seminars

• Discussion of shared issues with colleagues
• Time for reading and searching
• Targets for writing and sharing
• Intentional plan is the product