Telecommunications%20in%20Education:%20Examining%20Distributed%20Teaching%20and%20Learning%20Environments%20(EDER%20677%20L.91)%20%20%20%20teaching%20and%20Learning%20Environments:%20Guides%20for%20Critical%20Reviews%20and%20Evaluation%20%20for%20the%20Asynchronous%20Class%20and%20for%20the%20Web%20Site%20on

1
Telecommunications in Education Examining
Distributed Teaching and Learning Environments
(EDER 677 L.91)teaching and Learning
Environments Guides for Critical Reviews
andEvaluationfor the Asynchronous Class
andfor the Web Siteon November 7from Calgary
2
Housekeeping
Housekeeping
Housekeeping
Housekeeping

• Some pointers
• If you would like to discuss your online project
  with me, I will be sure to be available at a time
  when you need
• If you would like to discuss your DL portfolio, I
  would be delighted to discuss that with you too
  just let me know

3
Presentation Overview

• In this lesson, we will
• Explore the issues around designing, developing
  and utilizing web based assessment techniques for
  online learning environments or their parts -
  from the teacher or designer perspective.
• Discuss the importance of formative and summative
  program/course evaluation in the online learning
  context - from the teacher or designer
  perspective.

4
Assessment and Evaluation - Definitions

• Assessment - determine changes in learner
  performance
• Evaluation - make judgments and decisions about
  learners
• Program or course evaluation - appraise the
  success of your program or course

5
Assessment and Evaluation - Self Activity
(something to ponder as you create evaluations
for your ownonline learning project)

• List examples of web based assessment activities
  that you prefer (see the resource section of this
  lesson)
• What types of student learning do these
  activities support (e.g. Blooms Taxonomy
  knowledge, comprehension, application, analysis,
  synthesis, evaluation)?
• Identify issues surrounding the use of web based
  assessment activities that you feel strongly
  about, and address those issues in your project.

6
Triangle of Assessment

• The multidimensional approach to assessment

Knowledge Transfer and Performance
Knowledge Constructs and Reflection
Concrete Knowledge Acquisition
Blooms Taxonomyhttp//www.coun.uvic.ca/learn/pro
gram/hndouts/bloom.html
7
Concrete Knowledge Acquisition

• Concept
• "Factual Foundation Knowledge" - provides the
  ground floor in a learning hierarchy. You can
  evaluate the things that must be known - facts.
• The working knowledge or language of a subject
  discipline can be considered as a foundational
  knowledge to succeeding in a learning event,
  online or anywhere.

8
Concrete Knowledge Acquisition

• Working Knowledge Examples
• Calculus
• The student can recognize and count numbers
• The student can add, subtract, multiply and
  divide
• Literary criticism and creative writing
• The student can apply the alphabet and the basic
  rules of grammar

9
Concrete Knowledge Acquisition

• Assessment Techniques
• Measuring student mastery of the factual
  foundation knowledge
• Some in instructional methodologies where you use
  specific learning evaluation techniques (Great
  source book Gronlund The Assessment of Student
  Learning 7th Ed.) We take this up in EDER 673 in
  detail. Drill and practice is one assessment
  technique\ with components designed to check the
  level of mastery of basic knowledge.
• Drill and Practice
• multiple choice
• true and false
• matching
• ordering
• short answer

10
Concrete Knowledge Acquisition

• More Assessment Ideas
• Give a Preview quiz on preparatory textbook,
  article or online readings
• Give a Review quiz on lecture content
• Give a Collaborative quiz online discussion
  about the questions and answers on a quiz
• Students can develop their own quiz questions
  learning how to design tests
• Online, other students can respond and evaluate
  the documents.
• For stronger students in the class the writing
  of mid-term and final exam questions can also
  result in credit for this online assignment.
• Online quizzes - Integrity test questions first
  or last question Did you do this quiz on your
  own? (on site supervised testing is a simple and
  effective option).

11
Concrete Knowledge Acquisition

• Web Based Tools for Evaluation See the Resource
  Section in the Lesson Plan for November 7th for
  more of these
• Hot Potatoes
• http//web.uvic.ca/hrd/halfbaked/
• Quiz Lab
• http//www.funbrain.com/quiz/index.html
• TeAch-nology.com's Crossword Puzzle Maker
• http//www.teach-nology.com/web_tools/crossword/
• Discovery School's PuzzleMaker (Word Searches)
• http//www.puzzlemaker.com/

12
Knowledge Constructs and Reflection

• The Concept of Knowledge Constructs
• This is the process of students constructing
  and/or discovering their own knowledge. ( Self
  evaluation with cognitive (model) direction).
• Internalization and integration of new
  information with previously held knowledge (in
  the good old bad old days, we called this
  synthesis -)

13
Knowledge Constructs and Reflection

• The Concept of Reflection
• Students reflecting on the construction of their
  knowledge as part of the evaluation process
• Zone of proximal development can be created
  online too the instructor watches and guides
  students to evaluate each other after they
  evaluate themselves.
• Students are comparing thoughts and ideas with
  others.

14
Knowledge Constructs and Reflection

• Assessment Techniques
• Structured journals (e.g. responding to specific
  questions)
• Open-ended reflection journals (e.g. responding
  to broad or open questions)
• Collaborative discussions and summaries

15
Knowledge Constructs and Reflection

• Assessment Techniques for Online Discussions
• See our home page links

16
Knowledge Constructs and Reflection

• Assessment Examples
• Example One - Class Discussion Scoring Guide
• http//www.tageducation.org/webct/webct/level1/6di
  scussrubric.htm
• a rubric for a 100-point project (60 for
  responses to weekly readings, 40 for responses
  to classmates postings)
• Example Two - Grading Online Discussions
• http//www.oznet.ksu.edu/dp_iet-edtech/TechforTeac
  hing/grading.htm
• various models based on support/clarity/helpfulnes
  s, an A-F scale, and groups grading each other

17
Knowledge Constructs and Reflection

• Assessment Examples
• Example Three - Evaluation and Grading Criteria
  for WebBoard Discussion Assignments
• http//www.ivcc.edu/eng1001/handout_evaluating_dis
  cussions.htm
• an outline of four evaluation criteria for
  discussion assignments
• Example Four - Discussion Guidelines and Grading
  Standards
• http//www.stedwards.edu/ursery/discuss.htm
• discussion grade guidelines on an A to F scale,
  as well as a 10-point guideline for participation
  

18
Knowledge Constructs and Reflection

• Recall the Modes of Web Based Computer Mediated
  Communication Tools
• Asynchronous
• E-mail
• File transfers
• Threaded discussion forums
• Synchronous
• Virtual chats
• Shared whiteboards, URLs, and applications
• MUDs(multi-user dimensions) and MOOs (multi-user
  dimension - object oriented)

19
Knowledge Constructs and Reflection

• Web Based Resources that help you evaluate your
  online learning environs, considering the mode of
  delivery / interaction chosen by the designer
• Using Asynchronous Conferencing to Promote
  Critical Thinking
• http//crlt.indiana.edu/publications/crlt99-8.pdf
• Instructional Design Principles for Teaching in
  Computer Conferencing Environments
• http//www.cvm.tamu.edu/wklemm/instruct.html
• Shaping Cyberspace Into Human Space
• http//css.sfu.ca/update/vol6/6.3-harasim.main.htm
  l
• Handbook for Instructors on the use of Electronic
  Discussions
• http//www.osu.edu/education/ftad/Publications/ele
  cdisc/pages/index.htm
• Check for Plagiarismhttp//www.turnitin.com

20
Knowledge Transfer Performance

• Concept
• Performance-based assessment - students
  demonstrate the (learned) behavior to be measured
  (by the evaluator).
• Authentic assessment - students demonstrate the
  acquisition and mastery of a 'real world' skill
  or concept.
• Learners applying knowledge and skills rather
  than simply remembering facts

21
Knowledge Transfer Performance

• Web Based Resources Hotlinks to Assessment
  tools
• Performance Assessment
• Performance Assessment Educational Issues Series
• Strategic Questions - What to Consider When
  Planning for Electronic Portfolios
• Electronic Teaching Portfolios
• Electronic Portfolio Checklist
• Portfolio Assessment
• Student Portfolios Classroom Uses
• Student Portfolios Administrative Uses
• Alternative Assessment and Technologyhttp//www.e
  d.gov/databases/ERIC_Digests/ed365312.html
• Assessment Terminology A Glossary of Useful
  Termshttp//www.newhorizons.org/assmtterms.html
• Collected Assessment Resources Center for
  Research on Evaluation, Standards, and Student
  Testinghttp//www.cse.ucla.edu/

22
Online Program/Course Evaluation - Remember this
Modelfrom earlier in the course? Now we look at
the Evaluate loop(More detail will come in
EDER 673)
23
Online Program/Course Evaluation - Definitions

• Formative Evaluation
• The ongoing analysis of your program/course
  design during development and implementation.
  Much more on this in EDER 673 and EDER 675
• Summative Evaluation
• Is conducted to determine whether your course or
  program is worthy of adoption or continuation
• leads to a go/no go decision on the program.

24
Online Program/Course Evaluation - Formative

• Some of the on-going process that designers use
  at each stage of the course development process
• They have colleagues and students constantly
  reviewing the development and implementation of
  your web course site internal review
• They develop an evaluation checklist or form
• They ensure that each module is piloted
• They make the necessary revisions based on
  informal and formal feedback mechanisms

25
Online Program/Course Evaluation (Summative)

• Judging the worth or merit of a program/course
• This evaluation is conducted after a
  program/course/design has been in place for a
  complete cycle usually an external review to
  the institutional course distributor
• Key Questions used in this type of program
  evaluation
• Effectiveness Do students learn the basic
  learning objectives or exceed them?
• Appeal Does the program/course/design engage
  learners and students do they like it?
  (Affective)
• Efficiency Is the program/course cost
  effective?

26
Program/Course Evaluation - Resources

• Article
• Shrock, S.A. and Geis, G.L. (1999). Evaluation.
  In H.D. Stolovitch and E.J. Keeps (Eds.) Handbook
  of human performance technology (2nd ed.) (
  Chapter 10 Evaluation, pp. 185-209). San
  Francisco, Jossey-Bass Pfeiffer.
• Web sites
• See the resources section in the lesson plan for
  hundreds of resources.

27
For next session

• Contribute to this weeks discussion forum.
• Check out the Web Site Home Page for further
  instructions (core readings).
• Your online project production should have
  started by now - your professor will review
  anything that you like ahead of time
• Your Personal DL portfolio web site should be
  under construction now.

Adieu from Calgary..