TCSG: -Q-S Curriculum Initiative

1
TCSG -Q-S Curriculum
Initiative

• - Outcomes, Levels of Learning and Assessment -
• Curriculum Database Process
• Dr. Tanya Gorman, VPAA
• DeKalb Technical College

2
Q-T Curriculum Initiative

• I Domains / Levels of Learning
• II Learning Outcomes / Assessment
• III. Curriculum Database Tool

3
SACS

• Core Requirement 2.7.2
• The Institution offers degree programs that
  embody a coherent course of study that is
  compatible with its stated purpose and is based
  upon fields of study appropriate to higher
  education

4
SACS

• Core Requirement 2.7.3
• In each undergraduate degree program, the
  Institution requires the successful completion of
  a general education component at the collegiate
  level that (1) is a substantial component of each
  undergraduate degree (2) ensures breadth of
  knowledge and (3) is based on a coherent
  rationale

5
SACS

• Comprehensive Standard 3.3.1
• The institution identifies expected outcomes,
  assesses the extent to which it achieves these
  outcomes, and provides evidence of improvement
  based on analysis of the results in each of the
  following areas (Institutional Effectiveness)
• 3.3.1.1 educational programs, to include student
  learning outcomes

6
SACS

• Comprehensive Standard 3.5.1
• The institution identifies college-level general
  education competencies and the extent to which
  graduates have attained them.

7
SACS

• Federal Requirement 4.2
• The Institutions curriculum is directly related
  and appropriate to the purpose and goals of the
  institution and the diplomas, certificates, or
  degrees awarded.

8
Student Learning Outcomes

• Bad Practices
• Doing assessment because ACCREDITORS say we have
  to
• Confusing course assessment with program
  assessment
• Insufficient concern about matching assessment
  tools to expected outcomes

9
Student Learning Outcomes

• What constitutes as evidence of student learning?
• Evidence can take many forms but must involve a
  direct examination of student levels of
  attainment
• Exams, capstone experiences, licensure exams,
  demonstrations, portfolios, assignments, etc.
• Evidence such as surveys, self-reports, focus
  groups, interviews, student satisfaction surveys
  are useful, but represent indirect evidence and
  are not sufficient for documentation of learning
  outcomes

10
Student Learning Outcomes Building Blocks

• Student learning outcomes defined
• A particular level of knowledge, skills and
  abilities (cognitive, psychomotor, affective)
  that a student has attained at the end (or as a
  result) of his/her engagement in a particular set
  of collegiate experiences

11
Student Learning Outcomes

• The definition of SLOs begins with the end in
  mind
• What does student success look like?
• How would I know it when I see it?
• What would students have to do, say, perform or
  behave like for me to determine they had mastered
  the competencies?

12
Student Learning Outcomes

• Curriculum, and consequently, SLOs must,
  therefore, derive from the Big Picture the
  end in mind (Program Outcomes)
• Curriculum cannot be in isolation jigsaw puzzle
  analogy (corners and borders)
• Be clear about outcomes you want to promote if
  you dont know where youre going youll never
  know if you got there

13
Student Learning Outcomes Construct Model

• Program when Joe finishes, what should he be
  able to do? (Generally 3-5)
• Course what Competencies should Joe accomplish
  and demonstrate to achieve and answer the first
  question? (Generally 5-10)
• Unit what learning outcomes, experiences and
  content need to be embedded to ensure
  accomplishment of SLOs and support Course
  Competencies?
• Unit what methods of assessment will
  demonstrate that Joe has learned what he needs to
  learn?

14
Domains Levels

• Basic understanding of Educational Methodologies
  (building blocks) is important in defining SLOs
• Cant build a house without a foundation
• Definition of learning desired ? choice of
  assessment method

15
Domains Levels

• Answer the Question Students will
• Know, Think, Do
• Cognitive, Affective, Psychomotor (Domains)
• How do you know? How will you measure? What
  will it look like?
• What instruments and benchmarks will you use?

16
Domains Levels

• Assessment methods and tools must align with
  competency / learning outcomes
• Assessment of outcomes must provide for
  comparative analysis across courses taught by
  multiple instructors
• Course outcomes / General Education Core
  competencies and SLOs must show logical
  progression in courses and programs of higher
  level outcomes and objectives

17
Domains Levels

• Learning Outcomes are statements of performance
  expectations
• Cognitive
• Affective
• Psychomotor
• Program / course completers must demonstrate
  performance in all three

18
Levels of Learning

• Examples of dissonance between knowing and
  doing and valuing
• Smoking
• Overeating
• Exercise
• Unless knowledge is combined with a value system,
  behavior (learning) will not change

19
Domains Levels

• Courses within a program of study must
• Be part of a package of
• Interrelated
• Coherent
• Experiences that result in student learning and
• lead to a functional and employable graduate
• Faculty must structure experiences and define
  learning outcomes toward which ALL instruction,
  learning and assessment relate

20
THE PLAN

• One Possible Reason Why Things Arent Going
  According to the PlanIs That There Never Was
• A Plan!

21
Construct

• Example
• Program Learning Outcome Provide competent and
  safe care in a healthcare environment
• Course Competency Demonstrate knowledge of the
  care of patients in renal failure
• Student Learning Outcome
• Describe S/S of renal failure (cognitive)
• Demonstrate proper care of dialysis catheter
  (psychomotor)
• Protect patients privacy (affective)

22
Construct

• Another Example
• Program Learning Outcome Function safely and
  competently in an automotive repair business
• Course Competency Demonstrate ability to
  diagnose, repair and replace transmissions
• Student Learning Outcome
• Describe parts of a transmission (cognitive)
• Disassemble a transmission (psychomotor)
• Identify problems and defective parts of a
  transmission (cognitive)
• Demonstrate respect for customers vehicle
  (affective)

23
Construct

• Another Example General Studies Core
• Core Ability Communicate effectively in writing
  using grammatically correct and appropriate
  language
• Course Competency Demonstrate ability to write
  an organized and defensible Theme Paper
• Student Learning Outcome
• Prepare a topical outline (cognitive)
• Produce a Theme Paper using MS Word (psychomotor)
• Reflect sensitivity to opposing values and views
  (affective)

24
Construct

• Curriculum Mantra
• Learning outcomes drive content
• Assessment depends on defined learning outcomes
• Program learning outcomes are broad
• Course competencies are focused
• Student learning outcomes support competencies
• Unit content is structured based on course
  learning outcomes
• Assessment methods must reflect learning outcomes

25
Nutshells

• Program Outcomes ?
• Course Competencies ?
• Student learning Outcomes ?
• Selection of Content ?
• Assessment / Evaluation Methods

26
Domains Levels

• Good Learning Outcomes
• Identify a specific student behavior
• Specify ONE learning outcome
• Why is this important?
• Are relevant (meaningful)
• Are measurable / assessable

27
Domains Levels

• Since learning occurs in all three Domains
  cognitive, affective and psychomotor learning
  outcomes, likewise, must be written for each
  domain and for varying levels within each course
  and program
• There must be a trackable, observable and
  documentable progression of learning evidenced in
  courses and programs
• Likewise, there must be an observable progression
  of assessment methods for higher levels within
  each domain

28
Levels of Learning

• Domains Learning (behavior change) occurs in
  three broad domains
• Cognitive, Affective, Psychomotor (CAP)
• Within each domain are levels of learning that
  drive assessment

29
Levels of Learning

• Domain Cognitive
• Knowledge recalling facts
• Comprehension seeing relationships
• Application using information in new ways
• Analysis breaking information into parts
• Synthesis forming new information
• Evaluation judging value

30
Levels of Learning

• Domain Psychomotor
• Perception awareness of need
• Set mental, physical emotional readiness to
  perform
• Guided Response skill performed by imitation,
  trial and error
• Mechanism habitual, skilled performance
• Complex Response smooth, efficient, automatic
• Origination adaptation to conditions

31
Levels of Learning

• Domain Affective
• Receiving willingness to hear
• Responding willingness to react
• Valuing demonstrating commitment
• Organization establishing pervasive values
• Characterization demonstrating characteristics
  of a unique individual

32
Levels of Learning

• Breakdown of the levels using overriding
  framework of learning about computers

33
Levels of Learning

• Cognitive
• Knowledge define CPU list software
• Application knowledge applied to selection
• Evaluation evaluate effectiveness / deficiencies

34
Levels of Learning

• Psychomotor
• Perception keyboarding / typing
• Guided Response practice, errors, correction
• Complex Response know where keys are
• Origination adjustments w/ broken finger

35
Levels of Learning

• Affective
• Receiving initial reaction hesitation, fear
• Valuing what transpired to get past initial
• reaction
• Characterization purchase a personal PC

36
Levels of Learning

• MAJOR POINTS
• Students must not only demonstrate learning in
  all domains but must demonstrate progression
  within the domains
• You would never want a student to be stuck in
  lower levels and never progress

37
Levels of Learning

• Reflection How to structure and promote higher
  levels of learning for students?
• Answer By ensuring a planned and structured
  learning environment with identified outcomes
  that address varying levels and demonstrations of
  learning

38
Learning Process

• Lets play in the Domains vis-à-vis the writing
  of Learning Objectives
• Cognitive Knowing
• Knowledge recalling facts
• Comprehension seeing relationships
• Application using knowledge
• Analysis breaking knowledge into parts
• Synthesis forming knowledge in new ways
• Evaluation judging knowledges value and
  appropriateness

39
Learning ProcessMeasurable Verbs Knowledge

• Cite
• Count
• Define
• Identify
• Choose
• Match

• Recite
• Repeat
• Tell
• Write
• Select
• Explain

40
Learning ProcessMeasurable Verbs Comprehension

• Associate
• Compare
• Describe
• Discuss
• Contrast
• Outline

• Predict
• Report
• Restate
• Translate
• Summarize

41
Learning ProcessMeasurable Verbs Application

• Apply
• Classify
• Determine
• Illustrate
• Choose

• Interpret
• Restructure
• Solve
• Use
• Develop

42
Learning ProcessMeasurable Verbs Analysis

• Analyze
• Appraise
• Differentiate
• Distinguish
• Categorize

• Examine
• Inspect
• Question
• Summarize
• Infer

43
Learning ProcessMeasurable Verbs Synthesis

• Assemble
• Compose
• Create
• Formulate
• Design

• Integrate
• Organize
• Propose
• Synthesize
• Adapt

44
Learning ProcessMeasurable Verbs Evaluation

• Appraise
• Assess
• Critique
• Evaluate
• Conclude
• Criticize

• Judge
• Defend
• Revise
• Validate
• Prioritize
• Disprove

45
Learning ProcessExample Cognitive

• Knowledge List the components of a good outcome
• Comprehension Discuss the 3 Domains
• Application Determine which of 3 outcomes is
  appropriately stated
• Analysis Classify 3 outcomes by domain level
• Synthesis Design outcomes in all domains for a
  given learning scenario
• Evaluation Revise these outcomes for alternate
  scenario

46
Learning Process

• Lets play in the Domains vis-à-vis the writing
  of Learning Objectives
• Psychomotor Skills
• Perception awareness
• Set preparation to perform
• Guided Response demonstration trial/error
• Mechanism habitual skill development
• Complex Response smooth, efficient skill demo
• Origination adaptation of skill to conditions

47
Learning ProcessMeasurable Verbs Perception /
Set

• Perception
• Listen
• Observe
• Ask

• Set
• Attend
• Organize
• Establish

48
Learning Process Measurable VerbsGuided
Response / Mechanism

• Guided Response
• Perform
• Demonstrate
• Establish

• Mechanism
• Apply
• Use
• Implement

49
Learning ProcessMeasurable Verbs Complex
Response / Origination

• Complex Response
• Integrate
• Activate
• Construct

• Origination
• Adapt
• Collaborate
• Prioritize

50
Learning ProcessExample Psychomotor

• Perception Ask re appropriate time to check BP
• Set Organize materials to check BP
• Guided Response Perform check with assistance of
  instructor
• Mechanism Implement routine to check 10 patients
  without assistance
• Complex Response Integrate BP checks w/ total
  assessment for a unit
• Origination Adapt BP check for pt. with
  bilateral arm fractures

51
Learning Process

• Lets play in the Domains vis-à-vis the writing
  of Learning Objectives
• Affective Valuing
• Receiving willingness to hear
• Responding willingness to react
• Valuing commitment
• Organization establish pervasive values
• Characterization create unique person

52
Learning ProcessMeasurable Verbs Receiving

• Listen
• Perform
• Comply

• Assist
• Care
• Maintain

53
Learning ProcessMeasurable Verbs Responding

• Adapt
• Tolerate
• Accept

• Attend
• Interact
• Display

54
Learning ProcessMeasurable Verbs Valuing

• Cooperate
• Anticipate
• Consider

• Seek
• Influence
• Follow

55
Learning ProcessMeasurable Verbs Organization

• Respect
• Resolve
• Encourage

• Offer
• Establish
• Relate

56
Learning ProcessMeasurable Verbs
Characterization

• Initiate
• Educate
• Appreciate

• Promote
• Share
• Motivate

57
Learning ProcessExample Affective

• Receiving Listen during class
• Responding Display interest in using new
  material
• Valuing Follow instruction re new project
• Organization Offer evaluation of the project
• Characterization Promote behaviors consistent
  with new learning

58
Key Questions

• Learning Outcomes
• What is actual, highest level learning expected ?
• What will it look like when the student
  performs it?
• What are essential elements of performance
  necessary to demonstrate learning?
• What are the most effective ways for students to
  master these learning outcomes?
• How will I know the student has achieved?
• How can I assess it effectively?
• How can I document learning outcomes?

59
Assessing Learning Outcomes

• The assessment of learning outcomes requires a
  mindset change movement from teaching to
  learning and learner-centered assessment.

60
Assessment

• Assessment is the ongoing process of
• Establishing clear, measurable expected outcomes
  of student learning
• Learning Outcomes statements
• Ensuring that students have opportunity to
  achieve the outcomes
• Learning Opportunities - methodologies

61
Assessing Learning Outcomes

• Differentiation between levels of Learning
  Outcomes and Assessments
• Institution
• Program
• Course
• Student

62
Assessing Learning Outcomes

• What is assessment of Student Learning Outcomes?
• Formalized processes to collect and analyze
  learning outcome data that will be evaluated to
  derive meaning and purposeful use of the data for
  learning enhancement.
• Assessment is meaningless unless evaluation
  occurs (BP example)

63
Assessing Learning Outcomes

• Student learning outcomes defined
• A particular level of knowledge, skills and
  abilities (cognitive, psychomotor, affective)
  that a student has attained at the end (or as a
  result) of his/her engagement in a particular set
  of collegiate experiences

64
Assessing Learning Outcomes

• This, then, begs the questions
• How to make assessment /evaluation meaningful?
• What outcome data should be assessed and
  evaluated?
• OBanions 2 questions related to teaching and
  learning are
• Does it (whatever happens in the classroom)
  enhance learning?
• How do we know?

65
Assessing Learning Outcomes

• Purpose and Principles
• Purpose of Assessment (Cross Papers)
• Faculty acquire information to enhance teaching
• Students gain insight into effectiveness of their
  own learning processes
• Provide support for student learning
• Provide a means for assessing strengths/weaknesses
  
• Evaluate effectiveness of teaching/learning
  strategies
• Show evidence of progress or areas for
  improvement
• Provide support for formative approach to improve

66
Assessing Learning Outcomes

• Principles of Assessment (Cross Papers)
• Learner-centered (learning vs. teaching)
• Assess what matters common expectations
  (taxonomies)
• Assess for improvement vs. accountability
• Banta if we undertake assessment just for the
  purpose of demonstrating accountability, we waste
  our time. We must do assessment in a way that we
  are getting information that faculty can actually
  use to improve what theyre doing.
• Fear of use for evaluation teach to the test

67
Assessing Learning Outcomes

• Research validates that teaching for analytical,
  creative and practical thinking, as well as for
  memory, boosts achievement on assessment that
  measures achievement broadly across subject
  matter areas i.e. assessment focused on higher
  level processes positively effects achievement
  and learning outcomes in all areas of learning
  regardless of topic or particular assessment
  methodology.

68
Assessing Learning Outcomes

• Foundations of Assessment
• Blooms Taxonomy of Learning
• Backward Design
• Teaching to Standards
• What / How of Assessment

69
Foundations of Assessment

• MAJOR POINTS Blooms Taxonomy
• Students must not only demonstrate learning in
  all domains but must demonstrate progression
  within the domains
• Assessment strategies must be compatible with the
  level of learning desired

70
Key Questions

• Why is it critical to
• Assess the stage of learning at which the student
  is functioning?
• Exhibit teaching strategies based on appropriate
  level?
• Implement assessment methods based on appropriate
  levels of learning and expectations of
  performance?
• Example
• Course Competency Write a Business Plan
• Assessment Method Multiple choice test

71
Foundations of Assessment

• Backward Design (Tomlinson)
• Imperative to define what students must know,
  think and do and understand as a result of
  instruction
• Teaching must match these expectations
• To teach for success, assessment must be aligned
  with knowledge, understanding and skill
  identified as learning outcomes assessment is an
  integral part of curriculum design not an
  afterthought.

72
Foundations of Assessment

• Teaching to Standards (Santoyo)
• Faculty must achieve consensus on the standards
  (outcomes) and expectations of student learning
  outcomes (domains, levels of learning, etc.)
• Faculty must achieve consensus on the assessment
  strategies and methods to be used to effectively
  elicit outcome data as evidence of student
  learning
• Examples of teaching to standards when no clear
  standards / assessment identified

73
Foundations of Assessment

• Standard (outcome) The student will understand
  and use percents in a variety of situations
• Variety of assessments used
• What is 50 of 20?
• What is 67 of 81
• Shawn got 7 correct answers out of 10 possible on
  a science test. What percentage of questions did
  he answer correctly?

74
Foundations of Assessment

• Assessments, contd
• J.J. Redick was on pace to set a college
  basketball record in career free throw
  percentage. Going into the NCAA tournament in
  2004, he had made 97 of 104 free throw attempts.
  What percentage of free throws had he made?
• J.J. Redick was on pace to set an NCAA record in
  career free throw percentage. Going into the NCAA
  tournament in 2004, he had made 97 of 104 free
  throw attempts. In the first tournament game,
  Redick missed his first five free throws. How
  far did his percentage drop from right before the
  tournament game to right after missing those free
  throws?

75
Foundations of Assessment

• Assessments contd
• J.J. Redick and Chris Paul were competing for the
  best free throw percentage. Redick made 94
  percent of his first 103 shots, whereas Paul made
  47 of 51 shots.
• (a) Which one had a better shooting percentage?
• (b) in the next game, Redick made only 2 of 10
  shots and Paul made 7 of 10 shots. What are
  their new overall shooting percentages? Who is
  the better shooter?
• (c) Jason argues that if J.J. and Chris each made
  their next 10 shots, their shooting percentages
  would go up the same amount. Is this true? Why
  or why not?
• (d) Describe in detail how you arrived at your
  answers.

76
Foundations of Assessment

• CENTRAL POINT re relationship b/w standards
  (outcomes) and assessment outcomes are
  meaningless until you define how you will assess
  them
• Without dialogue, faculty may and do define
  standards (outcomes) according to personal level
  of expectations which differ radically and,
  therefore, teach to varying levels of mastery
• Assessment without consensus ? poor outcomes

77
Forms of Assessment

• Use of the appropriate assessment tool requires
  consideration of the purpose of assessment
• Why are you assessing? (what purpose
  development feedback mastery, etc.)
• What do you want to assess?

78
Assessing Learning Outcomes

• Assessment Types (Earl and Tomlinson)
• Formative ongoing, periodic assessment of
  learning assessment for learning
• Summative end-point assessment assessment of
  learning
• Informative assessment as learning

79
Forms of Assessment

• What and How of Assessment
• Selection of the type of assessment depends on
  what faculty hope to learn from the assessment
  and what is being evaluated
• Quizzes and exams snapshots in time of recall
  ability and memory
• Portfolios longitudinal record of work and
  progress over time
• Essays ability to communicate in writing
  analyze and solve problems integrate ideas and
  information

80
Forms of Assessment

• Although educators may denigrate the validity,
  reliability and usefulness of tests/quizzes to
  effectively evaluate learning, it is, however,
  the most dominant form of assessment
• Tests / exams / quizzes can be successful when
  questions focus and reflect learning outcomes and
  assess learning in different domains (Bloom)

81
Forms of Assessment

• Example
• If students know they will be tested on facts,
  they will memorize
• If students know they will be assessed for
  comprehension and application, they will study
  and learn differently
• Creation of a table (Cross) that reveals
  percentage of time spent on concepts at various
  domain levels helps define the nature of
  assessment methodologies

82
Assessing Learning Outcomes

• Domain Questioning Strategy
  Concept 1 Concept 2 Concept 3
• Knowledge Recall information and facts
  25 10 15
• Comprehension Understand and interpret concepts
  25 35 10
• Application Employ concepts in new
  situations 15 15
• Analysis Classify information into components
  20 15
• Synthesis Produce structures /patterns create
  new meaning 30 30
• Evaluation Make judgments or valuations 30 30
  

83
Assessing Learning Outcomes

• Good Assessment Practices
• Start with clear statements
• Assess what is taught
• Collect more than one kind of evidence
• Make assignments and testing clear
• Ensure that assignments and testing relate to
  learning objectives (matrices)
• Score assignments fairly and consistently
  (rubrics)
• Evaluate the outcomes of assessment efforts and
  revise strategies

84
Assessing Learning Outcomes

• Assessment Data
• Quantitative numbers indicate learning or
  achievement (points, grades)
• Qualitative quality differences vs. amount
  judgment-based observation of performance,
  interviews, etc.

85
Assessing Learning Outcomes

• Assessment Score Interpretations
• Norm-referenced comparison of student
  performance to other people in a referenced
  population relative standing vs. specific
  mastery (grading on the curve)
• Criterion-referenced comparison to precise score
  that indicates mastery or competence (COMPASS,
  licensure scores, etc.)

86
Assessing Learning Outcomes

• Assessment Characteristics
• Reliability degree to which scores are
  consistent over repeated applications of the
  assessment
• Validity ability of the assessment to measure
  what it purports to measure measurement aligned
  with learning outcomes and free of bias

87
Assessing Learning Outcomes

• Formative Assessment
• Assessment for learning allows corrections to
  be made in the t/l process evidence used to
  adapt teaching to meet learning needs

88
Assessing Learning Outcomes

• Elements of Formative Assessment
• Specific learning goals / outcomes clearly
  communicated to students
• Specific criteria for meeting these goals /
  outcomes clearly communicated (e.g. rubric)
• Frequent interaction b/w faculty and students for
  clarification and reinforcement of expectations
• Timely feedback re assessment results
• Willingness of faculty to modify curriculum and
  t/l strategies based on results

89
Assessing Learning Outcomes

• To be effective and provide maximum learning
  value, a variety of assessment techniques should
  be applied numerous times during a course.
• Comprehensive assessment includes formative and
  summative strategies - with variety of
  methodologies. Students tend to excel at 1 or 2
  methods to the detriment of others who might
  excel at others. A variety ensures that all
  students have opportunity to demonstrate learning.

90
Assessing Learning Outcomes

• Steps of Assessment formative and/or summative
• ID learning outcome to be assessed
• Select assessment technique that will accurately
  and appropriately measure the outcome
• Apply the assessment technique
• Analyze the results of assessment
• Share results with students and provide feedback
• Respond to the results and effect changes

91
Assessing Learning Outcomes

• Tools of Assessment
• Tests quizzes, T/F, multiple choice, essays
• Performance demonstrations
• Portfolios
• Rubrics eliminates ambiguity re criteria to
  meet learning goals, expectations of performance
  and how evaluation will occur
• Qualitative and Quantitative rubrics acceptable

92
Assessing Learning Outcomes

• Elements of Rubrics
• Identified levels of mastery (excellent, good,
  5,4)
• Organizational groupings of skill sets teamwork,
  problem-solving, etc.
• Commentaries description of elements to be
  identified in the work
• Description of consequences

93
Rubric Qualitative
94
Rubric Qualitative/Quantitative
95
Rubric Quantitative
96
Assessing Learning Outcomes

• Benefits of Assessment Students
• Clear expectations upon which assessment is based
  helps students focus
• Assessment motivates students
• Feedback helps understand strengths and
  weaknesses (must be timely)

97
Assessing Learning Outcomes

• Benefits of Assessment Faculty
• Assessment encourages faculty discussion on
  issues what taught why standards expectations
• Assessment links courses together to form
  coherent programs how courses provide foundation
  for subsequent courses
• Assessment results are evidence of quality T/L

98
Assessing Learning Outcomes

• No one right or best way
• Strategic Questions
• What do you want assessment to do?
• What questions do you want it to answer?
• What assessment method will answer these
  questions?

99
Reflection Assessment

• Do outcomes determine what is assessed or does
  ease of assessment determine outcomes?
• Are achievements inferred from process completion
  or affirmed by assessment?
• Are assessment criteria and methods relevant and
  meaningful for outcomes being evaluated?
• How valid and reliable is the assessment of
  outcomes?
• Who assesses? Who evaluates? Are these
  assessors and evaluators selected for convenience
  or effectiveness?

100
Reflection Assimilation

• Does assessment occur on a regular, purposeful
  basis?
• How are assessment findings used for planning and
  decision making?
• How are assessment findings used to improve
  institutional, unit and program effectiveness?
• How are assessment findings used to improve
  teaching effectiveness and student learning?

101
Reflection Assimilation

• Is higher order thinking assessed when evaluating
  student learning?
• Does the assessment relate to and support course
  / program outcomes?
• Does the assessment support the development of
  stated learning outcomes and competencies?
• Should course outcomes, test questions, or
  assignments be revised?

102
Reflection Assimilation

• Do assessment tools accurately reflect and assess
  identified learning outcomes?
• Are faculty in consensus as to assessment
  tools/methods and content so that learning
  outcomes for a given course can be assessed
  regardless of who has taught the course?
• Is there a common final exam, standardized
  assessment tool, portfolio, presentation, etc.
  that could provide across-course common outcomes
  assessment?

103
Reflection Assimilation

• Evaluate Assessment Tools
• Use outcome-measure comparison matrix to map
  learning outcomes to assessment instrument
• If gaps, use outcome-test item comparison matrix
  to map assessment items to learning outcomes

104
The Bridge

• Level Outcomes Assessment
• College College-wide Employment Rate
• Programs that support workforce College-wide
  Retention Rate
• development College-wide Graduation Rate
• Program Outcomes Automotive
• 1. Employed in field Program Placement Rate
• 2. Function safely, competently,
  ethically Employer Survey
• 3. Graduate ASE Certifications Graduate Success
  Rate on ASE Certifications
• 4. Self-Starter and Team Player Employer Survey

105
The Bridge

• Level Outcomes Assessment
• Program Outcome 1 Employed in field
• Course AUT 220 Automotive Technology Course
  Grade
• Internship
• Competencies
• (1) Demonstrate appropriate Employability
  Skills
• Learning Outcome Communicate effectively oral
• and written
  skills Supervisor Evaluation
• (2) Acceptable Job Performance
• Learning Outcome Perform a thorough automotive
• assessment,
  recommend and
• perform
  necessary repairs Skills performance

106
The Bridge

• Level Outcomes Assessment
• Program Outcome 2 Function safely, competently,
  ethically
• Course AUT 120 Introduction to Automotive
• Technology Course
  Theory Grade
• Course Shop Grade
• Competencies
• (1) Demonstrate safe practices in lab setting
• Learning Outcome Analyze hazardous conditions
  and
• implement
  safety precautions Shop Performance
  Safety Unit
• (2) Summarize legal / ethical responsibilities
• Learning Outcome Describe legal and ethical
• implications
  r/t billable hours Unit Exam

107
Wrap-Up

• Pat Cross Basic Assumptions
• The quality of student learning is directly,
  although not exclusively, related to the quality
  of teaching. Therefore, one of the most
  promising ways to improve learning is to improve
  teaching.
• To improve teaching, teachers need first to make
  their goals and objectives explicit and then to
  get specific, comprehensible feedback on the
  extent to which they are achieving those goals
  and objectives.
• To improve learning, students need to receive
  appropriate and focused feedback early and often
  they also need to learn how to assess their own
  learning.

108
Curriculum Database Process

• Application of Theory to Curriculum Database Tool

109
The Relationship between Programs, Courses and
Competencies
Program Course Course Competency
Learning Outcome Learning Domain Levels
of Learning Curriculum Integration (Core
Ability) Learning Outcome Learning
Domain Levels of Learning Curriculum
Integration
110
Database Population
111
Database Population
112
Database Population
113
Database Population
114
Database Population
115
Database Population
116
Database Population
117
Database Population
118
(No Transcript)
119
Database Population
120
(No Transcript)
121
(No Transcript)
122
(No Transcript)
123
(No Transcript)
124
(No Transcript)
125
(No Transcript)
126
(No Transcript)
127
(No Transcript)
128
(No Transcript)
129
(No Transcript)
130
(No Transcript)
131
(No Transcript)
132
(No Transcript)
133
Additional Resources Learning Objectives

• Angelo, Thomas A. and Cross, K. Patricia.
  Classroom Assessment Techniques A Handbook for
  College Teachers. 2nd Edition. Jossey-Bass
  Publishers. San Francisco, CA. 1993
• Cross, K. Patricia. Learning is About Making
  Connections. The Cross Papers, Number 3. League
  for Innovation in the Community College
  Educational Testing Service. 1999
• Lowman, Joseph. Mastering the Techniques of
  Teaching, 2nd edition. Jossey-Bass Publishing,
  San Francisco, CA. 1995.
• Popham, W. James Baker, Eva L. Establishing
  Instructional Goals. Prentice-Hall, Inc., New
  Jersey. 1970
• Suskie, Linda. Assessing Student Learning A
  Common Sense Guide. Anker Publishing Company,
  Bolton, MA. 2004.
• Wilson, Cynthia D., et.al,. Learning outcomes
  for the 21st Century Report of a Community
  College Study. League for Innovation in the
  Community College The Pew Charitable Trusts.
  2000.

134
Additional Resources Assessment and Evaluation

• Banta, T.W. et. al., (Eds.), (1996), Assessment
  in Practice Putting Principles to Work on
  College Campuses. San Francisco Jossey-Bass
  Publishers. ISBN 0-7879-0134-2
• Erwin, T. Dary, (1991), Assessing Student
  Learning and Development. San Francisco
  Jossey-Bass Publishing. ISBN 1-55542-325-6
• Nichols, James O., (1995), A Practitioners
  Handbook for Institutional Effectiveness and
  Student Outcomes Assessment Implementation. New
  York Agathon Press. ISBN 0-87586-113-x
• Nichols, James O., (1995), Assessment Case
  Studies Common Issues in Implementation with
  Various Campus Approaches to Resolution. New
  York Agathon Press. ISBN 0-87586-112-1

135
Additional Resources Assessment and Evaluation

• Palomba, C. A.. Banta T. W., (Eds.), (1999),
  Assessment Essentials Planning, Implementing,
  and Improving Assessment in Higher Education.
  San Francisco Jossey-Bass Publishing. ISBN
  0-7879-4180-8
• Suskie, Linda, (2004), Assessing Student
  Learning A Common Sense Guide. Bolton, MA
  Anker Publishing Company. ISBN 1-882982-71-1
• Woolvard, Barbara E., (2004), Assessment Clear
  and Simple A Practical Guide for Institutions,
  Departments, and General Education. San
  Francisco Jossey-Bass Publishers. ISBN
  0-7879-7311-4