Stages of Concern During Curriculum Change: Formative Evaluation - PowerPoint PPT Presentation

1 / 60
About This Presentation
Title:

Stages of Concern During Curriculum Change: Formative Evaluation

Description:

Stages of Concern During Curriculum Change: Formative Evaluation India Broyles EdD and Mildred Savidge PhD University of New England, College of Osteopathic Medicine – PowerPoint PPT presentation

Number of Views:152
Avg rating:3.0/5.0
Slides: 61
Provided by: dataAacom
Category:

less

Transcript and Presenter's Notes

Title: Stages of Concern During Curriculum Change: Formative Evaluation


1
Stages of Concern During Curriculum Change
Formative Evaluation
  • India Broyles EdD and Mildred Savidge PhD
  • University of New England,
  • College of Osteopathic Medicine
  • 11 Hills Beach Rd, Biddeford, ME 04005

2
What is the Difference in Formative and
Summative Evaluation?
  • "When the cook tastes the soup, thats
    formative.
  • When the guests taste the soup, thats
    summative."

http//www.designedtoat.com/food2.shtml
3
Learning Outcomes
Workshop Objectives
  • The workshop will allow participants to
  • Value formative evaluation during change
  • Complete a sample of the SoC questionnaire
  • Adjust the questionnaire to meet their own
    innovation or curriculum change
  • Score the questionnaire and to make preliminary
    analyses of different types of profiles
  • Predict ways in which SoC data can facilitate the
    change process

4
Agenda
  • 10 minutes Formative Evaluation of the change
    process
  • 20 minutes Complete the SoC questionnaire with
    competency-based education as the innovation.
    Score and report.
  • 15 minutes Understanding the stages of concern
    and the SoCQ
  • 15 minutes What innovations are you
    implementing? Rewrite the questions. Exchange
    with a partner to review/edit the items.
    Questions?
  • 20 minutes Analysis of the data
  • 10 minutes What does the college do with the
    formative evaluation data?

5
Formative Evaluation
Formative Evaluation
  • Formative evaluation can serve as a guide for a
    college in the change process and improves the
    chances of a successful curriculum innovation
    that will be both well accepted and long lasting.

6
Formative Evaluation
  • Formative evaluation is a method of judging the
    worth of a program while the program activities
    are forming or happening.
  • Formative evaluation focuses on the process.
  • (Bhola 1990).

7
Formative Evaluation typically involves
  • gathering information during the early stages of
    your project or program
  • a focus on finding out whether your efforts are
    unfolding as planned
  • uncovering any obstacles, barriers or unexpected
    opportunities that may have emerged
  • identifying mid-course adjustments and
    corrections which can help insure the success of
    your work (NW Regional Education Lab)
  • How do we do this?

8
The process of developing and implementing an
innovation is
  • Change
  • So we look at the literature on educational
    change.

9
Individual Innovativeness TheoryCarl Rogers
(1995)
  • Innovators
  • Early Adopters
  • Early Majority
  • Late Majority
  • Laggards

10
Change is an ongoing process, not a short-term
event.
  • Change requires ongoing support and resources
    and it takes time. Sometimes new users of an
    innovation get discouraged when they don't see
    immediate results. It is important to have
    realistic expectations about the time it will
    take to see significant progress and to make sure
    that other stakeholders in the community
    understand this as well. Failure to address key
    aspects of the change process can either add
    years to, or even prevent, successful
    implementation.
  • Gene Hall, Shirley Hord, Archie George.
    Southwest Educational Development Lab

11
Change occurs in individuals first, then in
organizations.
  • The best curriculum in the world will not succeed
    in your institution unless the people
    implementing it are ready and willing to make it
    a success. However, individual change is
    difficult if the organization is not supportive
    of the change.
  • Gene Hall, Shirley Hord, Archie George.
    Southwest Educational Development Lab

12
One Change in Medical Schools
  • Competency-based Education with emphasis on
    national core competencies.
  • Please complete the questionnaire. Takes about
    10 minutes. Score using quick-scoring guide.
    Create your profile. You will have 20 minutes
    total. Give the quick scoring guide to Millie to
    record. Be sure your name is on it.

13
Agenda
  • 10 minutes Formative Evaluation of the change
    process
  • 20 minutes Complete the SoC questionnaire with
    competency-based education as the innovation.
    Score and report.
  • 15 minutes Understanding the stages of concern
    and the SoCQ
  • 15 minutes What innovations are you
    implementing? Rewrite the questions. Exchange
    with a partner to review/edit the items.
    Questions
  • 20 minutes Analysis of the data
  • 10 minutes What does the college do with the
    formative evaluation data?

14
What are Concerns?
  • The world around us is complex. It is not
    humanly possible to focus at one one time on all
    of the many different issues. Certain aspects of
    our world are of higher priority.
  • The composite representation of the feelings,
    preoccupation, thought and consideration to a
    particular issue or task is called a concern.

15
Concerns
  • Depending on ones closeness to and involvement
    with an innovation, ones concerns will be
    different in type as well as in intensity.
  • Many types or levels of concerns can be
    experienced concurrently.
  • There is a predictable pattern to the movement of
    intensity of concern across types.

16
Based on studies at UTEXAS, Austin since 1974
  • Stages have been identified with a developmental
    movement through these stages.
  • Earlier concerns must be resolved before later
    concerns emerge.
  • Timely provision of affective experiences and
    cognitive resources can facilitate resolution of
    early concerns and development of higher
    concerns.
  • Personalized interventions are important.

17
Stages of Concern
  • Stage 0 -- Awareness. Users have little concern
    or involvement with the innovation.
  • Stage 1 -- Informational. Users have a general
    interest in the innovation and

  • would like to know more about it.
  • Stage 2 -- Personal. Users want to learn
    about the personal ramifications of the
  • innovation.
    They question how the innovation will affect
    them.
  • Stage 3 -- Management. Users learn the
    processes and tasks of the innovation.

  • They focus on information and resources.
  • Stage 4 -- Consequence. Users focus
    on the innovation's impact on learners.
  • Stage 5 -- Collaboration. Users
    cooperate with other users in implementing the

  • innovation.
  • Stage 6 -- Refocusing. Users
    consider the benefits of the innovation and

  • think of additional alternatives that might work
    even better.

18
Concerns-Based Adoption ModelGene Hall, Shirley
Hord, Archie George. Southwest Educational
Development Lab
  • Stages of Concern
  • Questionnaire
  • Interviews
  • Levels of Use
  • Interviews
  • Innovation Configuration

19
SoCQ Instrument
  • Reliability
  • Coefficients of internal reliability
  • Test/Retest correlations
  • Validity
  • Statistical
  • Content
  • Face

20
Coefficients of Internal ReliabilityN 830
21
Multiple studies Coefficients of Internal
Reliability for each stage
22
Test-Retest Correlations N132
23
Validity Studies Statistical
  • Factor analysis
  • 7 stages explained 60 of variance
  • All had Eigen values above 1.0
  • Hypothesized scales matched factor scales
  • Each scale measured an independent construct
  • Correlations between scale scores shows strong
    positive relationship between contiguous stages

24
Factor Analysis
25
Correlations between Scale Scores
26
Validity Comparison Studies
  • Focus on accuracy in measuring highest and lowest
    concerns
  • Interviews and open-ended responses used
  • Statements matched to scale scores/ profiles
  • Qualitative analysis showing greatest and least
    concerns correlated with scale scores
  • External judges used to establish relationships
    between interview analysis and scores

27
Validity Longitudinal Studies
  • Two 2-year longitudinal studies
  • Showed concerns changing over time in accordance
    with concerns theory
  • Concerns measured by the instrument moved from
    high Awareness, Informational and Personal
    concerns to lower concerns on all levels, or
    higher Consequence, Collaboration and Refocusing
    concerns

28
Use in clinical education
  • Lewis and Watson (1997) measured the concerns of
    57 nursing faculty about the use of computer
    technology. Their pre-post study results suggest
    that the primary concerns of the faculty were
    informational and that addressing these concerns
    through workshops increased interest in the
    innovation.
  • Gwele (1996) measured the concerns of nurse
    educators (n93) at four nursing colleges during
    the implementation of a major legislated
    curriculum reform. Concluded that when staff is
    required to adopt a major curriculum change the
    normal progression through the stages of concern
    is impeded. They suggested that in these
    situations it may be important to delay adoption
    until participants can come to terms with the
    need to adopt the new curriculum.
  • Arwer, Harris Dusold (2004) assessed the
    concerns of staff during the installation of a
    telemedicine system and to assure that concerns
    were addressed during system implementation.
    Survey findings were used successfully to modify
    the implementation and training phases of the
    program to better meet the needs of the staff.

29
Components
  • Cover Letter / Introductory Page
  • 35 items
  • Demographics
  • Create a SoCQ for an innovation in your own
    institution. (15 minutes)

30
Interpretation of SoC Data
  • Look at your scores/profile on CBE
  • Peak Stage Score
  • 1st and 2nd High Stage Score
  • Profile Interpretation

31
Peak Stage Scores
32
Group Data
  • Tally the number of individuals that are high on
    each stage. This gives a clear picture of the
    range of peak stage scores within a group (show
    of hands - how many at each stage).
  • Aggregate individual data by developing a profile
    that presents the mean scores of each stage of
    the individuals.

33
Typical Non-User
34
Institutional Profile of Stages of Concern
35
Profile Interpretation
  • As individuals move from unawareness and nonuse
    of an innovation into beginning use and more
    highly sophisticated use, it is hypothesized that
    their concerns develop from early to late
    concerns
  • Use clinical interpretation techniques
  • The total score is not generally used in analyses
    because it does not have a unique meaning.

36
Institutional Profile, 2006
37
Negative One Two Split
38
Negative One -Two Split with Tailing Up at Stage
6
39
Stages of Concern
  • Stage 0 -- Awareness. Users have little concern
    or involvement with the innovation.
  • Stage 1 -- Informational. Users have a general
    interest in the innovation and

  • would like to know more about it.
  • Stage 2 -- Personal. Users want to learn
    about the personal ramifications of the
  • innovation.
    They question how the innovation will affect
    them.
  • Stage 3 -- Management. Users learn the
    processes and tasks of the innovation.

  • They focus on information and resources.
  • Stage 4 -- Consequence. Users focus
    on the innovation's impact on learners.
  • Stage 5 -- Collaboration. Users
    cooperate with other users in implementing the

  • innovation.
  • Stage 6 -- Refocusing. Users
    consider the benefits of the innovation and

  • think of additional alternatives that might work
    even better.

40
Intense Management Concerns
41
Consequence Concerns
42
High Collaboration and Consequence Concerns
43
Single High Collaboration Concerns
44
Stages of Concern
  • Stage 0 -- Awareness. Users have little concern
    or involvement with the innovation.
  • Stage 1 -- Informational. Users have a general
    interest in the innovation and

  • would like to know more about it.
  • Stage 2 -- Personal. Users want to learn
    about the personal ramifications of the
  • innovation.
    They question how the innovation will affect
    them.
  • Stage 3 -- Management. Users learn the
    processes and tasks of the innovation.

  • They focus on information and resources.
  • Stage 4 -- Consequence. Users focus
    on the innovation's impact on learners.
  • Stage 5 -- Collaboration. Users
    cooperate with other users in implementing the

  • innovation.
  • Stage 6 -- Refocusing. Users
    consider the benefits of the innovation and

  • think of additional alternatives that might work
    even better.

45
High Refocusing Concerns
46
High Management Concerns With Ideas
47
Stages of Concern
  • Stage 0 -- Awareness. Users have little concern
    or involvement with the innovation.
  • Stage 1 -- Informational. Users have a general
    interest in the innovation and

  • would like to know more about it.
  • Stage 2 -- Personal. Users want to learn
    about the personal ramifications of the
  • innovation.
    They question how the innovation will affect
    them.
  • Stage 3 -- Management. Users learn the
    processes and tasks of the innovation.

  • They focus on information and resources.
  • Stage 4 -- Consequence. Users focus
    on the innovation's impact on learners.
  • Stage 5 -- Collaboration. Users
    cooperate with other users in implementing the

  • innovation.
  • Stage 6 -- Refocusing. Users
    consider the benefits of the innovation and

  • think of additional alternatives that might work
    even better.

48
Profile of Impact-Concerned User and Coordinator
49
Unconcerned Innovation User
50
Guidelines
  • Establish a holistic perspective
  • Look at High and Low scores
  • Look at individual item responses
  • When scores are used in statistical analyses,
    developers recommend use of raw scores.
    Conversion to percentiles greatly affects the
    distribution of the scores.

51
Demographics
  • In previous research, no outstanding
    relationships between standard demographics age,
    gender, teaching experiences, etc.
  • What demographics would you choose?

52
Basic Scientist Profiles
  • Responses from 8 basic scientists show high
    concerns for awareness, but very different
    concerns for other stages. All but two have
    tailing up at Stage 6.

53
Clinical Faculty Profiles
  • Clinicians show more similar profiles at
    Awareness, Information, and Personal Concerns.
    Five clinicians have a high concern at
    refocusing. Although unusual for non-users, we
    are still in the development of several elements.

54
Staff Profiles
  • Similar to clinicians in early concerns, as
    expected in non-users. Low management and
    consequence concerns may reflect non-classroom
    duties. The have strong concerns about
    collaboration.

55
Follow-up Interviews
56
Themes from interviews
57
What we learned
  • The process of curriculum revisions needs to be
    systematic with timelines and responsibilities.
  • Everyone needs to be kept informed of the process
    and the outcomes of deliberations with
    information coming in multiple formats
  • a website
  • hallway bulletin boards
  • written material
  • key descriptive and research articles.
  • Care should be taken to release information in a
    logical fashion so that faculty and staff not
    directly involved in the development stages can
    understand the reasons for current plans and have
    an opportunity to respond if they are unhappy
    with plans at any given stage.

58
Additional resources will be needed
  • qualified assistance with effective use of WebCT
  • grant-writing support
  • faculty coaching
  • implementation of effective evaluation approaches

59
Understanding the stages of concern can result
  • in more targeted strategies
  • more relevant workshops
  • directed planning to implement the new curriculum
    plan thereby creating successful,
    institutionalized change

60
We believe
  • The importance of setting up a curriculum
    revision process based on both individual and
    institutional concerns has the potential for
    influencing the development and implementation of
    a new curriculum.

61
  • Lewis, D. and Watson, J.E. Nursing faculty
    concerns regarding the adoption of computer
    technology. Computers in Nursing. 1997
    15(2)71-6.
  • Gwele, N.S. Concerns of nurse educators regarding
    the implementation of a major curriculum reform.
    Journal of Advanced Nursing. 1996 24607-14
  • Arwer, J.M., Harris, K. and Dusold, J.M.
    Application of the concerns-based adoption model
    to the installation of telemedicine in a rural
    Missouri nursing home. Journal for Nurses in
    Staff Development. 2004 2042-9.
Write a Comment
User Comments (0)
About PowerShow.com