A New Vision for 21st Century Education - PowerPoint PPT Presentation

Loading...

PPT – A New Vision for 21st Century Education PowerPoint presentation | free to view - id: bb649-ZDc1Z



Loading


The Adobe Flash plugin is needed to view this content

Get the plugin now

View by Category
About This Presentation
Title:

A New Vision for 21st Century Education

Description:

San Francisco, California. June 27, 2006. Innovations for Assessing 21st Century Skills ... Personal Responsibility. People Skills. Self Direction. Social ... – PowerPoint PPT presentation

Number of Views:80
Avg rating:3.0/5.0
Slides: 86
Provided by: mcn792
Category:

less

Write a Comment
User Comments (0)
Transcript and Presenter's Notes

Title: A New Vision for 21st Century Education


1
Innovations for Assessing 21st Century Skills
A New Vision for 21st Century Education
Panel Presentation on Assessment of 21st Century
Skills CCSSO National Conference on Large Scale
Assessment San Francisco, California June 27, 2006
Insert Presenter Name Insert Presenter Title
Company Insert Event Name Insert
Date PLEASE NOTE This is only a template
presentation you may add examples and additional
slides based on your audience EDUCATION COMMUNITY
AUDIENCE
2
Overview
Key Message
In order to produce students with 21st century
skills, we need to assess 21st century skills.
3
Overview
  • Overview and Introduction Ken Kay
  • Who is the Partnership
  • Why are 21st Century Skills so critical?
  • What is the framework for 21st Century Skills?
  • Why is the assessment of 21st Century Skills so
    important?
  • What are the potential ways to assess 21st
  • Century Skills Kathy Comfort, Steve Klein,
  • and Terry Egan
  • III. What are some general observations about
    21st
  • Century Skills and Assessment Jim Popham

4
Who is the Partnership?
5
Who is the Partnership?
6
21st Century Skills
Why are 21st Century Skills so Critical?
5 Reasons
7
21st Century Skills
  • 1. We need our students to become effective
    21st Century Citizens.

8
21st Century Skills
2. The U.S. is falling behind.
9
21st Century Skills
Ranking of G8 countries 10th grade math
problem solving
OECD Ranking
Problem Solving
Math
Science
Reading
14th
15th
15th
18th
18th
24th
24th
2000
2000
2000
2003
2003
2003
2003
Source PISA, 2000, 2003
Courtesy of Cisco
Systems
10
21st Century Skills
3. The magnitude of our competition is changing.
11
21st Century Skills
China India
300 Million Skilled Workers
Japan
25 Million Skilled Workers
2025
1985
12
21st Century Skills
4. The nature of work is changing.
13
21st Century Skills
How many of your Parents Grandparents had only
one or two jobs in their lifetimes?
14
21st Century Skills
How many jobs will a young person have today
between age 18-38?
…10.2 jobs
15
21st Century Skills
21st Century
20th Century
1 2 Jobs
10 15 Jobs
Number of Jobs
Flexibility And Adaptability
Mastery of One Field
Job Requirement
Integration of 21st Century Skills into Subject
Matter Mastery
Subject Matter Mastery
Teaching Model
Subject Matter Mastery
Integration of 21st Century Skills into Subject
Matter Mastery
Assessment Model
16
21st Century Skills
5. To help all children reach higher levels of
academic achievement.
17
What is the Framework for 21st Century Skills?
18
21st Century Skills Framework
20th Century Education Model
19
21st Century Skills Framework
20
21st Century Skills Framework
Core Subjects
  • - English
  • Reading or Language Arts
  • Mathematics
  • Science
  • Foreign Languages
  • Civics
  • Government
  • Economics
  • Arts
  • History
  • Geography

21
21st Century Skills Framework
Thinking and Learning Skills
  • Critical Thinking Problem Solving Skills
  • Creativity Innovation Skills
  • Communication Information Skills
  • Collaboration Skills
  • Contextual Learning
  • Information and Media Literacy

22
21st Century Skills Framework
  • ICT Literacy
  • Information and communications technology (ICT)
    literacy is the ability to use technology to
    accomplish
  • thinking and learning skills
  • Critical Thinking Problem Solving Skills
  • Creativity Innovation Skills
  • Communication Information Skills
  • Collaboration Skills
  • Contextual Learning
  • Information and Media Literacy

23
21st Century Skills Framework
  • Personal Skills
  • Leadership
  • Ethics
  • Accountability
  • Adaptability
  • Productivity
  • Personal Responsibility
  • People Skills
  • Self Direction
  • Social Responsibility

24
21st Century Skills Framework
  • 21st Century Content
  • Global Awareness
  • Financial, Economic, Business and
    Entrepreneurship Literacy
  • Civic Literacy
  • Health Wellness Awareness

25
21st Century Skills Framework
  • The Design Specs for 21st Century Education
  • Core Subjects
  • Thinking Learning Skills
  • ICT Literacy
  • Personal Skills
  • 21st Century Content

26
Why is Assessment of 21st Century Skills so
important?
27
Assessment
Accountability and metrics are here to stay.
But, what we measure matters!
28
Assessment
If we want 21st Century innovative learners we
need to measure metrics of innovative learning!
29
Assessment
Assessment of 21st Century Skills The
Current Landscape June 2005 Partnership for
21st Century Skills www.21centuryskills.org
30
Conclusion
  • Every student in this nation must be
  • An analytic thinker
  • A problem solver
  • Innovative and Creative
  • An effective communicator
  • An effective collaborator
  • Information and media literate
  • Globally aware
  • Civically engaged
  • Financially and economically literate

31
Conclusion
In order to assure our student possess 21st
century skills, we need to assess 21st century
skills.
32
PASS
The Partnership for the Assessment of
Standards-based Science CCSSO Large-scale
Assessment Conference Partnership for 21st
Century Skills Panel San Francisco, CA June 27,
2006 Kathy Comfort, WestEd
33
PASS The Partnership for the Assessment of
Standards-Science
  • Developed in 1996 with NSF support
  • Test design and specifications grounded in
    research
  • Aligned to the National Science Education
    Standards
  • Contains different types of measures including
    performance tasks
  • Assess what is valued (not just easily measured)
  • Appropriate for grades 4-12
  • Students in 22 states and Puerto Rico have been
    assessed with PASS

34
The PASS Assessments and the 21st Century
Framework
35
PASS Assessment Components
36
Sample Performance Task
37
Sample Performance Task
38
Sample Performance Task
39
Sample Performance Task
40
Sample Performance Task
41
Sample Performance Task
42
Sample Performance Task
43
Sample Performance Task
44
Sample Performance Task
45
PASS Current and Future
  • PASS Current
  • Students record responses in test booklets
  • Scored by independent trained readers
  • PASS Future
  • Technology-based pilot found it is feasible to
    use computers to deliver the tasks to students
    and to
  • machine score their open-ended answers
  • Simulations (virtual labs)

46
PASS Assessments are Technically Sound
  • Valid - consensus on correspondence to standards
  • Statistical analysis have found PASS scores to be
    reliable and sensitive to the effects of
    instruction
  • High inter/intra reader consistency
  • Disaggregated data for gender and ethnic/racial
    groups

47
The Real Deal…
Feasibility Yes--it can be done as part of a
large-scale assessment, and it has.
  • However…
  • It is not easy to develop valid and reliable
    tasks. Pilot testing takes on a whole new
    meaning.
  • - If you cant get pumice to float neither
    will the
  • 10,000 teachers administering the
    test.
  • You need a lot of stuff to do performance
    assessment. (The stuff can be expensive and
    not easily found.)
  • Hand scoring takes time and readers must be
    highly trained and calibrated for reliable
    results.

48
Contact Information
Kathy Comfort PASS at WestEd WestEd 730 Harrison
St. San Francisco, 94107 415-615-3161 or
kcomfor_at_wested.org
49
Collegiate Learning Assessment (CLA)
  • Stephen Klein Council for Aid to Education (CAE)
  • June 27, 2006

50
Overview
  • Five principles driving the CLA
  • CLAs distinguishing features
  • CLAs measures
  • Reporting results
  • Program participation

51
Five Principles Driving the CLA
  • One test cannot assess overall quality
  • Can measure some important abilities
  • Need benchmarks to assess progress
  • Use results appropriately
  • Use psychometrically sound tests

52
CLAs Distinguishing Features
  • Use open-ended tests that are
  • Realistic work samples Typical performance
  • Engaging and psychometrically sound
  • Focused on high level abilities
  • Appropriate across schools and majors
  • School is unit of analysis
  • Sample students within schools
  • Matrix sample tasks
  • Paperless administration, scoring, reporting
  • Benchmark schools results to comparable
    institutions to provide value-added metric

53
CLAs Measures
  • Analytic writing (essay) prompts
  • Make-an-argument (45 minutes)
  • Break-an-argument (30 minutes)
  • Performance tasks (90 minutes)
  • Document based
  • Contextualized questions
  • Split screen/dialogue box format
  • Analytic and holistic scoring
  • Background questionnaire

54
Make-An-Argument Prompt
  • In our time, specialists of all kinds are
    highly overrated. We need more generalists
    people who can provide broad perspectives.
  • Directions 45 Minutes, agree or disagree and
    explain the reasons for your position. Student
    selects one of two prompts to answer.

55
Break-An-Argument Prompt
  • Students are asked to discuss how well reasoned
    they find an argument to be (rather than simply
    agreeing or disagreeing with it).
  • A respected professional journal with a
    readership that includes elementary school
    principals published the results of a two-year
    study on childhood obesity. This study sampled 50
    children, ages 5-11, from Smith Elementary
    School. A fast food restaurant opened near the
    school just before the study began. After two
    years, students who remained in the sample were
    more likely to be overweightrelative to the
    national average. Based on this study, the
    principal of Jones Elementary School decided to
    address her schools obesity problem by opposing
    the opening of any fast food restaurants near her
    school.

56
Performance Tasks
  • 90-minute real life problems
  • General directions and context
  • Need to combine information from different
    types of documents
  • A few open-ended questions, no one right
    answermust explain rationale
  • Split screen
  • Right Document Library
  • Left Question and answer block

57
Swiftaire 235
  • You advise the president of DynaTech
  • DynaTech makes precision airplane navigation and
    communication instruments
  • DynaTechs sales manager suggests buying a
    Swiftaire 235 to visit customers and demo (and
    show confidence in) its products
  • Recent accident wing fell off in flight
  • Students tasks
  • Review document library
  • Write memo discussing pros and cons
  • of DynaTech getting a Swiftaire 235
  • Justify your recommendations

58
Document Library
  • Newspaper article about accident
  • Federal report about in-flight breakups
  • DynaTech email exchanges regarding reasons to buy
    or lease a Swiftaire 235
  • Excerpt from trade magazine article that compares
    Swiftaire 235s performance and safety
    characteristics to similar planes
  • Manufacturer specifications and required pilot
    training for Swiftaire 180 and 235

59
Scoring Rubric
  • Writing skills clear, organized, persuasive
  • Analysis, problem solving, reasoning skills
  • Integrates information from different sources
  • Recognize flaws and issuesnot swayed by
    emotional arguments, faulty logic, irrelevant
    information, etc.
  • Anticipates consequences and implications of
    alternative solutions and strategies
  • See pros and cons of competing explanations,
    points of view, and arguments
  • Weighs evidence based on its credibility

60
Variety of Performance Tasks
  • Airplane
  • Brain Boost
  • Catfish
  • Crime
  • Expedition
  • Lake to River
  • Museum
  • Parks
  • Skating

61
Psychometrics
  • Median inter-reader correlation .82
  • Internal consistency reliability
  • Student level .80
  • School level .97
  • Student level correlations with SAT
  • PT score .56
  • Essay score .44
  • School level correlations with SAT
  • PT score .92
  • Essay score .79
  • Total battery .88
  • N 15,000 students, 113 schools

62
Two Definitions of Value-Added
  • Improvement within an institution over time
    (e.g., seniors score higher than freshmen with
    the same SAT scores)
  • Improvement between classes within an institution
    relative to the improvement between classes at
    other institutions
  • Two types of benchmarks/reference groups
  • ACT/SAT scores only
  • Peer Group (school characteristics)

63
Fig. 1 Relationship Between Mean ACT Scores and
Mean Total CLA Scores for Freshmen
31
Your Institution (Freshmen) Others (Freshmen)
27
CLA Score
23
Regression Intercept 8.02 Slope 0.66 R-square 0.80
19
15
15
19
23
27
31
ACT Score
64
Fig. 2 Relationship Between Mean ACT Scores and
Mean Total CLA Scores for Seniors
31
Your Institution (Seniors) Others (Seniors)
27
CLA Score
23
Regression Intercept 11.96 Slope 0.62 R-square 0.7
5
19
15
15
19
23
27
31
ACT Score
65
Fig. 3 Relationship Between Mean ACT Scores and
mean Total CLA Scores for Freshmen and Seniors
31
27
CLA Score
23
19
15
15
19
23
27
31
ACT Score
66
Percentage of Schools by Carnegie Classification
IPEDS (N 1421)
CLA (N 115)
  • Carnegie Classification
  • Doctoral/Research Universities - Extensive 11 11
  • Doctoral/Research Universities - Intensive
    8 11
  • Masters Colleges Universities I 35 37
  • Masters Colleges Universities II 8 3
  • Bachelor Colleges Liberal Arts 16 19
  • Bachelor Colleges General 23 17

67
Program Participation
  • School Year Institutions Students
  • 2003-04 46 2,040
  • 2004-05 57 8,009
  • 2005-06 Fall Spring 121 gt30,000

68
Frequently Asked Questions
  • Does maturation affect score gains between
    classes?
  • Does student motivation affect scores?
  • How do you control for selection effects?
  • Is there an interaction between academic major
    and prompt type?
  • Why not use portfolios or graduate or
    professional school admissions tests?

69
Thinking Beyond Technology Partnership for 21st
Century Skills Panel CCSSO Conference San
Francisco, CA June, 2006 Teresa Egan Educational
Testing Service
ICT Literacy Assessment
70
ICT Literacy A Bridge
  • Information and Communication
  • Technology Literacy
  • Can I find information on the web?
  • Can I create a persuasive presentation?

Information Literacy
  • Technical Literacy

Database
Word Processing
Presentation
Access
Evaluate
Use
  • Can I bold a word?
  • Can I open a database?
  • Can you find information?
  • Can you evaluate authority?

71
Why are ICT Literacy skills an important
component of a 21st century model for education?
  • Being ICT literate impacts the way we live,
    learn, and work in todays information society.
  • Technical skills are not enough. To be successful
    citizens and productive workers in the 21st
    century, we must be able to apply technical
    skills to cognitive tasks retrieving,
    synthesizing and communicating information
    effectively.
  • There is a lack of information about the ICT
    literacy skills of students, and debate about how
    best to address this issue in academic curriculum.

72
What were the challenges in developing an
innovative measure of ICT Literacy proficiency?
  • Design an assessment with face validity
  • Authentic, relevant, performance-based
  • Measure higher-order thinking skills in a digital
    environment
  • Include ethical and legal considerations in the
    use of information
  • Ensure that it is based on solid measurement
    principles
  • Quality
  • Validity
  • Reliability
  • Provide both students and institutions with
    useful data and feedback

73
Where to Start?
  • 2001 Convene International Panel
  • Australia, Brazil, Canada, France, U.S.
  • Include representatives from business, education,
    and governmental organizations
  • Results Produced Digital Transformation a
    framework for ICT Literacy, available at
    www.ets.org/ictliteracy
  • Design an assessment based on this construct and
    the Association of College Research Libraries
    (ACRL) Information Literacy Competency Standards
    for Higher Education (January, 2000)
  • Engage subject matter experts from IHEs to
    develop and pilot the ICT Literacy Assessment

74
Where to Start?
Define
ICT Literacy Using cognitive skills in a
technological environment
Access
Manage
Integrate
Evaluate
Create
Communicate
75
Basic Design Features
  • Two levels of difficulty
  • 75 minutes in length
  • Interactive tasks using simulated software with
    look and feel of typical applications (databases,
    search engines, spreadsheets, email, etc.)
  • 3-5 minute and 15-minute tasks
  • Real-life scenarios
  • Get back on track mechanisms
  • Multiple scorable elements per task
  • Online score reporting

76
ICT Literacy Assessment Content
  • Content Areas
  • Humanities
  • Social Sciences
  • Natural Sciences
  • Practical Affairs
  • Popular Culture
  • Contexts
  • Academic
  • Business
  • Personal
  • Technology Tools
  • Word Processor
  • Presentation Software
  • Email
  • Database Search Engine
  • Web Browser/Search Engine
  • Spreadsheet or Table
  • Graphing Software
  • File Manager
  • Electronic Bulletin Board
  • Instant Messenger

77
Sample Score Report
78
What have we accomplished to date?
  • Administered two versions of the assessment
    large scale and individual
  • Two levels of difficulty
  • Advanced
  • Core
  • Over 10,000 students tested at 68 institutions

79
What are some of the findings from this initial
testing?
  • When selecting a research statement for a class
    assignment, 25 of test takers picked statements
    that did not address the assignment.
  • When asked to narrow an overly broad search, more
    than 80 of students could not correctly describe
    the specific problem with the initial search.
  • When asked to evaluate a set of websites, 48 of
    test takers identified the website that met the
    criteria of currency, authority and objectivity.

80
Student feedback from first administration
81
Student Comments
  • I feel that I probably did better on the tasks
    that dealt
  • with things that I use regularly email and
    instant
  • messaging.
  • I do not think that I am the most ICT literate
    person
  • in the world. I love to use computers and
    other
  • technologies for recreational uses, and see
    the vast
  • benefits of their other uses as well, I just
    have not
  • totally used them to their potential.
  • I had trouble with most of the test, but I
    expected this.
  • I need help getting started in this new
    technology.
  • All around, I think it's ok, but like my skills,
    there's
  • room for improvement.

82
What have been our biggest impediments to date?
  • Getting buy-in on the importance of measuring
    ICT Literacy funding!
  • Who owns the responsibility for ICT Literacy on
    campus? Faculty? Librarians? Administrators?
  • Delivery challenges computer labs, student
    recruitment, proctors, technical issues
  • Interpreting and utilizing data to effect changes
    in instructional design
  • Implications evident for professional development
    needs

83
Next Steps Ongoing Validity Studies
  • Comparison with self-report measures
  • Alignment with expectations of employers
  • Expert review of assessment design, tasks, and
    scoring
  • Cognitive strategies on ICT literacy assessment
    and naturalistic tasks
  • Educational outcomes of ICT literacy instruction
  • Comparison with writing portfolios

84
Contact Information
  • For More Information
  • Pick up a CD for demo tasks and more information
  • Visit the ETS booth for product demo
  • Visit our website
  • www.ets.org/ictliteracy

85
Contact Us
The Partnership for 21st Century Skills 177 North
Church Avenue, Suite 305 Tucson, AZ 85701 (520)
623-2466 www.21stcenturyskills.org
About PowerShow.com