David C. Berliner

1
(No Transcript)
2
David C. Berliner Regents Professor
Emeritus Mary Lou Fulton Teachers College Arizona
State University
How High-Stakes Testing is Hurting American
Education Institute of Education,
Higher School of Economics
Moscow, September, 2014
3
The theory of action underlying NCLB Is
this Lazy teachers and lazy students abound,
and that they can be motivated to work harder in
school. Teachers and students will learn more
if we have external tests with which to judge
them and pressure them. Through the tests we
can identify those lazy teachers and kids, as
well as the schools with low expectations, and
punish all those bad teachers and students for
their laziness and improper beliefs. Side Issue
of great importance A great bipartisan switch
took place under President George Bush, with the
aid of hundreds of other elected leaders and the
business community Congress decided to worry
about outputs, and ignore inputs. Let me
illustrate
4
Framing School Reform New York Times, 1981-2006
5
Framing School Reform Los Angeles Times,
1981-2006
6
Framing School Reform Boston Globe, 1981-2006
7
Framing School Reform Chicago Tribune, 1981-2006
8
Framing School Reform Education Week, 1981-2006
9
Framing School Reform Washington Post, 1981-2006
10
Framing School Reformwww.whitehouse.gov,
2001-2007

• Achievement Gap
• 344 documents
• Equal Educational Opportunity
• 3 documents

11
How you frame the problem (George Lakoff) often
determines how you try to solve it!
We have been subtly moved from concerns for
social justice in order to have the schools we
want for our children, to concerns about how
those children score on tests. We have been moved
from worrying about equal educational
opportunity, and health care, and livable wages,
and food insecurity, and neighborhoods that are
filled with drugs, crime and poor role models,
and a host of other variables that affect the
achievement of students who walk into our
schools for that we have substituted instead
concern for their test scores and what skills
they have when they walk out of our schools, as
if the two are unrelated! And if we find our
students deficient when they leave schools, its
natural to blame the school people, those that
worked with them last, instead of looking in the
mirror and saying we are all responsible for the
performance of all our children.
12
The question we had How might we test the
theory of action that underlies NCLB? Our
Solution Scale the states on pressure and check
how pressure affects NAEP scores. Our method
Prepare thick and rich descriptions of the
pressure felt by teachers, administrators,
parents and students in each state, as determined
from a sampling of newspaper articles in each
state. Then compare each state with every other
state (a 25 x 25 state matrix, requiring gt300
judges). Have one and only one judgment made per
judge Which of the two state portfolios you
are looking at seems to be exerting more pressure
for high performance on the achievement tests
that are used to comply with NCLB?
13
(No Transcript)
14
(No Transcript)
15
(No Transcript)
16
So, point one is made NCLB is probably not now
working. Point two is that NCLB cannot work,
ever. NCLB cannot work for many reasons, (e. g.
100 proficient and all kids reaching proficiency
at the same time). But the most important
reason it cannot work is that it violates a
fundamental social science law demonstrated
through myriad incidents to be applicable to a
wide range of human endeavors. This is Campbells
law.
17
Campbells Law

• The More any quantitative social indicator is
  used for social decision-making, the more subject
  it will be to corruption pressures and the more
  apt it will be to distort and corrupt the social
  processes it is intended to monitor.
• (Donald Campbell, 1975, p.35)

18
The uncertainty principle

• George Madaus, 2002, pointed out that this law
  leads to the social science equivalent of the
  Heisenberg uncertainty principle
• The higher the stakes involved in testing, the
  less likely you are to get an accurate
  measurement of the construct you most want to
  measure.
• So you simply cannot have both high-stakes and
  high validity because the higher the stakes the
  more corrupt the measure becomes.
• Thus, uncertainty about the meaning of a test
  score rises with increases in the consequences
  associated with scores on that test.

19
(No Transcript)
20
(No Transcript)
21
(No Transcript)
22
(No Transcript)
23
We found schools that did some odd
classification of special education and English
Language learners and even reclassified mixed
race children in odd ways. We found schools
that move kids around to they didnt have to test
them. We found schools that claimed they were
safe when they were not. We found that the
tests often forced a restriction in the cognitive
processes specified in the standards. We
found schools that vastly narrowed the curriculum
in social studies, art, music, and physical
education.
24
What we have learned is that tests may not
predict the future very well. This is especially
true when the future is Volatile, Uncertain,
Complex, and Ambiguous--when a VUCA future is
likely. And this serious lack of predictive
validity is magnified when the scores are
compromised. When the every-day problems of
long-term predictive validity for tests are
combined with high-stakes testing, the construct
validity of the tests are compromised as well.
That is, we may no longer be measuring what we
think we are measuring due to the corruption of
the indicator and the corruption of those who
work with that indicator. Campbells law is
ubiquitous! This is a social science law of great
power, defied by Texas and the federal government
at our great risk. High-stakes testing is wicked
policy It not only corrupts it may hurt the
economy of the nation, as well.
25
The longest running high-stakes testing program
in history CHINAS CIVIL SERVICE
EXAMS 606-1905----1,298 years of very
high-stakes tests 4 findings test takers
cheated a lot test preparers cheated a
lot test scorers cheated a lot test proctors
cheated a lot Around the year 1400 test changes
were made and China changes at that time as
well. Could it be causal?
26
The Bicycle Ride (1946)

• I awakened early, jumped out of bed and had a
  quick breakfast. My friend, Mary Quant, was
  coming to our house at nine oclock as we were
  going for a long bicycle ride together.
• It was a lovely morning. White fleecy clouds
  floated in a clear blue sky and the sun was
  shining. As we cycled over Castlemore bridge we
  could hear the babble of the clear stream beneath
  us. Away to our right we could see the brilliant
  flowers in Mrs. Caseys garden. Early summer
  roses grew all over the pergola which stood in
  the middle of the garden.

27
A Day in the Bog (1947)

• I awakened early and jumped out of bed. I wanted
  to be ready at nine oclock when my friend,
  Sadie, was coming to our house. Daddy said he
  would take us with him to the bog if the day was
  good.
• It was a lovely day. White fleecy clouds
  floated in a clear blue sky. As we were going
  over Castlemore bridge in the horse and cart, we
  could hear the babble of the clear stream beneath
  us. Away to our right we could see the brilliant
  flowers in Mrs. Caseys garden. Early summer
  roses grew all over the pergola which stood in
  the middle of the garden.

28
A Bus Tour (1948)

• I awakened early and sprang out of bed. I wanted
  to be ready in good time for our bus tour from
  the school. My friend, Nora Green, was going to
  call me at half-past eight as the tour was
  starting at nine.
• It was lovely morning. White fleece clouds
  floated in the clear blue sky and the sun was
  shining. As we drive over Castlemore bridge we
  could hear the babble of the clear stream beneath
  us. From the bus window we could see Mrs. Caseys
  garden. Early summer roses grew all over the
  pergola which stood in the middle of the garden.

29
The lesson from the study of China is to be
wary of tests that reflect a mimetic bias--they
influence what is valued and thus what is taught.
The lesson of Ireland is to give teachers
some autonomy so they dont want to please the
test makers or the governement. For
individuals and for societies the more VUCA-like
the future, the less likely that the knowledge of
the past will serve as a reliable guide for
success in that future. Creativity, problem
identification, knowledge seeking, problem
solving, media literacy, ability to sort facts
from fiction in a knowledge rich world, Bayesian
and probabilistic estimation, systems thinking,
collaborative work skills, and so forth, are more
likely to be the skills that youth need if a
society is to compete well in the 21st century.
The US curriculum and its assessments do not
reflect such ideas. The US is becoming much more
of a nation that values the mimetic and not the
transformative roles of the teacher! THAT IS
A MISTAKE!
30
Teachers Voice Colorado..We dont
take as many field trips. We dont do community
outreach like we used to, like visiting the
nursing home or cleaning up the park because we
had adopted a park and that was our job, to keep
it clean. Well, we dont have time for that any
more.
31
Another Colorado teacher We only teach to
the test, even at 2nd grade, and have stopped
teaching science and social studies. We dont
have assemblies, take few field trips, or have
musical productions at grade levels. We even
hesitate to ever show a video. Our 2nd graders
have no recess except for 20 minutes at lunch.
32
A teacher in Florida says 2 Our total
curriculum is focused on reading, writing, and
math. There is no extra time for students to
study the arts, have physical education, science,
or social studies. Our curriculum is very
unbalanced.
33

John Adams I must study politics and war that
my sons may have liberty to study mathematics and
philosophy. My sons ought to study mathematics
and philosophy, geography, natural history, naval
architecture, navigation, commerce and
agriculture in order to give their children a
right to study painting, poetry, music,
architecture, statuary, tapestry, and porcelain.
34

Screwing up our kids views of schooling
Testing Ms. Malarkey Book by Judy
Finchler Illustrations by Kevin OMalley
35
(No Transcript)
36
(No Transcript)
37
(No Transcript)
38
(No Transcript)
39
(No Transcript)
40
(No Transcript)
41
(No Transcript)
42
(No Transcript)
43
(No Transcript)
44
(No Transcript)
45
(No Transcript)
46
(No Transcript)
47
Testing and teaching to us is the same thing.
Secretary Paige quoted in the Herald-Tribune
(Southwest Florida), July 14, 2004.
48
http//www.youtube.com/watch?v8dAujuqCo7s