PISA 2003 A Comparison of the German federal states Lnder

1
PISA 2003 -A Comparison of the German federal
states (Länder)
Silke Rönnebeck, IPN Leibniz-Institute for
Science Education
PISA 2006 The Performance of Education Systems
in Countries and Regions OECD/PISA-Conference,
3-4 April 2008, Trento/Italy
2
Outline

• The educational system in Germany
• The conception of the Länder comparison
• Students performance in ...
• mathematics
• reading
• science
• The Länder at a glance
• Students performance and social background
• Conclusions

3
The educational system in Germany
4
Germany a federal state

• 3 city states and 13 area states (Länder)
• Education policy is subject to federal state
  authority
• 16 (more or less similar) educational systems
  with different school types and curricula

5
The German educational system
integrative
6
School types and educational contribution

• Gymnasium in 16 Ländern, contribution between
• 25.2 and 34.5
• Realschule in 12 Ländern, contribution between
• 9.6 and 33.5
• Hauptschule in 10 Ländern, contribution between
• 10.6 and 32.2
• Gesamtschule in 8 Ländern, contribution between
• 7.3 and 50.1
• Middle, standard, secondary, regional or combined
  schools (schools with multiple study programs)
• in 7 Ländern, contribution between 5.1 and 66.3
  

7
What has happened since PISA 2000?

• Standing Conference of the Ministers of Education
  and Cultural Affairs of the Länder in the Federal
  Republic of Germany implemented 7 fields of
  action, e. g.
• preschool advancement (e. g. improvement of
  language skills)
• avoid delayed educational careers by more
  flexible but preferably early enrollment,
  prevention of deferments
• quality monitoring and assurance by the
  implementation of educational standards,
  countrywide comparative examinations, central
  leaving examinations etc.
• School structure debate (e. g. abandoning the
  Hauptschule, 6 years of primary school, general
  university entrance qualification after 12 years
  etc.)

8
Conception of the Länder comparison
9
National extensions of PISA 2003 in Germany

• In Germany, PISA 2003 consisted of three samples
• PISA-I (PISA-international sample)
• PISA-E (extended sample) comparison of the
  German Länder
• 3. PISA-I-Plus supplement to the
  international sample two additional 9th grade
  classes per sampled school reassessed in 2004
  (increase in competency, multi-level-analysis)

10
Population and PISA samples in Germany
11
Comparison of the German Länder

• Sampling, test design, assessment and data
  analysis followed the international PISA
  technical standards
• For the Länder comparison 44580 students at 1487
  schools were tested.
• In all Länder, the student participation rates
  met the international criteria (ranging from 85
  to 96 ).
• Aims
• A profile of students performance in
  mathematics, reading, science (and
  problem-solving)
• Information about the relationship between
  performance and characteristics of students
  social background
• Trends in students performance between PISA 2000
  and 2003

12
Are the results reliable? - Elimination of bias

• Mainly low performing students refuse to
  participate
• No significant differences in the school grades
  between sampled/tested students
• Some Länder perform teaching to the test
• No significant improvement in performance for
  students who had been trained
• Students are tired of being tested
• The test motivation in PISA 2003 was a little bit
  higher than in PISA 2000.

13
Mathematical Literacy
14
Percentiles and normal distribution
15
Percentiles for the performance in mathematics
German Länder and OECD-countries
d 62
16
Percentage of students at the lowest and highest
proficiency levels on the mathematics scale
17
Performance on the four mathematics subscales

• In the international comparison, German students
  show a relative strength on the Quantity (and,
  less pronounced, Change and Relationship)
  subscale and a relative weakness on the
  Uncertainty subscale.
• These relative strengths and weaknesses can also
  be found in almost all of the Länder.
• The ranking of the Länder on a national level is
  more or less identical for all four subscales
  (with Bayern, Sachsen, Baden-Württemberg and
  Thüringen at the top of the scale and Hamburg and
  Bremen at the bottom).

18
Comparison of the mean values on the mathematics
subscale Change and Relationship between PISA
2000 and PISA 2003
19
Comparison of the mathematical competency at the
Gymnasium mean values and percentiles according
to the Land (combined mathematics scale)
d 51
d 65
d 46
20
Comparison of the mean values on the mathematics
subscale Change and Relationship between PISA
2000 and PISA 2003 at the Gymnasium
21
Reading Literacy
22
Performance in reading in the Länder comparison

• 3 Länder above the OECD-average and 4 within
• Great variance across (51 points) and within the
  Länder
• Relative weakness in low performing region
  (difference of 53 points on 5th percentile
  compared to 28 points on 95th)
• In 12 states amount of at-risk students is 1/5 of
  the population or more
• 5 Länder show significant increases in
  performance compared to PISA 2000 (ranging from
  12 to 27 points)
• Biggest increases are found for Länder that
  performed very poorly in PISA 2000

23
Comparison of the mean values on the reading
scale between PISA 2000 and PISA 2003 at the
Gymnasium
d 32
24
Scientific Literacy
25
Performance in science in the Länder comparison

• 12 Länder above or within the OECD-average
• Great variance across (53 points) and within the
  Länder
• Relative weakness in low performing region
  (difference of 59 points on 5th percentile
  compared to 29 points on 95th)
• 12 Länder showed significant increases in
  performance compared to PISA 2000 (ranging from
  11 to 32 points)
• Increases mainly due to improvements in the
  medium competency region

26
Comparison of the science competency at the
Gymnasium mean values and percentiles according
to the Land
d 41
27
Comparison of the mean values on the science
scale between PISA 2000 and PISA 2003 at the
Gymnasium
28
Key results

• In PISA 2003 ....
• 12 Länder in mathematics and science and 7 Länder
  in reading, respectively, are located within or
  above the OECD-average on the respective
  competency scales.
• The differences in performance across the Länder
  are between 51 score points in reading and 62
  score points in mathematics (comparable to
  approximately 1 1.5 years of schooling).
• These huge differences are mainly caused by
  students in the low performing region of the
  scale. In all Länder, the percentages of at-risk
  students are far too high in 10 and 12 Länder,
  respectively, they amount to more than 1/5 of the
  population.

29
The Länder at a glance
30
Grade distribution and delayed educational career

• Percentage of 15-year-olds in the 10th grade
  varies between 16.1 and 32.6
• Percentage of 15-year-olds in the 9th grade
  varies between 53.4 and 90.7
• Percentage of 15-year-olds in the 8th grade
  varies between 8.4 and 25.4
• and
• Percentage of students that have at least once
  been deferred or repeating a class varies between
  20.3 and 47.4

31
Percentiles for the competencies and percentage
of school types in Bayern
32
Social background and performance
33
Comparison of the economic, social and cultural
status (ESCS) with respect to the Land
34
Mathematical competency and explained variance
(R2) by the ESCS-index
35
Gymnasium attendance according to social
background
36
Odds ratios for attendance of the Gymnasium
according to ESCS and Land
37
Conclusions from the Länder comparison
38
Performance compared to the OECD average for all
domains
significantly above the OECD average -
significantly below the OECD average o not
significantly different from the OECD average
39
Gains in all domains compared to PISA 2000
significant gain o no significant difference
40
However, .....

• In all Länder and for all domains the variance in
  performance is still very high.
• In all Länder and for all domains the percentage
  of academically weak students is still too high.
• In all Länder and for all domains the linkage
  between performance and social background is
  still too strong.
• All Länder are more or less prodigal with the
  resource lifetime.

41
Further Reading
Leibniz-Institut für die Pädagogik der
Naturwissenschaften (IPN)
Leibniz Institute for Science Education
42
Thank you very much for your attention!
43
Distribution of students according to Land and
school type
44
Distribution of PISA-E-schools according to Land
and school type
45
Participation rates of 15-year-olds for the test
and student questionnaire according to Land and
school type (in )
46
Comparison of the school grades for all sampled
students and students actually assessed according
to Land
47
Percentiles for the performance in reading
German Länder and OECD-countries
48
Percentage of students at the lowest and highest
proficiency levels on the reading scale
49
Comparison of the mean values on the reading
scale between PISA 2000 and PISA 2003
50
Percentiles for the performance in science
German Länder and OECD-countries
51
Comparison of the mean values on the science
scale between PISA 2000 and PISA 2003
52
Distribution of 15-year-olds on grades and grade
repetition in Bayern (percentage of students)
53
Mean values in mathematics and percentage of
students at the Gymnasium
54
Mean values in reading and percentage of students
at the Gymnasium
55
Mean values in science and percentage of students
at the Gymnasium
56
Expenditure of time for mathematics and being
late for classes

• Relatively high conformity across the Länder and
  school types with respect to regular classes and
  homework
• Percentage of tutoring varies more between
  school types than between the Länder.
• and
• between 12.7 and 37.4 of the students have
  been late for classes during the last two weeks.

57
Expenditure of time for mathematics in Bayern for
the different study programmes
58
Relationship between mathematical competency and
the economic, social and cultural background in
PISA-E