HOW DID WE DO IN MATHS AND SCIENCE?

1
HOW DID WE DO IN MATHS AND SCIENCE?
TIMSS 2007

• Linda Sturman and Graham Ruddock
• NFER
• TIMSS Schools Conference, 22 May 2009

2
WHAT IS TIMSS?

• World-wide survey, every four years
• Trends in maths and science achievement
• 9-10 and 13-14 year olds (grades 4 and 8)
• In the UK, England and Scotland take part (Y5/Y9
  and P5/S2)
• Aims inform educational policy improve teaching
  and learning in maths and science
• www.nfer.ac.uk/timss

3
TIMSS 2007

• Each pupil took one test
• Two sections maths and science (36 mins or 45
  mins each)
• Pupil questionnaire
• Administered by NFER
• School and teacher questionnaires
• Participation targets

4
WHAT CAN TIMSS TELL US?

• How well we do in maths and science
• How well we do in the content and cognitive
  domains
• How we compare with other countries
• How performance has changed over time
• How we perform at each benchmark
• Which contextual factors are associated with
  achievement

5
TIMSS 2007
ASSESSMENT FRAMEWORK
Subject Grade 4 Grade 8
Science Life science Physical science Earth science Biology Chemistry Physics Earth science
Maths Number Geometric shapes and measures Data display Number Algebra Geometry Data and chance

• More on this later

6
TIMSS 2007 RESULTS
Grade 4 Science (Y5) 37
Countries
3 Countries Outscored England 3 Countries Outscored England 3 Countries Outscored England England outscored 26 other countries including England outscored 26 other countries including England outscored 26 other countries including England outscored 26 other countries including
Score Correct Score Correct
Singapore 587 69 Germany Germany 528 56
Chinese Taipei 557 63 Australia Australia 527 57
Hong Kong 554 62 Sweden Sweden 525 56
England Part of a Group of 8 Countries England Part of a Group of 8 Countries England Part of a Group of 8 Countries Netherlands Netherlands 523 56
Japan 548 60 Denmark Denmark 517 54
Russian Federation 546 60 New Zealand New Zealand 504 52
Latvia 542 60 Scotland Scotland 500 51
England 542 59 International average score International average score 500 49
United States 539 59 Norway Norway 477 47
Hungary 536 58
Italy 535 58
Kazakhstan 533 58
7
TIMSS 2007 RESULTS
Grade 8 Science (Y9)
50 Countries
4 Countries Outscored England 4 Countries Outscored England 4 Countries Outscored England England outscored 41 other countries including England outscored 41 other countries including England outscored 41 other countries including England outscored 41 other countries including
Score Correct Score Correct
Singapore 567 60 Russian Federation Russian Federation 530 52
Chinese Taipei 561 59 United Stages United Stages 520 49
Japan 554 57 Lithuania Lithuania 519 49
Korea 553 57 Australia Australia 515 48
Sweden Sweden 511 47
England Part of a Group of 5 Countries England Part of a Group of 5 Countries England Part of a Group of 5 Countries International average score International average score 500
England 541 54 Scotland Scotland 496 44
Hungary 539 53 Italy Italy 495 44
Czech Republic 539 53 Norway Norway 487 42
Slovenia 538 53 International average correct International average correct International average correct 40
Hong Kong 530 52
8
TIMSS 2007 RESULTS
Have we improved in science?
G4 science trends G4 science trends G4 science trends G8 science trends G8 science trends G8 science trends
Year Scale score Change Year Scale score Change
2007 542 No signif change 2007 541 No signif change
2003 540 ? from 1995 2003 544 No signif change
1999 538 No signif change
1995 528 Baseline 1995 533 Baseline
9
TIMSS 2007 RESULTS
Grade 4 Maths (Y5) 37
Countries
4 Countries Outscored England 4 Countries Outscored England 4 Countries Outscored England England outscored 28 other countries including England outscored 28 other countries including England outscored 28 other countries including England outscored 28 other countries including
Score Correct Score Correct
Hong Kong 607 77 United States United States 529 59
Singapore 599 74 Germany Germany 525 57
Chinese Taipei 576 69 Denmark Denmark 523 57
Japan 568 67 Australia Australia 516 55
Italy Italy 507 53
England Part of a Group of 5 Countries England Part of a Group of 5 Countries England Part of a Group of 5 Countries Sweden Sweden 503 51
Kazakhstan 549 64 International average score International average score 500
Russian Federation 544 62 Scotland Scotland 494 50
England 541 61 New Zealand New Zealand 492 49
Latvia 537 60 International average correct International average correct International average correct 48
Netherlands 535 59
10
TIMSS 2007 RESULTS
Grade 8 Maths (Y9) 50
Countries
5 Countries Outscored England 5 Countries Outscored England 5 Countries Outscored England England outscored 39 other countries including England outscored 39 other countries including England outscored 39 other countries including England outscored 39 other countries including
Score Correct Score Correct
Chinese Taipei 598 71 International average score International average score 500
Korea 597 71 Australia Australia 496 47
Singapore 593 70 Sweden Sweden 491 46
Hong Kong 572 66 Scotland Scotland 487 45
Japan 570 66 Italy Italy 480 43
England Part of a Group of 6 Countries England Part of a Group of 6 Countries England Part of a Group of 6 Countries Norway Norway 469 40
Hungary 517 53 International average correct International average correct International average correct 39
England 513 52
Russian Federation 512 51
United States 508 50
Lithuania 506 49
Czech Republic 504 49
LARGE GAP
11
TIMSS 2007 RESULTS
Have we improved in maths?
G4 maths trends G4 maths trends G4 maths trends G8 maths trends G8 maths trends G8 maths trends
Year Scale score Change Year Scale score Change
2007 541 ? from 2003 and 1995 2007 513 ? from 2003, 1999 and 1995
2003 531 ? from 1995 2003 498 No signif change
1999 496 No signif change
1995 484 Baseline 1995 498 Baseline
12
TIMSS 2007 RESULTS

• Have other countries improved?

13
TIMSS Trends 2003 - 2007
Country Country Grade 4 Maths Grade 4 Maths Grade 4 Science Grade 4 Science Grade 8 Maths Grade 8 Maths Grade 8 Science Grade 8 Science
Singapore Singapore ó ñ ò ó
Chinese Taipei Chinese Taipei ñ ñ ñ ò
Hong Kong Hong Kong ñ ñ ò ò
Japan Japan ñ ó ó ó
Korea Korea ñ ò
England England ñ ó ñ ó
United States United States ñ ó ó ó
Italy Italy ó ñ ó ó
Hungary Hungary ò ó ò ó
Netherlands Netherlands ó ó  
Sweden Sweden   ò ò
Norway Norway ñ ñ ñ ò
Russian Federation Russian Federation ó ñ ó ñ
Scotland Scotland ó ó ò ò
New Zealand New Zealand ó ò
Australia Australia ñ ó ó ò
Overall Up ñ 9 ñ 10 ñ 11 ñ 9
No change ó 10 ó 10 ó 12 ó 12
Down ò 2 ò 1 ò 10 ò 12
14
TIMSS 2007 RESULTS

• Other aspects of performance

15
TIMSS 2007 RESULTS

• International Benchmarks
• ? Increased since 1995

Benchmark Advanced (625) cumulative High (550) cumulative Intermediate (475) cumulative Low (400) cumulative
G4 science 14 48 ? 81 ? 95 ?
G4 maths 16 ? 48 ? Further increase from 2003 79 ? Further increase from 2003 94 ?
G8 science 17 48 79 94
G8 maths 8 35 ? 69 90
16
TIMSS 2007 RESULTS

• Content domains - science

Grade 4 science Grade 4 science Grade 4 science Grade 8 science Grade 8 science Grade 8 science
Physical science 543 ? Physics 545 ?
Earth science 538 ? Biology 541 ?
Life Science 532 ? Chemistry 534 ?
Earth science 529 ?
No significant differences between content domains No significant differences between content domains No significant differences between content domains No significant differences between content domains No significant differences between content domains No significant differences between content domains
indicates that the score difference is
significant ? indicates that the score is
significantly above the TIMSS scale average
More on this later
17
TIMSS 2007 RESULTS

• Cognitive domains - science

Grade 4 science Grade 4 science Grade 4 science Grade 8 science Grade 8 science Grade 8 science
Applying 543 ? Reasoning 547 ?
Reasoning 537 ? Knowing 538 ?
Knowing 536 ? Applying 530 ?
No significant differences between cognitive domains No significant differences between cognitive domains No significant differences between cognitive domains No significant differences between cognitive domains No significant differences between cognitive domains No significant differences between cognitive domains
indicates that the score difference is
significant ? indicates that the score is
significantly above the TIMSS scale average
More on this later
18
TIMSS 2007 RESULTS

• Content domains - maths

Grade 4 maths Grade 4 maths Grade 4 maths Grade 8 maths Grade 8 maths Grade 8 maths
Geometric shapes and measures 548 ? Data and chance 547 ?
Data display 547 ? Geometry 510 ?
Number 531 ? Number 510
Algebra 492
Weakest in number Weakest in number Weakest in number Best in data and chance Weaker in algebra (but n.s.) Best in data and chance Weaker in algebra (but n.s.) Best in data and chance Weaker in algebra (but n.s.)
indicates that the score difference is
significant ? indicates that the score is
significantly above the TIMSS scale average
More on this later
19
TIMSS 2007 RESULTS

• Cognitive domains - maths

Grade 4 maths Grade 4 maths Grade 4 maths Grade 8 maths Grade 8 maths Grade 8 maths
Reasoning 537 ? Reasoning 518 ?
Knowing 540 ? Knowing 514 ?
Applying 544 ? Applying 503
No significant differences between cognitive domains No significant differences between cognitive domains No significant differences between cognitive domains No significant differences between cognitive domains No significant differences between cognitive domains No significant differences between cognitive domains
indicates that the score difference is
significant ? indicates that the score is
significantly above the TIMSS scale average
More on this later
20
TIMSS 2007 RESULTS
Gender

• No overall gender differences at either grade in
  either subject
• Only 7 of 26 countries with results at both
  grades had no gender differences
• Minimal (ns?) differences in key stage tests and
  GCSEs
• Differences in PISA 2006 (boys overall and
  explaining phenomena scientifically)
• Some TIMSS domain differences

21
Gender domain differences
Grade 4 Difference Grade 8 Difference
Mathematics overall - Mathematics overall -
Number - Number Boys (16 scale points)
Algebra -
Geometric shapes and measures Girls (10 scale points) Geometry -
Data display - Data and chance -
Science overall - Science overall -
Life science Girls (7 scale points) Biology -
Physical science - Chemistry -
Physics Boys (15 scale points)
Earth science Boys (10 scale points) Earth science Boys (13 scale points)
22
Gender domain differences
Grade 4 Difference Grade 8 Difference
Mathematics overall - Mathematics overall -
Knowing - Knowing -
Applying - Applying -
Reasoning - Reasoning -
Science overall - Science overall -
Knowing - Knowing Boys (12 scale points)
Applying - Applying Boys (12 scale points)
Reasoning Girls (7 scale points) Reasoning -
23
TIMSS 2007 ATTITUDES

• Generally, half to two-thirds of pupils have high
  enjoyment and confidence in maths and science
• One exception only 40 enjoyed G8 maths
• Boys more confident than girls (both subjects,
  both grades), despite no overall differences in
  attainment
• Increase in high confidence in maths
• Drop in highly positive attitude, both subjects,
  both grades

24
TIMSS 2007 ATTITUDES

• No comparable attitude index in 2003

High positive category () High positive category ()
Positive attitude () Confidence ()
Grade 4 science 59 (-13 since 1995) 55
Grade 4 maths 62 (-14 since 1995) 64 (5 since 2003)
Grade 8 science 55 (-21 since 1999) 53
Grade 8 maths 40 (-25 since 1999) 53 (6 since 2003)
Low positive category () Low positive category ()
Less positive attitude () Confidence ()
24 (9 since 1995) 14
21 (7 since 1995) 10
25 (12 since 1999) 15
35 (19 since 1999) 15 (-4 since 2003)
25
TIMSS 2007 ATTITUDES

• Profile of high performance but relatively low
  enjoyment common in other high scoring countries
• Do attitudes matter?
• Confidence in maths related to attainment in both
  subjects, both grades
• Confidence and enjoyment in science related to
  attainment in science

26
TIMSS 2007

• VALUING LEARNING IN ENGLAND
• Grade 8 pupils value their learning, despite
  relative lack of enjoyment
• 74 highly value maths, 52 science
• Both increased since 2003
• Science 7 percentage points
• Maths 10 percentage points
• Does valuing learning matter?
• Some relationship with attainment
• Though not a significant factor in national
  analysis

27
TIMSS 2007

• VALUING LEARNING IN ENGLAND

Science - High value Science - High value Science - High value Medium value Medium value Medium value Low value Low value Low value
Mean score Difference from 2003 Mean score Difference from 2003 Mean score Difference from 2003
52 552 7 ? 31 542 -4 ? 17 515 -3 ?
Maths - High value Maths - High value Maths - High value Medium value Medium value Medium value Low value Low value Low value
Mean score Difference from 2003 Mean score Difference from 2003 Mean score Difference from 2003
74 515 10 ? 21 514 -6 ? 5 505 -3 ?
28
TIMSS 2007 TESTING

• G8 TEACHERS GIVING TESTS
• Pupils in England tested less frequently than
  pupils in other participating countries (teacher
  report)
• No change in frequency over time
• Tests and quizzes

Every 2 weeks or more () About once a month () A few times a year or less ()
Science in England 15 51 34
Science international average 34 42 24
Maths in England 9 38 53
Maths international average 46 39 16
29
TIMSS 2007 THE PUPILS

• HOMEWORK IN ENGLAND
• Pupils report less homework than pupils in other
  countries
• Teachers report less emphasis on homework than
  teachers in other countries
• Both grades and both subjects
• G8 - decrease in high emphasis on homework for
  science teachers (18 high emphasis, -11 points
  from 2003)
• No changes for grade 8 maths or at grade 4
• No clear international relationship between
  amount/ frequency of homework and attainment

30
TIMSS 2007

• International survey, tells us about nations not
  schools
• How could it help improve teaching and learning
  in your school or classroom?
• Linda Sturman, Graham Ruddock
• Department for Research in Assessment and
  Measurement, NFER
• The National Foundation for Educational Research,
• The Mere, Upton Park, Slough, SL1 2DQ, UK
• phone [phone] fax [phone]
  691632
• www.nfer.ac.uk/timss
• Linda Sturman 44 (0)1753 637144
  l.sturman_at_nfer.ac.uk
• Graham Ruddock 44 (0)1753 637140 g.ruddock_at_nfer.
  ac.uk