The literacy divide: territorial differences in the Italian education system

1
Statistical Methods for the analysis of large
data-sets University G. d'Annunzio -
Chieti-Pescara September 23, 2009 September 25,
2009
The literacy divide territorial differences in
the Italian education system
Claudio QUINTANO, Rosalia CASTELLANO, Sergio
LONGOBARDI University of Naples
Parthenope claudio.quintano_at_uniparthenope.it
lia.castellano_at_uniparthenope.it
sergio.longobardi_at_uniparthenope.it
2
Overview
Italian data from the last PISA (Programme for
International Student Assessment) survey edition
(2006)
DATA

• Investigating the determinants of student
  achievement
• Highlighting the influence of the TEST-TAKING
• MOTIVATION on territorial differences

GOAL
A MULTILEVEL REGRESSION MODEL is applied This
approach is suggested by the hierarchical
structure of the PISA data where students
(level-one units) are nested in schools
(level-two units)
METHOD
3
The OECDs PISA Programme for International
Student Assessment survey is an internationally
standardised assessment administered to 15 years
old students
PISA 2006
57 Countries
The survey has involved
400.000 students (21.773 in Italy)
14.300 schools (806 in Italy)
4
PISA 2006
Reading literacy
The survey assesses the students competencies in
three areas
COGNITIVE TEST
Mathematical literacy
Scientific literacy
FAMILY ENVIRONMENT OF STUDENT
STUDENT QUESTIONNAIRE
The OECD collects data on
SCHOOL QUESTIONNAIRE
SCHOOL CHARACTERISTICS
5
International ranking -Reading literacy-
Italy ranked 33th in reading among 57 countries
ITALY 469 points OECD average 492 points
6
International ranking -Mathematical literacy-
Italy ranked 38th in mathematics among 57
countries
ITALY462 points OECD average 498 points
7
International ranking -Scientific literacy-
Italy ranked 31th in Science among 57 countries
ITALY475 points OECD average 500 points
8
The literacy divide
MATHEMATICS
READING
SCIENCE
9
Determinants of learning outcomes
Many studies (Marks, 2006 Korupp et al., 2002)
emphasize the role of socio-economic background
for determining learning outcomes and explaining
the territorial differences. This work aims to
asses how much the DIFFERENCES IN THE TEST-TAKING
MOTIVATION boost the effect of socio economic
background on the Italian literacy divide
10
Low stake test
The PISA test is considered as a low stake test
since the students perceive an absence of
personal consequences associated with their test
performance. Without an adequate effort, test
performance is likely to suffer, resulting in the
examinees test score underestimating his or her
actual level of proficiency (Wise and De Mars,
2005 Wolf Smith, 1995 Wolf, Smith,
Birnbaum, 1995)
11
Index of student effort
An index of student effort is computed on the
basis of three variables

1. the Test non-response rate computed on the
   basis of the number of missing or invalid answers
   in the PISA cognitive test

2. the Questionnaire non-response rate
computed on the basis of the number of missing or
invalid answers in the PISA student questionnaire
(family environment data)
3. the Studentsself-report effort in the PISA
questionnaire measured on a 10-point scale
12
Effort and performance
Variation of the index of student effort in
correspondence of the average performance at
macro region level. Italy100
The correlation coefficient between this index
and the science performance is equal to 0,553 at
national level
13
Multilevel approach
A two-level random intercept regression model is
adopted
Variables at school level
Variables at student level
Outcome of ith student of jth school (Science
plausible values)
Error components
eij IID-N(0, s2) U0j IID-N(0,
t2) cov(U0j, eij) 0
14
Causal structure of multilevel model
Scholastic context -School mean of
ESCS -Parentspressure -Private vs public
LEVEL 2 School j
Study programme Technical, Professional,
Vocational, Lower secondary vs Classical studies
Macro area North East, North West, South, South
and Islands vs Centre
Scholastic resources -Computers with web -Quality
of educational resources -Teacher shoratge
Immigration background 1. or 2. generation vs
native
Student performance (Science Test score)
Gender Girls vs boys
LEVEL 1 Student i within School j
Home educational resources
Self confidence in ICT
Hours per week spent on homework
15
Estimation strategy
A block entry approach is adopted (Choen and
Choen, 1983) which consists to the gradual
addition of the first and second level
covariates The process starts with the simplest
model, denoted the empty model, and then
progressively adds complexity introducing school
and student variables In the last model the index
of student effort is considered with 8 school
variables and 5 students variables
16
Block Entry Approach
Empty model
Student variables
Scholastic context
Study programme
School location
Scholastic resources
Student effort
17
Pre-R2
Proportion reduction in variance or variance
explained (PRE-R2)
Variance component EMPTY MODEL Full model without effort index Full model with effort index
Variance between schools 5,347.98 1,025.56 589.76
Variance within schools 4,674.31 4,470.30 4,474.87
Variance expalined at school level - 80,82 88,97
18
Effect of the student variables
19
Effect of the school variables
20
Main findings

• The index of student effort allows to explain a
  larger amount of variance, indeed after
  controlling for students motivation the accounted
  total variance total variance among schools
  increases from 80 to 89
• The analysis confirms that the socio-economic
  context plays an important role on the student
  achievement
• The North-South divide has been overestimated by
  the PISA test since the score differences are
  also influenced by the lower effort and
  engagement of the Southern students

21
Territorial dummies
With effort index-25.46
With effort index0.82
With effort index-48.47
With effort index-47.50
22
Collaboration indexes
23
Transformation steps
COLLABORATION INDEX
1)
(growing when the non-response rate declines)
Rescaling procedure in order to obtain a common
scale (0-1) for each indicator
2)