Assessing for managing and monitoring a national policy in France

1
Assessing for managing and monitoring a national
policyin France
2
What are we talking about ?

• Illiterate people international definitions "A
  person is functionally literate if he is able to
  engage in all those activities in which literacy
  is required for effective functioning of his
  group and community and also to enable him to
  continue to use reading, writing, and calculation
  for his own and the communitys
  developmentspecific definitions for test
  designs,
• Finally, for those who have just arrived in
  France and don't speak French, the challenge is
  to learn a new language. In France, this training
  is referred to as "French as a foreign language
  
• For people who never attended school, the
  specific French term "analphabétisme" is applied.
  These people require access to initial training.
  
• For people who attended school in France yet did
  not acquire a sufficient command of reading,
  writing and the basic skills to become
  self-sufficient in simple everyday situations, we
  use the word "illettrisme".

3
Latest trends in the national policy for fighting
illiteracy
Pop 18-65 y.o..
AGENCE NATIONALE DE LUTTE CONTRE LILLETTRISME
2
4
Representation of French situation by flews
5

• Make the fight against illiteracy
• a National Priority
• July 29 1998 The French law
• to Fight Exclusion officially promoted the fight
  against illettrisme as National priority
  
• Following this Law the Labour Code stated "the
  fight against illetrisme is part of lifelong
  education. The State, local governments, public
  institutions, private and public teaching
  establishments, associations, professional, union
  and family organizations, and companies each have
  their role to play. Initiatives to fight
  illetrisme are training initiatives.

6
APPELATION DU POWER-POINT
National Agency for Fighting Illiteracy 2000

• Following Geffroys report (1999) ANLCI was
  created in October 2000 to make the priority of
  fight against illettrisme effective, to federate
  and optimize the resources allocated by the
  state, local authorities and private companies to
  fight illettrisme illettrisme.
• A place for collaboration, a meeting place for
  all those working to help the fight against
  illettrisme move forward.
  
• ANLCI working motto is Unite to act more
  effectively at the national and the local
  levels.
  

AGENCE NATIONALE DE LUTTE CONTRE LILLETTRISME
2
7
Action of Anlci
To measure
To organise
To equip
8
POLICIES
Informal education
Formal education
Non-formal education
Prevention
Pop 18-65
PopGain of competences
Pop65
Facing difficulties
Facing difficulties
Illiterates
Young adults
At Workplace
In prisons
Cultural Environment
Literate Environment
9
Measuring illiteracy in France

• In the army, tests for incorporated men, until
  1998
  
• Since 1989, studies (about adults) based on self
  declarations (INSEE)
  
• In 1990/91, IEA study  How in the world students
  read ? , France, second on 34 countries,
  
• International survey IALS 1994 (France withdrew,
  prose literacy 43 of French people at level 1)
  
• Since 1998 during the JAPD (defense day),
  assessment of 17-19 yo youngsters (2000 for
  girls)
  
• 1998 France refused to join OECD survey ALL, a
  group of ministerial statistics departments
  started to work on alternative proposition for
  literacy surveys
• Since 2002 in prisons illiteracy tests

10
Measuring illiteracy in France

• Development in training organisations
  positioning tests, evaluation tests, following
  local or national frameworks
  
• 2000, PISA focusing on Literacy (France in
  average, 505)
  
• National IVQ pre-test in 2002 (INSEE/ANLCI), a
  sub sample is retested with IALS prose, 15 of
  french adults at level 1
  
• National IVQ survey in 2004/2005 (INSEE/ANLCI)
• Regional surveys from 2004 to 2008(INSEE/ANLCI)
• 2008 studies to connect IVQ to other assessment
  surveys, Internet assessment tool
  
• IVQ repeat 2009 PIAAC in 2012 ?

11
Measuring literacy in the World

• International adult literacy surveys
• IALS 1994-1998
• SACMEQ
• ALL 2003
• LAMP
• PIAAC in 2012 ?
• National adult literacy surveys, USA (NALS, NAAL
  2003), England (National survey of literacy and
  numeracy 2002, British Cohort Studies, China,
  Bostwana, France

12
Methodological choices

• Two types of surveys
• Two methods identification or measuring
• National or local studies / international
  studies
  
• Links, often a similar vocabulary, but different
  realities and priorities
  
• Two goals monitoring vs comparison
• Priority to the profile or to the measuring tool
• Priority to external validity or internal
  validity
  
• Different mathematical scales ordinal or by
  interval
  

13

• Mathematical scales ordinal or by interval
• Mathematical Critic
• The mean
• 1 2 3 4 5 6 7 8 9 10 11 12
  13 14 15 16 17 18 19 20

a
b
c
d
In the literacy competence field
(A D)/2 (B C)/2 ??????
B-A D-C ??
14
Mathematical critics

• The choice to set up thresholds defined by
  experts and practitioners a priori Top down
  strategy
  
• Unidimentionality is literacy unidimentional ?
  To have a set of items supporting
  unidimentionality you have to exclude items not
  correctly functioning after testing Bottom up
  strategy.
• PISA (meeting of NPM Oct. 1998 p13)  bottom up
  strategy is less recommendable than top down
  strategy 
  
• Thresholds strategy has been recommended in OECD
  meeting Feb. 2008 in Paris

AGENCE NATIONALE DE LUTTE CONTRE LILLETTRISME
2
15
Assessment of low levels of Literacy Evaluer les
bas niveaux de compétence à lécrit
EXPERIENCE FROM USA

• Many audiences for the assessment results
  sought to make standards-based inferences about
  NALS they wanted to know how many adults were
  illiterate and how many had the skills needed
  to function adequately in society. As we have
  discussed, however, the test development process
  used for NALS and repeated for NAAL was not
  intended to support such claims. An alternative
  approach to test development, similar to that
  used for certification and licensing tests, would
  allow such inferences to be made about the
  results. p184
• Measuring literacy performance levels for adults
  Committee on performance levels for adult
  literacy, National research council, National
  academies press, Washington, D.C. 2005

AGENCE NATIONALE DE LUTTE CONTRE LILLETTRISME
2
16
MODEVAL project

• Philosophy of the project
• Assessing above ranking Assessing to manage
• (at all levels)
• To assess in the same way in order to speak about
  the same thing and to understand each other,
  between peers within the different levels of
  intervention
• To assess and compare in order to improve, to
  challenge oneself (within their practices, within
  their orientation choices, etc.)
  
• To assess and compare in order to manage action
  policies at all levels.
  
• An example of work to be implemented the
  harmonisation of indicators concerning the fight
  against illiteracy in France.
  

17
Indicators in France
Survey about training ?
?
Census report
Census report
Pop 18-65
Pop
Pop 65
Facing difficulties
Illiterates
Facing difficulties
Loss of competences
According to IVQ
JAPD
?
IVQ
18
Assessment of low levels of Literacy Evaluer les
bas niveaux de compétence à lécrit
Building an indicator net
the choice of a reference

• IVQ
• A household survey, face to face, computer-
  assisted interview,
  
• 5 test modules testing oral comprehension,
  reading literacy, writing literacy, numeracy. 1
  biographical module
  
• A random sample of housing units, in all
  metropolitan regions,
  
• 10 086 respondents
• From October 2004 to January 2005 in metropolitan
  France
  
• 2006 in Martinique, 2007 in Ile de la Réunion,
  2008 in Guadeloupe
  
• 2009 IVQ repeat, used in local surveys

AGENCE NATIONALE DE LUTTE CONTRE LILLETTRISME
2
19
Assessment of low levels of Literacy Evaluer les
bas niveaux de compétence à lécrit
Building an indicator net
the work to do

• To carry on technical surveys to now in which
  extend existing data are comparable April 2008
  with Army (japd) and Justice (prisons)
  
• To implement Regional extensions of IVQ,
  Guadeloupe June 2008
  
• To repeat of IVQ national survey 2009, with
  repeat of regional extensions and new extensions
  
• To develop a tool for assessment through Internet
  compatible with IVQ, in progress, first tests en
  June
  
• To carry out new surveys in particular contexts
  (workplace, NGOs, local associations, ) with new
  tool. Already a lot of candidates.
  
• To centralise the data for building an Illiteracy
  data base and necessary to provide actor with
  information they need and open to researchers

AGENCE NATIONALE DE LUTTE CONTRE LILLETTRISME
2
20

• Thank you for your attention
• JP Jeantheau
• ANLCI 1 place de lécole BP 7082
• 69348 Lyon Cedex France
• Fax [phone] Tel [phone]
• jean-pierre.jeantheau_at_anlci.fr
• http//www.anlci.gouv.fr

21
(No Transcript)