Researching and Evaluating innovations in schools and systems with a special focus on the use of information technology

1
Researching and Evaluating innovations in schools
and systems with a special focus on the use of
information technology

• John Schostak
• Institute of Education
• Manchester Metropolitan University
• j.schostak_at_mmu.ac.uk
• This powerpoint may be found at
• http//www.enquirylearning.net/ELU/Issues/Informat
  ion20Technology/taiwan.ppt.htm

2
Purpose

• To describe the innovation/change problematic
  with reference to three case studies giving
  insights in
• The micro-level of action
• The system level of a nation
• The global-local level of distributed
  communities
• To provide insights into the process of
  researching and evaluating educational contexts

3
Introduction A first lesson in what not to do.

• 1981-1984 - computers in schools. Learning the
  need
• for all decision makers to have a broad
  philosophical or conceptual understanding of the
  purpose of an innovation
• to understand the nature of the social system or
  organisation in which the innovation is to be
  implemented
• for dialogue with practitioners
• for professional development appropriate to the
  innovation
• for appropriate resourcing

4
The Innovation problematic

• Open systems
• complexity and uncertainty produces the need for
  political and ethical strategies
• Often thought of as practical realities
  requiring qualitative research designs

• Closed systems
• the desire for mastery or control
• Often thought of as an ideal and leading to the
  gold standard of RCTs or, of audit trails

versus
In the realities of day to day practise, research
and evaluation focuses on the tension between the
closed and the open and the points of conflict
and struggle that are produced in essentially
complex, uncertain, frameworks of social
interaction.
5
The ideal structure (closed)
From Schostak, J.F. (2002) Understanding,
Designing and Conducting Qualitative Research in
Education. Framing the Project. Open University
Press
6
The real structure a stealth architecture
Competing interest groups
Alternative practices
Different resourcing
Unpredictable outcomes
From Schostak, J.F. (2002) Understanding,
Designing and Conducting Qualitative Research in
Education. Framing the Project. Open University
Press
7
3 questions

1. Micro-level (school) how to get people involved,
   empowered and so generate and embed cultural and
   organisational change?
2. Macro-level (State/system) how to set the
   agenda, provide the vision and catalyse change
   across a system?
3. Local-global level (on-line, distributed
   communities) how to reconceptualise educational
   practices in communities that are an interface
   between real (local) and virtual (global,
   distributed) spaces or places?

8
http//www.enquirylearning.net/ELU/Issues/Educatio
n/archivesEarlyyears.html Early years talking
and listening project 1988-9

• Expression by each teacher of their key
  educational values and philosophies
• Whole school (teachers, teaching assistants,
  kitchen staff, caretaker, pupils) approach to
  reflecting on and researching practice - broadly
  Action Research employed as a mechanism/procedure
  to formulate and guide change
• Material architecture and equipment of the school
  reconceptualised in terms of how it could support
  the new evolving practices
• Outcomes
• Culture and ethos of the school changed
• Innovation embedded in new professional practices
• School radically revitalised

9
Linking aims, approach and evidence

• To encourage and develop children's
  responsibility for their own actions and
  behaviour. How? The procedure involved
• recognition of problem/dispute
• making the problem/dispute explicit
• individuals taking responsibility for the
  solution to their own problems/disputes - an
  adult may help to ensure 'turn taking' so that
  all views are heard
• generating possible solutions
• agreeing a solution
• ensuring no one is upset by a solution
• keeping to the solution unless a further problem
  arises
• Evidence through Action Research, videos,
  observations, interviews were carried out,
  analysed and used as a basis for policy
  development as well as professional development

10
Evaluation of Finnish National Curriculum 1995

• A senior Education Board official
• when trying to define the inspiration for the
  reforms pointed not to key facts and practices
  but to key values which were, for him, summed up
  as Truth, Beauty, Goodness.
• (fieldnotes 14/2/95)
• The Framework Curriculum for the Comprehensive
  School 1994, stated all human solutions are
  connected with values and that the balanced
  development of physical, psychic and social
  resources makes it necessary for us to bring up
  questions which have to do with our health and
  well-being (p.10).

11
For many but not all schools

• As pointed out in our evaluation report
• The head teacher of a large project school
  explained to us that they had gone through the
  two year process, had found it tiring but useful,
  but now that the project was finished, they had
  returned to their traditional methods as these
  were more efficient and easier to manage.

12
CIEL Project 2003-4 the emergent global-local
community

• Collaboration In E-Learning (CIEL)
• DfES funded Forum Trust, 200,000
• 5 case studies of the challenges involved in
  sharing practice across organisations
• one of these, a study of the use of a Tiki in a
  primary school. A Tiki is an open source content
  management system ideal for creative
  collaborations
• The case became an example of the material
  environment problematising both philosophy and
  practice

13
The on-line environment
14

• Some Early Implications
• The new environment requires different
  pedagogical and learning strategies to those of
  the traditional classroom because
• Hierarchical structures are replaced by
  horizontal structures
• No central control
• Order emerges rather than imposed
• Children/learners are active agents, decision
  makers
• The boundaries between classroom, home and world
  dissolve the new environment is anywhere,
  anyplace, anytime
• New communities of participation and interest
  emerge parents, teachers, researchers, policy
  makers, creative practitioners . etc
• Does this imply the need for micro and macro
  innovations like those implemented by the
  previous two case studies?

15
Back to the future

• Conceptual changes are required hence, the
  re-emergence, by stealth, of child centred,
  progressive education under the name of
  personalisation
• New practices are required hence, breaking
  into the curriculum or hacking as a means of
  developing the kinds of communities of practice
  or intelligence communities implicit in the
  e-Park concept

16
E-Park - the infrastructure
17
Three conclusions for research and evaluation

• For effective change and desired outcomes
  identify and ensure coherence of the dynamic
  relations between
• The conceptual frameworks held by competing
  communities of interest
• The alternative practices favoured by each
  community of interest
• The organisation of material resources

18

• Decision making under conditions of complexity
  and uncertainty will involve all actors in
• Political decisions, i.e. any decision that
  advantages one group will disfavour or threaten
  others
• Ethical decisions concerning justice, proper
  behaviour, and the good society - different
  communities have different traditions, values,
  belief systems
• Hence formative structures for the negotiation
  of values and the agreeing of decisions are
  required

19

• To deal with the complexity and uncertainty
  during change, research and evaluation must be
  integrated at every level to
• inform the decision making of all interested
  parties by identifying problems and opportunities
• identify the range of alternative rationales for
  action and the deployment of resources
• explore the consequences of choosing one course
  of action rather than another
• develop approaches to professional development
• embed cultural and institutional change
  throughout systems
• Hence, research and evaluation are not
  peripheral to decision making and the
  implementation of innovations. They are
  fundamental to their success

20

• References
• Archer, M., Bhaskar, R., Collier, A., Lawson, T.,
  and Norrie, N., (eds) (1998) Critical Realism.
  Essential Readings, London and New York
  Routledge
• Bhaskar R. (1975 and 1978) A realist theory of
  science, Hassocks, Sussex Harvester Press
• Dewey, J. (1938) Experience and Education, New
  York, Collier
• Freire, P. (1973) Education The Practice of
  Freedom, London, Writers and Readers Publishing
  Cooperative.
• Hargreaves, D. H. (2003) Education Epidemic
  Transforming Secondary Schools through Innovation
  Networks, London, Demos
• Holt, J. (1969) How Children Fail, Harmondsworth,
  Penguin
• Illich, I. (1971) Deschooling Society, London
  Calder and Boyers
• Norris, N., Aspland, R., McDonald, B., Schostak,
  J.F. Zamorski, B. (1995) An Independent
  Evaluation of Comprehensive Curriculum Reform in
  Finland, CARE, UEA Finnish Board of Education
• Pawson, R., and Tilley, N. (1997) Realistic
  Evaluation, London Thousand Oaks, California
  Sage
• Rogers, C. (1969, 1983) Freedom to Learn, revised
  as Freedom to Learn for the 80s, Columbus,
  London Merrill
• Sayer, A. (1993) Method in Social Science. A
  Realist Approach, London, New York Routledge
• Schostak, J. F. (ed.) (1988) Breaking into the
  Curriculum the impact of information technology
  on schooling, London, New York. Methuen
• Schostak, J.F. (2002) Understanding, Designing
  and Conducting Qualitative Research in Education.
  Framing the Project. Open University Press
• Schostak, J. F. (2002) 'The gobalisation of
  Education - what are the challenges?',
  Information Technology Strategy Committee Seminar
  Series, Hong Kong Institute of Education
• Smith, J., Schostak, J., Phillips, E., Hough, M.,
  Fleet, K., Davies, B., and Brewer, L. (2004)
  Collaboration in E-Learning (CIEL). A report for
  the Department of Education and Skills, Norwich,
  Forum Trust
• Stenhouse, L. (1975) An Introduction to
  Curriculum Research and Development, London
  Heinemann