A Systems Approach to Establishing Scientific Integrity in Evidence Based Policy Making

1
A Systems Approach to Establishing Scientific
Integrity in Evidence Based Policy Making

• Prof. Dr. Wijnand J. Swart
• Centre for Plant Health Management (CePHMa)
• Faculty of Natural and Agricultural Sciences
• University of the Free State
• Bloemfontein
• South Africa
• swartwj.sci_at_ufs.ac.za

2
EVIDENCE in Policy Context

• Sound, credible and robust evidence, whether
• quantitative
• qualitative
• statistical
• economic
• attitudinal / behavioural
• anecdotal
• social
• opinion based
• or review based .....
• ......is an essential and necessary part of
  the enabling environment for formulating policies
  that are coherent and effective in terms of their
  outcomes.

3
The Policy Making Process

• Techniques, analyses and judgements used to
  evaluate and formulate data and information into
  knowledge / evidence for making effective
  policies are critical.

4
What is Sound Evidence?

• Concept of sound and credible evidence is very
  complex.
• Dependant on inter alia
• types
• sources
• generating techniques
• context
• understanding
• ...... of data / information / knowledge

5
Sound Evidence Context Understanding
TRUTH?

• Data are facts (e.g. numbers, names, symbols) and
  have little value in themselves.
• Information relates to description, definition,
  or perspective (what, who, when, where).
• Knowledge comprises strategy, practice, method,
  or approach (how).
• Wisdom embodies principle, insight, morals, or
  archetypes (why).
• Absolute Truth ?

WISDOM
understanding principles
KNOWLEDGE
CONTEXT INDEPENDENCE
understanding patterns
INFORMATION
understanding relations
DATA
UNDERSTANDING
6
Knowledge vs Science

• Science is organized knowledge. Herbert Spencer
• Science a knowledge of principles and
  causes.(Webster's Revised Unabridged
  Dictionary)
• Science is a way of thinking much more than it is
  a body of facts. Carl Sagan
• Good Science could be defined as those
  practices which contribute most to advances in
  understanding.
• Sindermann The Joy of Science 1985.
• Science is to see what everyone else has seen but
  think what no one else has thought. Albert
  Szent-Gyorgyi

7
Science and Policy Making

• Creative and innovative contributions of
  scientists and policy makers and the trust
  engendered in them by the public, to whom they
  are accountable is of paramount importance.
• Follows therefore that fostering an environment
  that promotes research or scientific integrity is
  an integral part of that accountability and the
  pursuit of new knowledge.

8
Scientific Integrity

• Integrity by definition
• honesty
• a state of being entire or whole
• Perspectives of scientific integrity
• Ethical issues relating to misconduct (fraud)
  or manipulation, suppression, or distortion of
  facts.
• 2. Striving towards wholeness or excellence
  in the search for knowledge.

9
Research Integrity

• Essential for maintaining scientific excellence
  and for keeping the publics trust.
• Research integrity characterizes
• Institutional integrity
• creating an environment that promotes responsible
  conduct and high levels of integrity
• embracing standards of excellence,
  trustworthiness, and lawfulness
• Individual integrity Scientists commitment to
  intellectual honesty and personal responsibility
  and is an aspect of moral character and
  experience. A good scientist must
• communicate well
• obtain research grants
• excel in teaching and mentoring
• engage in ethical decision making
• use knowledge wisely to plan and execute
  research.

10
Institutional IntegrityPolitics vs Science

• Politicization of science as old as science
  itself e.g Galileo's theory that the Earth
  revolves around the sun perceived as a challenge
  to the authority of the Catholic church.
• Political interference threatens the integrity of
  government science and policy making all over the
  world.
• Manipulation, suppression, and distortion of
  government science misinforms public and leads to
  poor policy decisions.
• Especially rife in developing countries, e.g. the
  assertion of the South African president Thabo
  Mbeki that AIDS is not caused by HIV flew in the
  face of decades of research and threatened to
  undermine proper treatment of the disease.

11
Bush Administration's Misuse of Science

• On February 18, 2004, 62 pre-eminent scientists
  AND researchers released a statement titled
  Restoring Scientific Integrity in Policy Making
  in the USA.
• Scientists charged the Bush administration with
  widespread and unprecedented manipulation of the
  process through which science enters into its
  decisions.
• Scientists accused Bush administration of
• Epidemic altering and concealing of scientific
  information by senior officials in various
  federal agencies.
• Active censorship of scientific information that
  the administration considered threatening to its
  own philosophies
• Restriction of the ability of government-supported
  scientists to freely communicate scientific
  ideas related to "sensitive" issues .

12
Integrity of Research Institutions

• The organizational structure and processes that
  typify the mission and activities of a research
  institution can either promote or detract from
  the responsible conduct of research.
• These process are in part determined by the
  external environment and are influenced by the
  dynamics between and among organizational
  members.
• Any element or part of an organization can be
  viewed as a system in and of itself.
• External conditions influence the inputs into an
  organization, affect the reception of outputs
  from an organizations activities, and directly
  affect an organizations internal operations.

13

• Open systems model of internal environmental
  elements of a research organization showing
• Inputs / resources for organizational functions
• Structures and processes that define an
  organizations operation
• Outputs / outcomes of activities carried out by
  individual scientists, research groups or teams,
  and other research-related programs.

Source National Academy of Sciences -
http//www.nap.edu
14

• Interrelatedness between research organizations
  and the various external influences that have an
  impact on integrity in research.
• Systems and subsystems of the external-task
  environment are embedded within the general
  sociocultural, political, and economic
  environment.
• Relationships also exist between and among the
  elements within the external environment.

Source National Academy of Sciences -
http//www.nap.edu
15
Holistic Knowledge / Evidence

• Rather than focus on the ethical or moral aspects
  of scientific integrity, focus here is on the
  process of generating data and information and
  integrating it into sound knowledge (sound
  evidence) for decision-making.

• Integrity by definition
• honesty
• a state of being entire or whole
• Integrate by definition
• to combine parts into a whole
• A whole.. is more than the sum of its parts.

Jan Christian Smuts, Holism and Evolution 1926
16
Sound Evidence A Holistic View

• A collection of data is not information.
• A collection of information is not knowledge.
• A collection of knowledge is not wisdom.
• Information, knowledge, and wisdom are more than
  simply collections
• Each concept represents more than the sum of its
  parts and has a synergy of its own.

17
A WHOLE AS A SYSTEM

• A system is defined as a set of interacting
  units with relationships among them. The
  properties (or behaviour) of the system as a
  whole emerge out of the interaction of the
  components comprising the system.
• The interactions of the parts become more
  relevant to understanding the system than
  understanding the parts.
• This definition of a system implies something
  beyond cause and effect.

18
The Ultimate System

• In truth only one system, the Universe"
• All systems are sub-systems of a larger system.

?
universe
galaxy
solar system
world
nation
state
community
COMPLEXITY
person
organ
cell
molecule
atom
particle
NUMBER OF SUB-SYSTEMS
19
Systems Thinking and Policy

• Science is a way of thinking much more than it is
  a body of facts. Carl Sagan
• Systems thinking offers a conceptual framework
  or model for thinking differently.
• Systems thinking has permeated many scientific
  fields including education, business management,
  human development, sociology, psychology,
  agriculture, ecology and biology, earth sciences.

20
Hard vs Soft Systems

• Adopting a systemic perspective on solving policy
  problems therefore appear to offer a useful way
  of correcting these deficiencies.
• In 1960s, a hard (quantitative) systems
  approach was touted as the policy science.
• However, hopes not realized for variety of
  reasons its comprehensive modelling too
  information-intensive and mathematical.
• The soft (qualitative) systems approach of
  systems thinking has increasingly been used since
  the 1990s as a paradigm in policy planning and
  implementation.
• Soft systems methods stress the self-organizing
  and adaptive capacities of appropriately designed
  systems.

21
Hard vs Soft Systems

• Soft systems methods
• Subjective (interpretive) philosophy
• Systems sociological theory base
• Flexible methodology
• Organizational problem-solving focus
• Creative / intuitive
• Analyst is facilitator
• Participative
• Organizational learning outcomes
• Several ambiguous outcomes

• Hard systems methods
• Objective philosophy
• Computer science systems theory
• Rigid method
• Data, process, database technical focus
• Scientifically analytical
• Analyst is expert
• Analyst dominated
• Computer design outcomes
• One correct solution

22
Agro-ecosystem
A hard systems view of a farming system a
biological network suitable for mathematical
solution.
A soft systems view of a farming system an arena
for gaining experience and increased
understanding.
Source Robinson, B. 2003. 11th Australian
Agronomy Conference)
23
Soft Systems Methodology (SSM)

• Can be used both for general problem solving and
  in the management of change.
• Used in the analysis of complex situations where
  there are divergent views about the definition of
  the problem "soft problems" or policy options
  (e.g. How to improve health services delivery
  How to manage disaster planning).
• At the heart of SSM is a comparison between the
  world as it is, and some models of the world as
  it might be. 
• Out of this comparison comes a better
  understanding of the world ("research"), and some
  ideas for improvement ("action").

24
SSM for Problem Solving

• Classic form of SSM consists of seven steps
• Problem unstructured by researchers
• Problem situation expressed to capture rich
  picture
• Create root definitions of relevant systems (i.e.
  social, political environmental)
• Making and testing conceptual models based upon
  world views
• Comparing conceptual models with reality
• Identifying feasible and desirable changes
• Acting to improve the problem situation

reality
understanding and improvement
conceptual models

• Differences between models and reality become the
  basis for planning and policy making process.

25
Multi-agent systems (MAS)

• Policy increasingly has to address topics that
  have to do with disequilibrium, dynamics, and
  locality.
• The overwhelming complexity of biophysical and
  socio-economic constraints that increasingly
  characterize rural areas in developing countries
  necessitates the development of more
  sophisticated tools to support policy making in
  these areas.
• Multi-agent systems (MAS) are a relatively new
  field in computer science that have been proposed
  as a modelling approach for establishing even
  higher levels of scientific integrity in the
  generation and evaluation of evidence for making
  policies.
• Analogous to artificial intelligence.

26
MAS for Policy Making

• Multi-agent models might be the preferred choice
  when heterogeneity and interactions of agents and
  environments are significant and policy responses
  cannot be aggregated linearly.
• MAS can thus complement bio-economic simulation
  models which cannot fully capture the
  heterogeneity in biophysical and socio-economic
  constraints and the interactions between them.
• There are several policy questions in the context
  of agricultural development of rural areas where
  MAS simulations may generate useful information
  for decision making on public investments in RD
  and the targeting of policy interventions.
• Examples of such policy questions
• Should funds be spent on crop breeding for stress
  resistance or in research for improved crop
  management?
• Should micro-finance be promoted or should
  agricultural inputs be subsidized?

27
Agents

• An agent is anything that can be viewed as
  perceiving its environment through sensors and
  acting upon that environment through effectors
• Agents may be persons, farms, markets, computer
  programmes or anything that is reactive,
  autonomous, and goal-oriented.
• Agents may have the ability to communicate with
  other agents, learning, mobility, and
  flexibility. May even have personality and show
  emotions!

AGENT
SENSOR
EFFECTOR
OUTPUT
INPUT
SYSTEM
ENVIRONMENT
28
Agent Flexibility

• An intelligent agent is capable of flexible
  autonomous action.
• FLEXIBLE
• Reactive A reactive system is one that maintains
  an ongoing interaction with its environment, and
  responds to changes that occur in it (in time for
  the response to be useful)
• Pro-active Generating and attempting to achieve
  goals not driven solely by events taking the
  initiative i.e. goal directed behavior
  recognizing opportunities
• Social Ability to interact with other agents via
  some kind of agent-communication language, and
  perhaps even cooperate with others.

29
(No Transcript)
30
BIBLIOGRAPHY

• Balmann, A. 2000. Modeling land use with
  multi-agent systems perspectives for the
  analysis of agricultural policies.
• Berger, T. and Ringler, C. 2002. Trade-offs,
  efficiency gains and technical change Modeling
  water management and land use within a
  multiple-agent framework, Quarterly Journal of
  International Agriculture 41119144.
• Berger, T., Schreinemachers, p., and Woelcke, J.,
  2006. Multi-agent simulation for the targeting
  of development policies in less-favored areas.
  Agricultural Syatems 8828-43.
• Checkland, P.  1981 Systems thinking, systems
  practice.  Chichester Wiley. 
• Checkland, P., and Holwell, S. 1998  Information,
  systems, and information systems making sense of
  the field.  Chichester, UK Wiley.  
• Checkland, P.  and Scholes, J.  1991 Soft systems
  methodology in action.  Chichester Wiley. 
• Harrison MI. 1994. Diagnosing Organizations
  Methods, Models, and Processes, 2nd ed. Thousand
  Oaks, CA Sage.
• Schreinemachers, P. Berger, T. and Aune, J.B.,
  2007. Simulating soil fertility and poverty
  dynamics in Uganda A bio-economic multi-agent
  systems approach. Ecological Economics
  64387-401
• Union of Concerned Scientists. 2004. Scientific
  integrity in policy making An investigation into
  the Bush administration's misuse of science.
  Cambridge (Massachusetts) Union of Concerned
  Scientists 49 pp.
• Woolridge, M. 2002. An Introduction to Multiagent
  Systems by John Wiley Sons (Chichester,
  England).

31
THANK YOU!