Computing Education Research

1
Computing Education Research

• Anders Berglund
• Informationsteknologi
• Uppsala universitet
• Uppsala
• anders.berglund_at_it.uu.se

2
Computing education research at Uppsala University

• How do our students understand computer science
  concepts?
• How to teach computer science?

3
Computing Education Research

• This talk
• What can we say about how grading in an
  internationally distributed project-based course?
• and
• Why and how to do research in computing
  education?

My secret agenda Inspire to Computing Education
Research
Two parallel stories
4
Computing Education Research

• Background
• A research approach?
• Phenomenography
• Results concerning grading
• The setting
• The teachers grades
• The students peer evaluation
• The experience of being graded
• Results concerning grading
• Summary
• Computing Education Research
• Some research results
• Choosing a research approach

5
What does it mean to learn something?

• Unfortunately (?).
• General case A meaningless question
• It all depends on what you mean by
  learning or how you see things

6
Quantitative/Qualitative research

• Quantitative research is grounded on
• the assumption that features of the social
  environment constitute an objective reality
  collecting numerical data on observable
  variables
• Qualitative research is grounded on
• the assumption that individuals construct a
  social reality in the form of meanings and
  interpretations. studying intensively in
  natural settings
• Implications for the role of the researcher, the
  concept of evidence, interpretation etc.
• (Gall, Borg Gall, 1996)

7
A research approach/methodology/ framework

• Offers a way to perform research in learning
• Organizes ways to see things
• A lens with a certain focus
• With a specific research approach Some
  issues get clearer, others blurred
• Offers theoretical stand on learning, ways to see
  possibilities and limitations, opens to
  communicate with other researchers etc.

8
Research approach/methodology/framework
Phenomenography (Marton Booth, 1997)

• Takes the learners perspective.
• Aims at analysing and describing the variation in
  students experience (understanding, learning).
• A empirical, qualitative research approach
• Data is often collected through interviews
• Outcome A few qualitatively different ways, in
  which something is understood within a student
  cohort

Examples TCP, experience of being graded
9
Phenomenography
The students study TCP
The researcher studies the different ways in
which the students understand TCP
Researcher
10
Why the learners perspective?

• A teachers understanding of how her students
  understand and learn about something (CS
  concepts, for example TCP) is a good tool for
  improving teaching.
• A change that is not perceived as good by the
  students does not improve learning.
• Example Grades are not the driving force for
  most students who take (a certain) project
  course.

11
Computing Education Research

• Background
• A research approach?
• Phenomenography
• Results concerning grading
• The setting
• The teachers grades
• The students peer evaluation
• The experience of being graded
• Results concerning grading
• Summary
• Computing Education Research
• Some research results
• Choosing a research approach

12
Background The Runestone initiativeA project
course in computer systems

• A project-based course in distributed systems,
  real-time programming and computer communication.
• Third/fourth year students majoring in CS.
• International collaboration for students who do
  not go on exchanges.
• Experience of collaboration over ICT tools.

13
Project course in computer systemsThe Runestone
project
USA Sweden
Communication by e-mail and chat

• 3 3 advanced CS students per team
• 16 teams in total
• No lectures
• Tutoring by e-mail and chat

14
Student project

• Student project Produce a software system to
  control a (modified) Brio labyrinth from any
  Web-browser.
• The task demands computer communication
  solutions.
• The task requires collaboration within the team
  of 6.

15
Grading in Runestone

• Both the process and the project are graded, in
  relation to the teams own plan
• Process grade is based on weekly meetings
• Components of grade
• Team performance, in relation to the teams own
  plan
• Individual contribution
• Peer evaluation
• The instructors decision.
• Team members are graded by their instructor
• Different grading schemes in Sweden and US
• Sweden pass/fail
• US A to E

Problematic???
16
Analysing the grading in Runestone

• Teachers distribution of grades
• (quantitative)
• Peer evaluation The students evaluation of
  each others contributions
• (quantitative)
• Students experienced purpose of being graded
  (qualitative)

17
Grades

• Grades awarded by the instructors, according to
  the Runestone scheme (Max 100, Pass 60)

18
Peer evaluation

• Each student awarded USD 120.- to his team-mates

Then, what is the driving force?
19
The experienced purpose of being graded
20
Results on grading

• Getting a good grade is not the driving force for
  most students in this project.
• Me in the team or My team in front of other
  teams is often important.
• How generalizable are these results?
• Research in computing education research are
  normally situationally bound
• How can we use this in our teaching?
• Both hard results and insights gained by doing
  the research are useful.

21
Computing Education Research

• Background
• A research approach?
• Phenomenography
• Results concerning grading
• The setting
• The teachers grades
• The students peer evaluation
• The experience of being graded
• Results concerning grading
• Summary
• Computing Education Research
• Some research results
• Choosing a research approach

22
On Computing Education Research

• Theoretically sound research in students
  learning in Computing can serve to improve
  teaching.
• Different research approaches offer various
  contributions to our understanding of students
  learning.
• The perspective on reality, what can be
  studied, what can be known, what the researchers
  role is, how research is performed etc. varies.

This talk Some examples of qualitative research
23
Example, research approach Constructivism

• A family of traditions
• Jean Piaget, 1896 - 1980
• Reality is rejected or irrelevant
• Knowledge is constructed by each individual
• No firm methodology
• Passive learning will fail
• Extremely influential in school teaching

24
Empirical results from constructivism in CS
Education

• Students construct rules for parameters. They are
  only sometimes successful. (Fleury, 1991)
• Students construct their own understanding of
  variables. (Paz, 1996 and others)
• Software visualization in itself does not help
  students understanding (Mulholland, 1997)

25
Applications of constructivism in Computer
Science Education

• Think twice when using visualizations
• Explicitly teach the model of the computer
• Dont start with abstractions
• Teach planning, teach to avoid bricolage
• Dont run to the computer
• Organize closed labs
• (from Ben-Ari, 2001)

26
Example, research approach A socio-cultural
perspective

• Thinking/Learning is not influenced by the
  environment
• ?
• Thinking/Learning is an interaction between the
  individual and the environment
• A family of traditions in research into learning
• Common source of inspiration Lev Vygotsky
  (1896 1934)

27
Applications of the soci-cultural tradition

• Why do teams of students interpret a programming
  task so differently? (Holland Reeves, 1996)
• Why do our students hand in incorrect programs?
  (Ben-David Kolikant, 2005)
• The example Open source community -
• Linux

28
Example, research approach Critical enquiry

• Critical enquiry (research with a mission,
  often to address power imbalances)
• Feminist research
• Cannot be defined in ontological stand or
  research methodology
• Instead, it questions fundamental principals and
  values and aims to implement changes.

29
Feminist research, an example of critical research

• Are there factors within computer science itself,
  that preserves the currently dominating gender
  structure? (Björkman Trojer, 2002)
• We consider it of vital and decisive
  importance that gender research is done from
  within computer science.
• Such research on the core of computer science
  and its knowledge production would serve to
  enrich computer science as well as education
  within computer science.

30
Why phenomenography? A personal
view
Selecting a research approach

• Appropriate for the research questions
• Complex answers desirable
• Statistical answers hard or impossible to get
• Close to learners
• Students perspective
• A way for students to talk to
  teachers/organisers
• Data stems from individuals
• Computer science is in focus
• Results talk to computer scientists
• Supports deployment of results in teaching
• Relevant for recognition of CSE research within
  CS
• Competence at hand/tradition
• Shirley Booth
• I like it ?

Create a network!
Read!
31
Literature
Read!

• Clancy, M., Stasko, J., Guzdial, M., Fincher, S.,
  Dale, N. (2001). Models and Areas for CS
  Education Research. Computer Science Education,
  11(4), 323-341
• Fincher, S., Petre, M. (2004) Computer Science
  Education Research, London, UK Taylor Francis
• Berglund, A., Daniels, M. and Pears, A. (in
  press). Qualitative Research Projects in
  Computing Education Research An Overview. To
  appear in the Proceedings of the 8th Australasian
  Computer Science Education Conference, Hobart,
  Australia.
• Berglund, A. 2005. Learning computer systems in a
  distributed project course The what, why, how
  and where. Acta Upsaliensis Universitatis.
  Uppsala Dissertations from the Faculty of Science
  and Technology 62

32
CeTUSS
Create a network!

• CeTUSS ska verka för anpassning av
  teknikutbildning till samhällets och studenternas
  behov.
• Genom att utveckla och sprida information om
  lärandemiljöer som är
• Personligt meningsfulla
• Socialt relevanta
• Gränsöverskridande
• Baserad på samarbete (lokalt och internationellt)
• Workshops, kurser etc.

33
(No Transcript)