The learning outcomes of an online reflective journal in engineering

1
The learning outcomes of an online reflective
journal in engineering

• Stuart Palmer, Dale Holt and Sharyn Bray
• Institute of Teaching and Learning
• Deakin University

2
Introduction

• This presentation
• explores the value of reflective journaling in
  engineering education
• presents a case study of the introduction of an
  online reflective journal
• summarises the results of the students initial
  perceptions of the online reflective journal,
• investigates the contribution of the online
  reflective journal to unit learning outcomes

3
Reflection and reflective journals

• In the context of learning, reflection refers to
  the active intellectual monitoring and evaluation
  of ones own formal learning and professional
  practice activities, to examine them for new
  understandings that add to the individuals
  accumulated knowledge and experience
• Reflective thinking based on experiential
  learning is a key skill required for the lifelong
  learner and the socially mature professional

4
Reflection and reflective journals

• The use of a reflective learning journal (due to
  the requirement to transfer thought processes
  into words) is thought to be a valuable tool in
  developing self- and critical reflection in
  students
• For the practicing professional, the use of a
  work journal offers additional benefits

5
Source http//legal.european-patent-office.org/dg
3/biblio/t930039ex1.htm
6

• Ex-Air Force Official Gets Prison Time
• Boeing Received Special Treatment in Procurement
• By Renae Merle and Jerry Markon
• Washington Post Staff Writers
• Saturday, October 2, 2004 Page A01
• A former high-ranking Air Force procurement
  official was sentenced to nine months in federal
  prison yesterday after admitting that she
  approved excessive prices on contracts awarded to
  Boeing Co. to enhance her job prospects with the
  company.
• Conceding that she lied to prosecutors, Darleen
  A. Druyun, 56, revealed that she committed the
  Air Force to buy 100 airplanes from Boeing at an
  inflated price of about 20 billion as a "parting
  gift" before her Pentagon retirement to
  ingratiate herself with her future employer. She
  also slipped to Boeing proprietary pricing
  information from a rival European bidder on the
  aircraft contract. Druyun awarded Boeing an
  unrelated 4 billion contract because she felt in
  debt to the company for hiring her daughter and
  future son-in-law, according to court documents.
  An "objective selection" process, she said, may
  not have picked Boeing from the four competitors.
• In a quivering voice, Druyun apologized before
  U.S. District Judge T.S. Ellis III in Alexandria,
  telling the court she felt "shame and remorse"
  that her 30-year tenure as a government employee
  "has been tarnished. . . . I understand that this
  was wrong and I regret any damage my conduct may
  have caused to the Air Force."
• Druyun's case is the highest-profile defense
  procurement scandal since the Operation Ill Wind
  investigation, which resulted in more than 60
  convictions starting in the late 1980s. It is
  expected to ripple throughout the industry,
  renewing concern about the potential pitfalls of
  the revolving door between government and the
  defense industry. Chicago-based Boeing, the
  Pentagon's second-largest contractor, will likely
  face fresh questions about several of its
  contracts, and the procurement system that
  allowed Druyun to favor one company over another
  will come under sharper scrutiny.
• Druyun, a civilian, was at the grade of a
  lieutenant general when she retired and became
  vice president in charge of Boeing's missile
  defense systems in January 2003. Druyun's Boeing
  salary -- 250,000 plus a 50,000 signing bonus
  -- was nearly double the top Pentagon pay for her
  position.
• Officials said Druyun admitted the extent of her
  deceptions only after being subjected to a
  polygraph test.
• She acknowledged altering her personal journal
  before turning it over to prosecutors.

Source http//www.washingtonpost.com/ac2/wp-dyn/A
64968-2004Oct1
7
Reflection and reflective journals

• It is recognised that reflection is not only an
  individual activity, but may contain
  social/learner-learner aspects as well
• It is reported that collaborative reflective
  activity and the ability to compare ones own
  thinking with that of other learners yields
  positive results and better facilitated learning
  than individual reflection

8
The context

• SEB421 Strategic Issues in Engineering
• This unit consisted of three modules
• Technological Forecasting and Assessment
• Policy Design in Engineering Organisations
• Issues in Productivity Improvement.

9
SEB421 student locations
Source http//commons.wikimedia.org/wiki/ImageWe
lt_Mercator_Atlantik.png
10
A reflective journal exercise

• At the completion of the weekly class, students
  were asked to respond in writing in their journal
  to the following two questions
• What did I learn today?, and
• How will this be of use to me in the future?
• Off-campus students completed their journal
  across the semester and submitted their written
  journal at the end of the semester

11
A reflective journal exercise

• Issues
• On-campus The journal took up a significant
  amount of class time
• Off-campus Some journals were obviously produced
  at the end of semester, rather than across the
  semester

12
An online reflective journal

• Students were asked to make their weekly journal
  entry in an online discussion area
• A separate discussion sub-area was created for
  each nominal class week to provide some structure
  and direction to students, and to break the large
  number of student postings into manageable
  sections

13
An online reflective journal - issues

• While it is recognised that criterion referenced
  marking of student reflective journals may
  potentially constrain student expression
• It can also encourage greater student engagement
  with the intended pedagogical aims of the journal

14
An online reflective journal

• The online nature of the system meant that all
  students could post their journal entries weekly,
  regardless of their study location or mode of
  study
• Because the journal entries were posted to a
  discussion area with open read access to all
  class members, all journal postings were
  potentially available to all students to read

15
Evaluation of initial trial

• Questionnaire
• demographic information age gender course of
  study location of study
• reflective journal did you understand its
  purpose? rate its value
• use of journal frequency of access did you
  read submissions of other students?
• online system rate the systems ease of use
• general what aspects of the online reflective
  journal were most useful? least useful?

16
Response rate and demographics
- International exchange students and
non-engineering students enrolled in a single
unit only
17
Value and use of reflective journal
18
Most useful aspects of journal
19
Least useful aspects of journal
20
Contribution to learning outcomes

• gender
• age
• mode of study (on- or off-campus)
• weighted average mark (WAM a proxy for
  general prior academic ability)
• semester of offer cohort grouping
• number of reflective journal entries posted (a
  proxy for quantity of engagement)
• average mark per journal entry (a proxy for
  quality of engagement)
• number of reflective journal entries read and
• final unit mark (out of 100 a proxy for unit
  learning outcome)

21
Pooled data characteristics summary
22
Regression model for final unit mark
ANOVA test F386 144.18, p lt 210-62 R2
0.53 Std Err of model 8.79
23
Contribution to learning outcomes

• prior academic performance was an important
  predictor of final unit mark
• the number of journal posts that a student made
  was an important predictor, and this was
  separable from WAM, and contributed at somewhat
  above the face value of up to 1 mark per
  posting
• even though mode was not significantly
  correlated to final unit mark on a direct
  pair-wise basis, after the effects of WAM and
  number of posts, mode became significant in
  accounting for some of the remaining variation in
  final unit mark, this being a negative influence
  for off-campus students

24
Conclusions / take-home messages

• It appears that it is the activity of formally
  reflecting on learning that makes a contribution
  to student learning outcomes,
• rather than the act of simply reading the posts
  of other students

25
Conclusions / take-home messages

• Although, student prior academic performance was
  a significant predictor of unit learning
  outcomes, the additional contribution made by
  each new reflective journal posting suggests that
  introducing an element of compulsion for
  students to make journal postings, such as
  through attaching marks to journal postings,
  would be beneficial

26

• Thank you for your time