Advice Giving and Advice Seeking

1
Advice Giving and Advice Seeking

• Janet A. Sniezek
• Department of Psychology and Beckman Institute
  for Advanced Science and Technology
• University of Illinois at Urbana-Champaign
• Visiting Scholar, Department of Psychology and
  Graduate School of Business
• Stanford University
• Alyssa Mitchell, Reeshad Dalal, Marcus Crede
• Department of Psychology
• University of Illinois at Urbana-Champaign

2
Decision Research Groupfor work on ONR projects

• Gunnar Schrah
• Patrick Wadlington
• Sasha Chernyshenko

3
Overview

• Decision Making as a cooperative joint activity
• Judge Advisor Systems Theory Research
• Advising Control Who initiates advising?
• Advice giving vs. Advice seeking
• Experimental results
• Communication variables
• Implications for intelligent tutoring with
  natural language dialogue

4
Tutoring Interactions

• How should interaction between the intelligent
  tutoring system and the trainee be structured?
• Who initiates interaction?
• When does interaction occur?
• What takes place during the interaction?
• etc.

5
Tutoring Interactions

• Who controls the interaction?
• Can the student ask for help?
• When does a student seek or not seek help?
• Does the tutor know when the student wants help?
  
• How does the tutor decide to intervene?

6
The Damage Control Environment

• Complex Uncertain Dynamic Reactive

• Perfect performance impossible
• No single correct action
• best answer
• multiple acceptable actions
• experts disagree
• Best action changes over time
• Best action depends on prior actions

7
Theory Research

• Damage control training (Liz will show)
• cooperative joint activities in dyads
• advice giving and advice seeking
• decision making in probabilistic environment
• Judge Advisor Systems Paradigm
• Judge Student
• Advisor Tutor
• Who controls advising interaction?

8
Judge Advisor Systems
(Sniezek Buckley, 1995 Sniezek Van Swol,
2001 Savadori, Van Swol, Sniezek, 2001)
JUDGE--responsible for decision
ADVISORgives advice Information
Recommendations Explanation
Expressions of uncertainty
9
Judge Behavior
Decision Problem
Judge
Advice Taking with Control of Advisor Budescu
Rantilla, 2000 Harvey Fischer,1997 Kuhn
Sniezek, 1996 Rantilla Budescu, 1999 Schrah,
2000 Stone Price, 2000 Yaniv Kleinberger,
2000 Yates, Price, Lee Ramierz, 1996
10
Judge vs. Advisor
Decision Problem
Judge
Advisor
Role Differences risk preferences (Hsee
Weber, 1997) this for me that for you (Kray
Gonzalez, 1998)
11
Joint Behavior
Decision Problem
Judge
Advisor
Mutual Influence Hedlund, Ilgen, Hollenbeck,
1998 Sniezek Van Swol, 2001 Sniezek, Schrah,
Dalal, 2001 Van Swol Sniezek, 2002
12
Judge Advisor System Research
Decision Problem
Judge
Advisor

• Individual Behavior Mutual Influence
• Role differences between Judge and Advisor
• Experimental manipulations of control over
  advising

13
Research Questions

• What are the differences between Judge and
  Advisor based solely on role?
• Is tutoring itself advantageous even if both
  parties have equal expertise?

14
Who controls the interaction?

• Advisor Control-unsolicited advice
• Advise
• Do not advise
• Judge Control- solicited advice
• Seek advice
• Do not seek advice

15
Advisor vs Judge Control

• What are the effects of Advisor vs Judge
  advising control on
• Frequency of advising?
• Quality of final decisions?
• Confidence in final decisions?

16
Advisor Control Methodology

• Lab study
• 202 students random assignment to role
• 101 JAS dyads, of peers
• Procedure
• Task decision making
• Reward allocation decision by Judge
• Post-assessments
• Reward

17
Procedure

• Task items
• 20 binary choice items on social psychology
• difficult, requires going beyond knowledge for
  both parties
• e.g. Hearing a message that is inconsistent with
  your existing attitude
• A increases distraction
• B induces counter-argumentation

18
Procedure

• Task Responses
• for each item
• Selection of one alternative
• expression of confidence probability of choice
  being correct on half scale, . 50-1.0

19
Outcome Variables

• Judge Advisor
• Accuracy proportion correct choices
• Goal is to maximize
• Confidence mean subjective probability of
  choice being correct
• Self evaluation of performance
• Goal is to maximize calibration and validity

20

JUDGE
ADVISOR
Independent choices and confidence
21

JUDGE
ADVISOR
Judge choices confidence communicated to
Advisor
22

JUDGE
ADVISOR
Judge choices confidence communicated to
Advisor
OPEN LINK
ADVISE
Unconstrained Dialogue
23

JUDGE
ADVISOR
Judge choices confidence communicated to
Advisor
OPEN LINK
ADVISE
Unconstrained Dialogue
(link closed)
Independent option to revise
24

JUDGE
ADVISOR
Judge choices confidence communicated to
Advisor
DONT ADVISE
Independent option to revise
25
Collaborative Technology for Advising

• Example interaction
• Judge on top
• Advisor on bottom

26
Collaborative Technology for Advising

• Advantages for research
• Advisor knows Judges initial choice and
  confidence before action is implemented
• Advisor can intervene in the decision process
  before final decision is implemented
• Advice giving is at the Advisors discretion
  Judge requesting is controlled (prevented), or
  vice versa
• Automatic recording of responses of both
  individuals (and the advising interaction) by the
  system

27
Results How much advice?

• Decision to advise M 22, 1-18 trials
• Frequency of advising
• is not predicted by
• Advisor original mean confidence
• Judge initial or Advisor score
• is predicted by
• difference between Advisor and Judge confidence
  and cognitive conflict
• R .33 R2 .106, p lt.005

28
Results When advise?
29
Results When advise?

• Smart decisions by Advisors When advise
  rather than pass, Judges were less accurate
  confident

Acc Conf
JUDGES ADVISORS
30
Results Taking Unsolicited Advice

• Judges made decisions about taking advice,
  sometime shifting and sometimes rejecting advice
• 83.2 of Judges changed their answer after
  receiving advice at least once.
• Mean 1.94 shifts
• Range 0-10 SD1.59

31
Results Taking Unsolicited Advice
32
Results When take advice?

• Smart decisions by Judges When shift rather
  than reject advice under conflict, Judges were
  less accurate confident

Acc Conf
JUDGES ADVISORS
33
Accuracy - -Confidence
Results Impact of Unsolicited Advice
A J A J

• Judges
• more accurate, but loss of potential
• more confident
• Advisors
• more accurate

p

Initial Final
Timing of Choice
34
Results Judges vs Advisors

• Advisors overestimated their influence
• Thought Judge took most of their advice
• Judge actually took just over half (53)
• Loss of potential
• Judges would make better decisions if they took
  more advice
• Asymmetric advising preferences
• Judges want more advice
• Strategic expression of confidence
• Advisors raise to get advice taken
• Judges lower (or raise) to encourage more (or
  less) advice

35
Interaction ControlAdvisor vs. Judge

• Judge Control experiment
• same task
• same population
• same reward system
• same procedure, except
• Judge decides whether to seek or not seek advice

36

JUDGE
ADVISOR
Independent choices confidence
37

JUDGE
ADVISOR
OPEN LINK
Unconstrained Dialogue
38

JUDGE
ADVISOR
Judge choices confidence communicated to
Advisor
OPEN LINK
Unconstrained Dialogue
(link closed)
Independent option to revise
39

JUDGE
ADVISOR
Judge choices confidence communicated to
Advisor
DONT SEEK ADVICE
Independent option to revise
40
Interaction Control Advisor vs. Judge

• What is the effect of Advisor vs Judge control
  on
• frequency of advising interactions
• final confidence levels of either partner
• the accuracy of final choices

41
Interaction Control Advisor vs. Judge

• The effect of Advisor vs Judge control on
  frequency of advising interactions
• Judges request far more help than Advisors offer
• Advisor control 22 of trials
• Judge control 52 of trials
• t 7.37, p lt .001

42
Interaction Control Advisor vs. Judge

• No significant differences were observed between
• the conditions in the initial scores and
  confidence levels of either partner
• the Advisors final outcomes
• A significant difference in final Judge decision
  accuracy
• the Judge Control advantage

43
The Judge Control Advantage
72
Judge Control
Advisor Control
68
proportion or prob
64
60
Confidence
Accuracy

• Judges are more accurate (plt.05) and confident
  (plt.02) when they and not the Advisor control
  advice-giving

44
Communication Analysis

• What is the nature of the unconstrained
  dialogue between Judge and Advisor ?
• Who speaks first? Most? Faster?
• What form is advice?
• Giving information , opinions, asking questions
• Communication of uncertainty
• How much? What valence
• Role differences?

45
Communication Analysis

• Advisor Control videoconference interactions
• 20 dyads 1 7 interactions each
• Total of 71 interactions
• Representative of total sample

46
Procedure Time Coding

• 1 rater divided each interaction into 3-second
  intervals and coded each interval for
• Speaker
• Form
• Expressions of Confidence
• Face Threatening Acts
• Politeness
• Rate of Speech

47
Procedure Frequency Coding

• 3 raters watched each interaction and counted the
  occurrence of
• Confidence expressions (positive/negative)
• 5 most common forms Opinion, Information,
  Question, Reply, Agreement
• acceptable reliability

48
Beginning the Interaction

• First Speaker
• Advisor 66.2 of interactions
• Judge 33.8 of interactions
• First Form

49
Role differences in speaking

• Average time speaking in interaction
• By Judges 21.6 seconds
• By Advisors 25.9 seconds
• Advisors spoke significantly more
• t (70) 2.18, plt.01
• Rate of speech
• Judges 1.85 words per second
• Advisors 2.12 words per second
• Advisors are also significantly faster
• t(69)2.72, plt.01

50
Positive Negative Confidence

• Positive Confidence Explicit statement about
  certainty, speaker is confident
• Im pretty sure...
• I was confident on this one...

• Negative Confidence Explicit statement of
  uncertainty, speaker is not confident
• I dont really know...
• I was just guessing...
• I had no idea...

51
Confidence Expressions

• Negative confidence was expressed much more often
  than positive confidence
• 167 intervals , 57 negative confidence
  expressions F(1, 231) 18.09, p lt .001
• 45 intervals, 18 positive confidence expressions
• Advisors make more confidence expressions of both
  kinds
• F(2, 231) 20.67, p lt .001
• Advisors have a higher ratio of negative to
  positive confidence intervals than Judges
• Interaction F(1, 19)6.655, p.018)

52
Confidence Expressions

• Mutual influence
• of Judge negative confidence expressions
  (intervals) is related to of Advisor negative
  confidence expressions (intervals)
• R .87, p.001, controlling for frequency of
  advising
• Judge and Advisor positive confidence
  expressions (counts) were positively correlated,
  R.59, p lt.05

53
Implications for intelligent tutoring

• Advising improves performanceeven though Advisor
  is a peer
• Student control over advising interactions may
  offer additional advantages to performance, while
  raising confidence
• Underutilization of advice in probabilistic
  environments may be an issue

54
Future research

• Evaluation of alternative intelligent tutoring
  techniques at Naval Postgraduate School
• Examination of role of self evaluation of
  performance