What Researchers Need to Know about the Statistical Consulting Process or Working with the Statistic

1
What Researchers Need to Know about
theStatistical Consulting ProcessorWorking
with the Statistician as a Research Colleague

• Walt Stroup
• Professor Chair, UNL Department of Statistics

2
(No Transcript)
3
Outline for Talk

• Role of a Statistician
• Statistical View of Research Process
• From planning to finished product
• Essential features of an effective partnership
• What is expected of you what you should expect
• What can go wrong what to do about it

4
Cool Quote Source Unknown

• Writers tell stories with words
• Painters tell stories with pictures
• Musicians tell stories with music
• Statisticians tell stories with numbers

However
There must be a story to tell or the words,
pictures, music, numbers --- and sophisticated
methodology --- are ...full of sound and fury,
signifying nothing...
5
So...Statisticians
The object of consulting is
6
Cool Quote Source Unknown

• There are three subjects everybody thinks they
  can be an expert in with no formal training
• Law
• Medicine
• Statistics

NOT !!!
7
Role of Statistics in Research

• Another cool quote, source also unknown
• Think of Statistics as the rules of evidence
  for scientific inquiry
• Webster Statistics is a branch of
  mathematics...
• Better Oxford English Dictionary Statistics
  is the science of data collection, analysis, and
  reporting for the purposes of characterizing a
  population, testing a theory, or making a
  decision

8
Some Influences
My Major Professor R.L. Anderson This is the
worst data set I ever saw! Why didnt they
talk to me first? Unbelievable!
90 of the value of statistics is in what happens
before any data are collected When people see me
after the fact I often feel that the only thing I
can do is write the death certificate
Gerald Hahn
Dont argue take data You need to learn to
think of yourself not as a statistical
consultant but as a research consultant
9
E. O. Wilson

• not the exact quote, but true to spirit
• What sets good research apart is
• usually not ones cleverness in finding
• the answer but ones skill in
• asking the right question, and
• phrasing it carefully and well

Theme? Planning!
to which I would add ...and collecting the
right data...
10
(No Transcript)
11
Steps in Research Planning Protocol Description
What is the Research Question?
Answer
?
Relevant Evidence to answer question (response
variables
Conclusions
Data analysis
New questions
Collect data
How will evidence be used to answer (hypothesis
test?)
How much evidence? (sample size?)
How will evidence be collected
design? (clinical trial model? sample survey? )
12
When is this Planning Applicable?

• Grant writing
• Preparing a dissertation proposal
• Journal Publication
• Informing Policy Decisions

13
late Senator Daniel Patrick Moynihan
we in this nation face a choice we can inform
public policy with sound research or we can
continue to make policy on the basis of
data-free ideology
14
Another of Walts mentors Dal Kratzer

• defined statistics as experimentation science
• Protocol description form
• Intended for pharmaceutical research
  development
• With word-smithing and tweaking, readily
  applicable as a template for research planning

15
Protocol Description Form
16
Working with Statistician

• Haste makes waste
• Build a relationship
• Plan to educate statistician about your research
• Two way street
• Youre not a statistical expert the statistician
  is not an expert in your discipline
• De-jargonization

17
What you dont want
18
Types of statistical investigations

• Data mining
• characterization, fishing expedition
• looking for patterns that might be there
• latent class, factor analysis, other multivariate
  methods
• Associations
• cross-tabs, log-linear models, path analysis
• often based on survey or retrospective data
• Treatment or factor effect
• based on experiment design or quasi-design
• clinical trial model closest thing to cause
  effect

19
Essential Parts of the Process - I

• What is the research question?
• It is easy to be vague
• It is easy to be vague when you think you arent
• Operating definitions are essential
• Is Math in the Middle effective?
• what do you mean by effective?
• compared to what?
• once effective has an operating definition, can
  you measure it? how?
• if you cant, you dont have an operating
  definition yet

20
Essential Parts of the Process - II

• Power
• First
• what is the population?
• what elements represent the population?
• how do you (ethically) observe the elements?
• Then
• what are the relevant sources of variation among
  the elements?
• what is the magnitude of variation assoc with
  these sources?
• how big an effect is considered important?

21
What power analysis is NOT
22
Essential Parts of the Process - III

• Power
• Should be a dialog
• At its best
• mutual learning
• dress rehearsal for data collection and analysis
• INCLUDE what can go wrong what is my plan?
• missing data happen
• missing data are inevitable

23
Essential Parts of the Process - IV

• Stay away from jargon and discipline-speak
• applies to you
• applies to statistician
• Ask questions
• dont let stat-speak go by
• its not your ignorance, its the statisticians
  language skills
• Answer questions
• if a good statistician asks you a question, its
  because she is curious trying to get herself in
  the best position to be helpful
• questions are not a hidden agenda to find fault
  or embarrass you

24
Essential Parts of the Process - V

• If possible, face-to-face meetings are better
• Haste makes waste
• last minute, rush-job consulting invites trouble
• Another mentor quote
• we dont have time to do it right, but we have
  plenty of time to do it over

EGO
25
Analysis

• Vastly easier in well-designed study
• Vastly better if statistician has been involved
  all along
• Parsimony
• if two analyses show essentially same results and
  plausibly satisfy assumptions, the simpler
  analysis is better
• 90 rule
• However
• just because it is traditional does not make it
  right
• computer has had huge impact on statistical
  methods

26
Analysis Reporting

• Part of statisticians job is to help write
  report
• material methods
• statistical design
• analysis
• result and discussion
• help explain and interpret
• Statistical science itself is a work in progress
• your project may motivate statistical research
• methods may be unfamiliar to reviewers
• statistician may need to educate referees /
  editors

27
Etiquette Ethics

• Authorship
• would research / grant have been possible without
  statisticians efforts?
• NO means a co-authorship should be offered
• Funding
• external funding is lifeblood of research
  university
• statisticians dilemma in academia
• do I close my door to colleagues in allied
  disciplines to write my own grants, or do I
  participate in interdisciplinary teams?
• for latter, co-PI grant arrangements with real
  essential

28
More Etiquette Ethics

• Special Note on Graduate (Dissertation) Research
• Common Unfortunate Situation Graduate Student as
  Ping-Pong Ball
• When statistician advises graduate student on
  design and/or analysis (ideally, and ed.)
  student, students major professor, and
  statistician should be on same page
• Ideal statistician on PhD committee
• But at least meet face to face periodically

29
Final Thought

• Most Statistics departments are seriously
  understaffed UNL no exception
• Not a complaint just a reality
• Conflicting demands
• teaching
• research
• consulting
• Applied statistics need interdisciplinary work
  just as quality research needs engaged statistics
  
• BUT you may have to be a nag

30
thats all thanks
Any Questions?