How Many Scientists Fabricate and Falsify Research? A Systematic Review and Meta-Analysis of Survey Data - PowerPoint PPT Presentation

About This Presentation
Title:

How Many Scientists Fabricate and Falsify Research? A Systematic Review and Meta-Analysis of Survey Data

Description:

How Many Scientists Fabricate and Falsify Research? A Systematic Review and Meta-Analysis of Survey Data Daniele Fanelli – PowerPoint PPT presentation

Number of Views:183
Avg rating:3.0/5.0
Slides: 15
Provided by: dfa82
Learn more at: https://ori.hhs.gov
Category:

less

Transcript and Presenter's Notes

Title: How Many Scientists Fabricate and Falsify Research? A Systematic Review and Meta-Analysis of Survey Data


1
How Many Scientists Fabricate and Falsify
Research?A Systematic Review and Meta-Analysis
of Survey Data
Daniele Fanelli
2
The million dollar question.
Data source Frequency () of misconduct
Government-confirmed cases in the US 0.001 0.01 (Marshall 2000, Steneck 2006)
Retractions due to misconduct, in PubMed 0.02 - 0.2 (Claxton 2005)
Manipulated images submitted to The Journal of Cell Biology 1 (Steneck 2006)
Investigators disqualified by FDA data audits (1977-1990) 2 (Glick 1992)
-Only misconduct that has been discovered, and
(presumably) proven to be intentional
Ultimately, only scientists know about their own
intentions!
3
Over the years, many surveys have asked
scientists directly
different things, in different ways
Form of misconduct
Question
Outcome
Since entering medical school have you?
Fabricated data
Yes No
Modified research or experimental results to
improve the outcome
Never SometimesFrequently
Have you participated in research involving
during the last 10 years?
Failing to present data that contradict one's
own previous research
0 1-5 gt5
Indicate the number of members you have
observed/experienced exhibiting within the
last 5 years
Seriously misleading interpretation of results
Results appeared inconclusive and dificult to
compare
4
Tricks in the analysis
  • How many committed or observed X at least once
  • Only questions on fabrication, falsification,
    alteration and QRP that distort scientific
    knowledge. No plagiarism, professional misconduct
    etc
  • Mixed questions were excluded

Question by question effect size and weight
  • No other measure of study quality (its
    controversial)
  • Included all eligible studies that specified
    their methods
  • Entered methodological factors in inverse
    variance weighted regression

5
The search
"research misconduct" OR "research integrity" OR
"research malpractice" OR "scientific fraud" OR
"fabrication, falsification" OR "falsification,
fabrication"
42 literature databases, 14 journals, 8 grey
literature db, 2 internet scientific search
engines, and references lists
Potentially relevant studies obtained from
electronic search (n3276)
Studies excluded because were not surveys on
research misconduct (n3207)
Studies excluded for one of the following reasons
(n48) -Did not have any relevant or original
data -Sample not exclusively composed of
researchers -Misconduct not related to research
(e.g. cheating on school projects) -Does not
distinguish fabrication and falsification from
other forms of misconduct not relevant to this
review -Presents data only in format not usable
in this review
Studies retrieved for examination of full text
(n69)
Studies included in review (n21)
Studies excluded from meta-analysis because did
not meet quality criteria (n3)
Studies included in meta-analysis (n18)
6
Characteristics of studies
  • Conducted between 1986-2005
  • USA (15), UK (3), multinational (2), and
    Australia (1)
  • Medical/clinical (8), biomedical (6),
    multidisciplinary (6), economists (1)
  • In total 85 questions
  • about fabrication, falsification, alteration,
    modification (meta-analysis)
  • Questionable research practices (systematic
    review only)

(Full data set available soon in PLoS ONE)
7
Scientists who admit fabrication, falsification,
or alteration of results
Scientists who know a colleague who fabricated,
falsified, or altered results
b -0.140.05 P0.006
1.97 (N7, 95CI 0.86-4.45)
If only asked fabrication, falsification 1.06
(N4, 95CI 0.31-3.51)
14.12 (N12, 95 CI 9.91-19.72)
If only asked fabrication, falsification 12.34
(N11, 95CI 8.43-17.71)
8
Questionable Research Practices
(e.g. failing to publish data that contradicts
ones previous research dropping data points
based on a gut feeling)
9
What influences admission rates?
Inverse variance-weighted regression
bSE
P
-
Asking about self vs colleagues
-4.530.81
lt0.0001
82 of variance explained (N15)
-
Using fabrication or falsification vs
alteration or modification
-1.020.39
0.0086

Handed-out surveys vs mailed
1.170.4
0.0032
Controlling for these factors, tested for
differences between
Year
USA / other
Researcher / other
n.s.
Medical / other
b0.850.28
Biomedical / other
P 0.0022
Social Sc. / other
10
Leave-one-out sensitivity analysis
Scientists who admit fabrication, falsification,
or alteration of results
Scientists who know a colleague who fabricated,
falsified, or altered results
Martinson 2005 is outstanding, as conservative!
11
Repairing misconduct
ID N cases Action taken
Tangney, 1987 78 Took some action to verify their suspicions of fraud or to remedy the situation 46
Rankin, 1997 31 (incl. Plag.) In alleged cases of scientific misconduct a disciplinary action was taken by the dean 32.4
Some authority was involved in a disciplinary action 20.5
Ranstam, 2000 49 I interfered to prevent it from happening 28.6
I reported it to a relevant person or organization 22.4
Kattenbraker, 2007 33 Confronted individual 55.5
Reported to supervisor 36.4
Reported to Institutional Review Board 12.1
Discussed with colleagues 36.4
Titus, 2008 115 (incl. Plag.) The suspected misconduct was reported by the survey respondent 24.4
The suspected misconduct was reported by someone else 33.3
Around half of recalled cases had no action
whatsoever taken against them
12
Summary of key findings
  • Data fabrication, falsification and alteration
    was
  • admitted on average by around 2 (1 - 4)
  • directly observed by 14 (10 - 20)
  • Questionable Research Practices were
  • admitted on average by up to 34
  • directly observed by up to 72
  • Overall admission rates (self-/non-self) were
    higher in
  • Non-self reports, questions not using
    fabrication or falsification, handed out
    questionnaires
  • Medical/clinical and related research

13
How Reliable Are These Numbers?
Self-reports
Conservative
Unlike surveys for other criminal
behaviour, Scientists always lose by admitting
misconduct
Non-self-reports
  • Unclear
  • Risk of multiple reporting
  • Muhammad Ali effect
  • Unaware of all cases
  • Unwilling to damage their field
  • Regression analysis
  • Medical research not robust to all sensitivity
  • Differences in methodology masked most effects
  • Non-significant effects not necessarily
    non-significant

14
Conclusions
  • On average, 2 of scientists admitted misconduct,
    and 34 QRP
  • Actual frequencies probably higher
  • Probably vary depending on field and many other
    factors, which meta-analysis currently cannot
    detect
  • Future surveys might benefit by
  • Focusing on correlates of misconduct
  • Common methodologies
Write a Comment
User Comments (0)
About PowerShow.com