European Communication on Research Awareness Needs WP6 - PowerPoint PPT Presentation

Loading...

PPT – European Communication on Research Awareness Needs WP6 PowerPoint presentation | free to download - id: 6ba1a0-MWMyM



Loading


The Adobe Flash plugin is needed to view this content

Get the plugin now

View by Category
About This Presentation
Title:

European Communication on Research Awareness Needs WP6

Description:

European Communication on Research Awareness Needs WP6 EBM guide for healthcare journalists to the critical interpretation of clinical trials – PowerPoint PPT presentation

Number of Views:28
Avg rating:3.0/5.0
Date added: 25 February 2020
Slides: 56
Provided by: NPieri
Learn more at: http://www.csermeg.it
Category:

less

Write a Comment
User Comments (0)
Transcript and Presenter's Notes

Title: European Communication on Research Awareness Needs WP6


1
European Communication on Research Awareness
NeedsWP6 EBM guide for healthcare
journalists to the critical interpretation of
clinical trials
2
ECRAN EBM guide for healthcare journalists
  • What is quality of scientific information?
  • Lets sort out the wheat from the chaff
  • The huge amount of research studies is a big
    challenge for health care journalists, who have
    to sort out the wheat of scientific excellence
    from the chaff of low quality or interested
    driven science. The purpose of this guide is to
    ensure excellence in medical journalism by means
    of
  • applying the core principles of journalistic
    coverage and investigation in the field of
    science and medicine
  • training journalists to properly cover news
    events, trends, and issues in all aspects of
    health care, sorting out the tangle of data about
    clinical trials and transferring them to medical
    practice in a comprehensive and multifactorial
    perspective

3
WP3 task 3.2 Others (1)
ECRAN EBM guide for healthcare journalists
  • enabling properly trained journalists to better
    convey the complex dimensions of health care
    challenges from an overall point of view
  • ensuring an independent coverage about health and
    research, analyzing health related data with
    fairness, accuracy and balance
  • in order to
  • increase the capacity of individuals and
    policymakers to adopt health care attitudes and
    policies that can improve the health of all the
    citizens
  • improve the ability of citizens to make wise
    decisions about behaviors that promote health,
    treatment options, and their choice of health
    care providers.


4
WP3 task 3.2 Others (2)
ECRAN EBM guide for healthcare journalists
  • What is science?
  • A system of certain knowledge in a period of
    time built up, arranged and ensured by research,
    teaching and accumulated data.
  • (translation from the German definition in
    Brockhaus 20032193)
  • Which is the goal of science?
  • The ultimate goal of science must always be to
    verify or reject existing (content-specific or
    methodical) knowledge and to replace it with new
    knowledge. Science is a cumulative process. The
    touchstone for assessing new knowledge is the
    comparison with existing knowledge. Systematic
    reviews, that collect and evaluate knowledge of
    several scientific studies, have been developed
    and implemented for 20 years as methodical tool
    for assessing research.

5
WP3 task 3.2 Others (3)
ECRAN EBM guide for healthcare journalists
  • How should good science look like?
  • Assessing good science according to its
    neutrality and objectivity is unrealistic,
    because of its several infrastructural, financial
    and performance-related implications. In the last
    decade the call of transparency has been playing
    a central role in this process, in order to
    ensure the integrity of research. Assessment of
    the framework conditions for research should be
    therefore integrative part of each journalistic
    investigation, in order to set an indicator for a
    preliminary assessment of the reliability of
    research outcomes.
  • How should good journalistic investigation look
    like?
  • Basic scientific understanding of scientific
    studies and their weak points enables journalists
    to facilitate orientation and to avoid lurking
    pitfalls in this field (e.g. look at the section
    Tips for understanding studies Common
    pitfalls of this guide).

6
ECRAN EBM guide for healthcare journalists
  • Toolkit for journalists
  • There are many challenges to achieving accuracy,
    balance and completeness in health care
    journalism. For a reporter, there are deadlines,
    editors, and corporate financial pressures.
  • Though quality in health care journalism is far
    more than this, because it plays a major role in
    educating consumers! Consumers need to be well
    informed to make sound choices that affect their
    health and well being.
  • The following rating criteria are an
    internationally validated methodology, that helps
    conjugate general and medical journalistic
    criteria to review stories claiming about
    specific treatments, tests, products or
    procedures (i.e. drugs or devices, vitamins or
    nutritional supplement, diagnostic and screening
    tests, dietary recommendations, surgical
    procedures).

7
ECRAN EBM guide for healthcare journalists
  • Rating criteria for healthcare journalists

1 Quantification of benefits (treatment/test/produ
ct/ procedure)
2 Quantification of harms of intervention
3 Proper evaluation of quality of evidence
4 Further independent source and declaration of
conflicts of interest
5 Story solely or largely based on news release?
6 Establishing true novelty of approach
7 Comparing new approach with existing
alternatives
8 Establishing availability of treatment/test/prod
uct/ procedure
9 Adequate estimate of costs of intervention
10 Is there potential for disease mongering?
8
ECRAN Rating criteria for healthcare
journalists
1
  • Quantification of benefits (treatment/test/product
    /procedure)
  • Stories are expected to describe the
    effectiveness of a benefit. Few stories provide
    helpful numbers to back the real benefit of a new
    treatment, test, product or procedure. Empirical
    investigations have shown that news reports, drug
    ads and some studies coverage tend to report
    benefits as the relative reduction instead of the
    absolute risk reduction in the frequency of
    failures.
  • Another challenging question may be How many
    patients have to be treated in order to prevent
    the problem for one patient?. This is the Number
    Needed to Treat (NNT) it is defined as the
    inverse of the absolute risk reduction and is the
    average number of patients who need to be treated
    to prevent one additional bad outcome.


9
ECRAN Rating criteria for healthcare journalists
1
  • What does significantly increased mean? How was
    this measured? Apart from this, statistical
    significance may not be equal to clinical
    relevance. The real question is What difference
    does it make in peoples lives? Reports
    indicating surrogates and intermediate endpoints,
    e.g. changes in blood test scores, may not
    influence individual health scores (increased
    life expectancy or lower mortality rate) or
    quality of live.
  • Moreover, anecdotal storytelling makes engaging
    reading but is not scientific data. Glowing
    personal stories or single unchallenged
    exaggerated success stories may not be a
    representative example of how a benefit works and
    require a critical appraisal for a balanced
    framing of it.

10
Criterion will not be fulfilled, in case of
ECRAN Rating criteria for healthcare journalists
1
  • Data of possible benefit is not available at all.
  • Using only relative, not absolute, risk reduction
    or benefit data.
  • Relying too heavily on what may be
    unrepresentative patient anecdotes about
    benefits.
  • Citing statistics appropriately, but allowing
    even a single unchallenged exaggerated quote to
    throw the story out of balance.
  • Question to clear at the end of the day
  • Is the benefit really as much more or less
    significant as it is presented?

11
Quantification of harms of intervention
ECRAN Rating criteria for healthcare
journalists
2
  • Be aware of appearances! When you hear someone
    saying that a new approach appears to be safe,
    look for evidence supporting it.
  • Stories may emphasize or exaggerate potential
    benefits of interventions, while minimizing or
    completely ignoring their potential harms. For
    some patients slight side effects may even have
    dramatic impact on their lives. An overall
    healthcare report should always consider possible
    even the slightest adverse effects of
    interventions, ideally reporting their frequency
    and severity.
  • Any healthcare decision implies tradeoffs, all
    treatments may have side effects, risks and
    complications.


12
ECRAN Rating criteria for healthcare journalists
2
  • Even a screening test may turn out to be harmful
    to the individual getting it in fact, it may
    produce a false positive result, which causes
    anxiety, and will probably lead to more
  • testing (i.e. biopsy) and possibly treatment
    (i.e. surgical intervention) that comes with its
    own harms. Moreover, enthusiasm for early tests
    in people without symptoms is often not supported
    by evidence.
  • PSA testing, for example, often turns out to
    cause more harms than benefits, including false
    positive results and overdiagnosis, both of which
    could lead to unnecessary treatments associated
    with significant side effects.

13
ECRAN Rating criteria for healthcare
journalists
2
  • Criterion will not be fulfilled, in case of
  • Failing to mention potential harms.
  • Failing to quantify potential harms.
  • Failing to describe the severity of potential
    harms.
  • Failing to account for minor side effects that
    could have a significant impact on a patients
    life.
  • Relying too heavily on a patient anecdote about
    safety.
  • Relying too heavily on a researchers comment
    that an approach appears to be safe without
    supporting data.
  • Question to clear at the end of the day
  • Which are the risks and the side effects of this
    intervention or product?

14
ECRAN Rating criteria for healthcare journalists
3
  • Proper evaluation of quality of evidence
  • Healthcare coverage should critically evaluate
    the evidence and not merely take published or
    presented research as gospel. Quality of evidence
    has to be classified according to evidence
    hierarchy

15
ECRAN Rating criteria for healthcare journalists
3
  • Science works cyclically with ebbs and flows
    knowledge stream keeps flowing and keeps changing
    in the course of time. So, ask yourself
  • At which point in the stream did I grasp the
    evidence?
  • Not all evidence is bullet-proof, even if it is
    published in high impact journals, because not
    all studies are equal. Eminence is not evidence
    and study stories often turn out to be ongoing
    stories.
  • A solid journalistic investigation is expected to
    scrutinize limitations of evidence, number of
    probands, duration of the study, relevance of
    outcomes to assess whether what appears as the
    latest story is already existing in larger and
    better-designed stories on the same topic or
    whether the statements generalize small group
    data (e.g. a single medical center with
    experienced surgeons).

16
ECRAN Rating criteria for healthcare
journalists
3
  • Criterion will not be fulfilled, in case of
  • Failing to dig out the limits of the study design
    (i.e. lack of control group, lack of blinding,
    missing drop out rate).
  • Not cautioning on the interpretation of
    uncontrolled data.
  • Conflating causation and association (i.e.
    limitations of observational studies).
  • Generalizing small group data (i.e. at one
    medical center with experienced surgeons).
  • Confounding surrogates with real health outcomes
    (illness or death vs. markers or scores).
  • Not mentioning that scientific meeting outcomes
    lack adequate peer review.
  • Not pointing out that findings from animal or lab
    experiments are not directly applicable to human
    health.
  • Presenting anecdotic data as evidence for
    intervention.
  • Question to clear at the end of the day
  • Does this study design justify these allegations?

17
ECRAN Rating criteria for healthcare
journalists
4
  • Further independent source and declaration of
    conflicts of interest
  • Claims about new treatments, tests, products, and
    procedures should be based on evidence, not on
    beliefs or business hopes. In health care
    journalism, it is therefore essential that second
    opinions be sought from independent experts who
    have no vested interest in the approach being
    discussed. For this scope two things must occur
  • there must be an independent expert source
    quoted (someone not directly connected with the
    research),
  • there must be some attempt to let the public know
    about potential conflicts of interest.

18
ECRAN Rating criteria for healthcare
journalists
4
  • Hyperbolic healthcare claims made by drug
    companies, medical device manufacturers and
    academic research centers are to be confronted
    with this question
  • Do they have something to gain?

May be that a trial is paid for by the drug
manufacturer researchers are getting fees from a
drug company claims are made by a spokesman for
a device manufacturer doctors are being early
adapters of a new device there are underlying
inventors interests.
19
ECRAN Rating criteria for healthcare journalists
4
  • A study funded by industry doesnt necessarily
    mean that findings lack integrity, but evidence
    from recent studies shows that potential
    conflicts might lurk around every corner in
    health care. So, the next time you go to your
    doctors office that is cluttered with drug
    company promotions, it is reasonable for you to
    ask him if he/she had any financial relationship
    with the makers of drugs, devices or test
    advertised for. Moreover, seek independent
    perspectives and beware of statements like
    Experts believe , doctor think . Instead
  • Give them a name, because vagueness is nameless!

20
ECRAN Rating criteria for healthcare journalists
4
  • Criterion will not be fulfilled, in case of
  • Failing to identify the source of the story (news
    release, journal article, editorial, scientific
    meeting presentation, etc.) and failing to make
    obvious the extent to which that source is likely
    to be conflicted (e.g. A PR consultant working
    for the company said or Dr. Smith, who
    received a company grant to perform the study,
    said)
  • Failing to include the input of a trusted
    independent source with expertise on the subject
    to comment on the claims made (ideally, more than
    one such independent source would be used).
  • Question to clear at the end of the day
  • To what extent are the information independent
    and how plausible is the expertise?

21
ECRAN Rating criteria for healthcare journalists
5
  • Is the story solely or largely based on
  • news release?
  • It is simply not sound journalistic practice to
    lift material directly from a news release
    without stating that this is the case. If we know
    that a news release was sent out and if we can
    get a copy of that release, we will review it
    along with the story.
  • Many news organizations facing financial
    pressures may be lead to become more open to
    using news releases as a way to publish content
    while cutting costs. Although news releases can
    be valid sources of some information, journalism
    is being asked to collect independent information
    with the consequence that it is unacceptable to
    rely on a news release as the sole source of
    information.

22
ECRAN Rating criteria for healthcare
journalists
5
  • Particularly in health care, there are many
    vested interests trying to influence consumer
    choices. Journalists are expected to use
    independent verification relying neither on news
    releases nor on company spokesmen!
  • A team from Dartmouth Medical School and the
    Veterans Affairs Outcomes Group published in 2009
    an analysis on the quality of press releases by
    medical centers concluding that Press releases
    from academic medical centers often promote
    research that has uncertain relevance to human
    health and do not provide key facts or
    acknowledge important limitations.

23
ECRAN Rating criteria for healthcare journalists
5
  • Criterion will not be fulfilled, in case of
  • There is clear evidence that the exact same
    wording from a news release appears in a story.
  • Lacking evidence of a press release and the worst
    a story can be rated is not applicable.
  • Criterion will be fulfilled, in case of
  • A story has several independent sources and
    proves NOT to rely only on a news release.
  • Question to clear at the end of the day
  • Is the news based only on a press release?

24
ECRAN Rating criteria for healthcare journalists
6
  • Establishing true novelty of approach
  • Many alledgedly new treatments, tests, products
    or procedures are not really novel. The agent may
    be another member of a well-established
    therapeutic class of drug. Even if it represents
    a new class, it may offer no more than the drugs
    that are widely available. In the press release
    for a new drug this essential information may be
    lost in the hype and the drug is portrayed as
    novel in order to increase initial sales. So
    ask yourself
  • Is it novel or simply new?
  • Journalists reporting should be able to make
    these distinctions on the real novelty of a
    product by researching or asking independent
    sources, e.g. specifiying that there are previous
    studies on this agent for the same indication.

25
ECRAN Rating criteria for healthcare
journalists
6
  • In health care, newer isnt always better, the
    first in a new class of drugs may offer no more
    than the drugs that are widely available. Drug
    companies are very good in promoting the alledged
    novelty of a drug in order to increase initial
    sales. Moreover, some claims for new drugs are
    made for off-label use, i.e. for not officially
    approved uses and under doctors careful
    monitoring. Off-label uses might also result in
    wasting money and harming patients and the
    respective costs may not be covered by the state
    or private insurance. By focusing on a new idea
    journalists should tell how well it works putting
    it into the context of existing alternatives.

26
ECRAN Rating criteria for healthcare
journalists
6
  • Mostly it is rare that a study appears out of
    nowhere and the big picture, where the
    novelty comes from, is often missing. To find
    it out journalists should search for the opinion
    of independent experts or reliable scientific
    sources like.

27
ECRAN Rating criteria for healthcare
journalists
6
  • Criterion is will not be fulfilled, in case of
  • Failing to accurately describe the new
    product/intervention and, as a consequence of
    this, misleading the consumer.
  • Failing to explain the real novelty of the
    product/intervention (preferably comparing it to
    existing alternatives on the market).
  • Failing to mention that the described new
    product/intervention is, in fact, for off-label
    uses.
  • Question to clear at the end of the day
  • Is the product/intervention a real novelty?

28
ECRAN Rating criteria for healthcare
journalists
7
  • Comparing new approach with existing alternatives
  • It is mandatory for a healthcare journalist to
    put the new treatment into the context of
    existing alternatives that have a longer, more
    proven track record. Whatever the new approach,
    the report should try to put it into the context
    of already available interventions, in order to
    compare the effectiveness of the various options.
    News stories on new treatments, tests, products
    and procedures should also take into
    consideration the value for money ratio.
  • In case a treatment should have been developed
    from scratch, the journalist is expected to
    explain that there are no alternative treatments
    or only supportive or soothing remedies.

29
ECRAN Rating criteria for healthcare
journalists
7
  • To wait and see is also a plausible alternative
    to the proposed treatment/test/product/interventio
    n. Active surveillance, declining immediate
    aggressive therapy, is a viable and rationale
    option.
  • It is wise for patients to know their treatment
    options. They may also consider to make lifestyle
    changes rather than taking a drug, or else they
    may pursue a drug or non-surgical option rather
    than having surgery.
  • Promotions of screening tests should also
    consider the option for individuals to choose NOT
    to be screened.

30
ECRAN Rating criteria for healthcare
journalists
7
  • Criterion will not be fulfilled, in case of
  • Discussing a surgical approach without mentioning
    non-surgical alternative approaches.
  • Discussing a new test without mentioning other
    tests that are available, including the option of
    not being screened in the case of a screening
    test.
  • Failing to discuss the advantages/disadvantages
    of the new idea compared with existing
    approaches.
  • Failing to discuss how the new treatment, test,
    product or procedure fits into the realm of
    existing alternatives.
  • Question to clear at the end of the day
  • Are there any other treatment options besides
    this procedure/product?

31
ECRAN Rating criteria for healthcare journalists
8
  • Establishing availability of
  • treatment/test/product/procedure
  • This criterion shall mostly be applied to
    coverage about a new drug in clinical trials (or
    in a trial on the new use of an established
    drug). A journalist is expected to tell his/her
    readers/public, if the drug/agent is available at
    the chemists (or may be soon) or if it is really
    only in an early-phase trial (e.g. pilot study or
    stage before approval).
  • It is important for a journalist to know that
    there may be a lot of marketing hype that
    suggests that something is well-studied and ready
    for use.
  • Questions should also be raised by the fact that
    something should have been approved in the U.S.
    and not or not yet in Europe.

32
ECRAN Rating criteria for healthcare
journalists
8
  • Some early studies focus on safety and not on how
    well something works. Some other focus on early
    efficacy and may not be large enough or designed
    to show safety of a drug/treatment in the long
    term.
  • Predictions about something to be approved by
    in a certain period should also be taken with a
    grain of salt, they usually come from someone who
    stands to benefit.
  • New approaches, devices or operations should be
    scrutinized for their availability (no one wants
    to be patient number one with a new
    device/operation).
  • Finally, have the personnel required to deliver
    the new approach properly been trained and are
    there enough of them? What is the learning curve
    for this?

33
ECRAN Rating criteria for healthcare
journalists
8
  • Criterion will not be fulfilled, in case of
  • Making not clear that the drug is not available
    now.
  • The approval of a drug is treated as a closed
    issue.
  • Something expected to be approved by and its
    availability on the market sounds more like a
    crystal ball prediction.
  • Time predictions about the market availability of
    the drug are solely based on declarations sourced
    to a company spokesperson.
  • Failing to describe to what extent a treatment is
    available (in every hospital or only in specialty
    clinics).
  • Promoting a special intervention without
    providing a sense of how many trained doctors
    there are in the field (e.g. a hi-tech surgical
    intervention).
  • Question to clear at the end of the day
  • Is this item available now or not?

34
ECRAN Rating criteria for healthcare journalists
9
  • Adequate estimate of costs of intervention
  • Journalists are expected to appropriately report
    on intervention/product costs, though it may be
    difficult to estimate costs of an experimental
    approach early in its development. Even if the
    real cost is probably known at the time of market
    entry, health reporters could at least cite costs
    of existing alternatives. Journalists should be
    able to get a reasonable estimate from their
    source/experts on the topic.
  • The rule of thumb is
  • If it is not too early to talk about
  • how well something might work,
  • then it is not too early
  • to start discussing what it may cost

35
ECRAN Rating criteria for healthcare journalists
9
  • A large amount of healthcare stories do not
    appropriately report on the cost issue.
    Healthcare takes a large part of the Gross
    Domestic Product in developed countries and we
    all better start thinking about costs of
    treatments, tests, products and procedures, we
    all pay for.
  • So, it is not enough to say
  • The cost is much lower than
  • but you should better ask
  • What does this actually cost?
  • How much lower?
  • What about the cost-effectiveness, if compared to
    ?
  • Journalists should also consider additional
    expenses for psychologists, dieticians,
    physiotherapists involved in the intervention or
    that the convalescence and rehabilitation may be
    more time-consuming.
  • Moreover, they should investigate, if insurances
    cover the intervention/drug or it is affordable
    for the patients needing them.

36
ECRAN Rating criteria for healthcare journalists
9
  • Criterion will not be fulfilled, in case of
  • Costs of an intervention are not mentioned at
    all.
  • Costs are played down without giving a
    satisfactory explanation therefore.
  • Not comparing the costs of a procedure with other
    available alternatives (more or less expensive
    ones).
  • Not mentioning, if the new procedure/drug will be
    covered by public/private health insurance.
  • Criterion will not apply, in case of
  • An experimental procedure in early development
    with no mention of the related costs.
  • An expert claiming that costs are quite
    unimportant (e.g. speaking of well known products
    like aspirin or weight-loss products).
  • Question to clear at the end of the day
  • Is it covered by health insurance or must I pay
    out of my pocket?

37
ECRAN Rating criteria for healthcare journalists
10
  • Is there potential for disease mongering?
  • This criterion assesses whether a condition is
    exaggerated or oversold. There are different
    forms of mongering
  • turning risk factors into diseases (e.g. low
    bone mineral density becomes osteoporosis)
  • misrepresentation or inaccurate description of
    the natural history and/or severity of a disease
    (e.g. early-stage low-grade prostate cancer)
  • medicalization of minor or transient variations
    in function (e.g. temporary erectile dysfunction
    or female sexual dysfunction)
  • medicalization of normal states (baldness,
    wrinkles, shyness, menopause)
  • exaggeration of the prevalence of a
    disease/disorder (e.g. using rating scales to
    diagnose erectile dysfunction).

38
ECRAN Rating criteria for healthcare journalists
10
  • Some statements are trying to make all of us feel
    like theres something wrong with us even if
    there isnt. Thats what we call
    disease-mongering. This is dangerous because it
    is an unnecessary emotional and financial burden
    for society and may distract us and precious
    public resources from important healthcare issues
    that should be addressed. Also turning a normal
    state or variation of health into a medical
    problem is disease mongering, like baldness,
    wrinkles, shyness or menopause. Pseudodiseases,
    that lower the threshold for common diseases but,
    in fact, only open new markets, may turn millions
    of people overnight into patients needed to be
    treated with drugs or vitamins against high
    cholesterol or osteoporosis. Often, women are
    targeted because of premenstrual dysphoric
    disorders, female sexual dysfunctions, overactive
    bladders or menopause men equally get their
    share being hammered because of low testosterone
    levels, balding etc.

39
ECRAN Rating criteria for healthcare journalists
10
  • Identifying disease mongering is a matter of
    judgment. Sometimes it is obvious. Sometimes
    theres a fine line about whether a statement on
    irritable bowel disorder, erectile dysfunction,
    restless leg syndrome or osteoporosis (all of
    which can be serious for some sufferers) is
    misrepresenting the condition to the public.
    Sometimes there may be multiple mongering with
    many hyperboles for the same issue.

40
ECRAN Rating criteria for healthcare journalists
10
  • Criterion will not be fulfilled, in case of
  • Spurious statistics are presented. Question
    prevalence estimates!
  • Statistics inflate the seriousness of a condition
    (e.g. millions of people suffering in silence
    with toenail fungus)
  • Someone selling sickness wants to scare you
    into testing or treatment.
  • Risk factors are treated as diseases to convince
    us that we should be concerned about a score
    (i.e. surrogate markers, e.g. changed blood
    parameters) instead of outcomes like illness or
    death.
  • Holding up worst-case patients experiences as
    representative examples of all with this
    conditions.

41
ECRAN Rating criteria for healthcare journalists
10
  • Criterion will not apply, in case of
  • A therapy/procedure/test/drug possibly being
    overestimated. There is no treatment mongering
    as such. This item should be best scrutinized
    under criterion Quantification of benefit or
    Proper evaluation of quality of evidence.
  • Question to clear at the end of the day
  • Should I really be concerned about this
    disease/circumstance?

42
ECRAN General journalistic criteria
11
  • Choosing the right issue
  • Is it a current, relevant and original
    contribution?
  • Topicality of an issue may be related to the
    event and the frequency of medium publication,
    the importance or the originality (i.e.
    curiosity) of its contents. Relevance may occur
    for instance, if the issue directly or indirectly
    regards a large part of the population, if a
    therapy determines high costs for the health
    system, if a new development has an ethical
    dimension, if the topic has political importance
    or an exemplary nature. If not relevant, an issue
    may instead be curios, humorous, surprising etc.
    Novelty of an issue may be characterized by
    latest data published e.g. today, yesterday, this
    week in a scientific journal, or outcomes of
    recently held conferences.
  • The rule of thumb is
  • If the issue should not be current, then it must
    be relevant. If not relevant, then it must be at
    least curios/original!

43
ECRAN General journalistic criteria
12
  • Delivery of message
  • Has the issue been successfully delivered or
  • does it even have model character for its format?
  • A successful message must be delivered in an
    intelligible language, with a plain structure
    avoiding long and complicated sentences.
    Specialist or technical terms should only be used
    in exceptional cases and, anyway, accordingly
    explained by means of context or examples.
  • Formal factors also play an important role the
    story must follow dramaturgical principles
    (personalisation, narrative elements, relation to
    everyday life) form and content must harmonise
    abstract thinking must be explained by pictures,
    illustrations, clear specimen texts good sound
    clips in a radio report should invite to listen
    to medical issues. In case of reports, WH
    questions should all be answered.
  • Rule of thumb delivery standard is conform with
    the format

44
ECRAN General journalistic criteria
13
  • Factual accuracy
  • Does the story report essential facts?
  • Factual accuracy means reporting historical facts
    without manifest or essential errors, e.g.
    accurate reporting of study conclusions or data,
    correct description of study design (i.e. number
    of participants enrolled/allocated/followed
    up/analysed in clinical trials). Accuracy is
    neither synonymous with objective truth nor
    completeness, e.g. a report must not comprise
    each and every aspect of the study but the
    described facts must be true. A single name error
    is not enough to invalidate this criterion, but
    many little mistakes may create the impression of
    a real sloppy work!
  • Apart from purely data errors or hyperboles in
    the headlines or teaser beware of surrogate
    markers, association vs. correlation, absolute
    vs. relative risks, NNT, off-label use, phases of
    clinical trials, confidence interval etc ...

45
ECRAN Tips for understanding studies Common
pitfalls
  • Surrogate markers
  • A surrogate marker is an event or a laboratory
    value that researchers hope can serve as a
    reliable substitute for an actual disease. A
    common example is blood cholesterol levels which
    are surrogates, or substitutes, for heart
    disease.
  • As a matter of fact, surrogate studies are
    indirect by definition and a journalist should be
    skeptical before celebrating the last medical
    breakthrough, because what is sound medical dogma
    today may fade over time. Surrogate research is
    an important research tool, that can open
    important therapeutic avenues and stimulate
    additional research. Those who earn their living
    and reputation from medical research may have a
    different view on surrogacy, but it is important
    to understand that exaggerations may damage
    patient care by highlighting surrogate outcomes
    instead of important results such as death,
    quality of life or functional capacity.

46
ECRAN Tips for understanding studies Common
pitfalls
  • Association vs correlation and language use
  • Look at this wording describing the results of an
    observational study
  • Frequent fish consumption was associated with a
    50 reduction in the relative risk of dying from
    a heart attack.
  • And look at the editors streamline adaption
    detrimental to meaning
  • Women who ate fish five times a week cut their
    risk of dying later from a heart attack by half.
  • The new wording suggests that the subjects fish
    consumption is responsible for their dying less
    frequently from heart attacks, but the original
    study does not support a conclusion of cause and
    effect.
  • Epidemiologic or observational studies
    examine the association between what is known in
    epidemiologic jargon as an exposure (e.g. a food,
    environmental influences, lifestyle) and an
    outcome (often a disease or death). Because of
    all the other exposures occurring simultaneously
    in the complex lives of free-living humans that
    can never be completely accounted for, such
    studies cannot provide evidence of cause and
    effect they can only provide evidence of some
    relationship (between exposure and outcome) that
    a stronger design could explore further.

47
ECRAN Tips for understanding studies Common
pitfalls
  • The only study design involving humans that does
    rise to the level of demonstrating cause and
    effect is a randomized trial. In this design,
    study subjects are assigned an exposure (or a
    control condition) at random, irrespective of any
    other exposures in their lives, and all such
    other exposures are then assumed to even out
    between the treated group and the control group
    of subjects (and this can be demonstrated). As a
    result, the only difference between the groups is
    whether they receive the exposure under study or
    the control condition. This approach is a true
    experiment. Any difference in outcome seen
    between the control and the experimental group
    should be due to the one factor or variable that
    differs.
  • A subtle trap occurs in the transition from the
    cautious, nondirectional, noncausal, passive
    scientifical language in reporting the results of
    observational studies to the active language used
    in mass media. Active language is fine in general
    (who wants to write like a scientist?), but
    problems can arise when the use of causal
    language is not justified by the study design. In
    practice, a shift to causal language can occur at
    any stage writing, editing, or headline
    composing, with similar effects on meaning, that
    could lead readers to overestimate the meaning of
    a given study and possibly even make life choices
    that the evidence does not warrant. Sometimes
    even scientists and press-release writers slide
    into causal language in expressing results of
    observational studies.  

48
ECRAN Tips for understanding studies Common
pitfalls
  • Absolute vs relative risks
  • Researchers, clinicians or journalists often
    report only relative differences in making claims
    about a new idea, but they should tell the rest
    of the story!
  • Absolute differences make the real sense out of
    such claims.
  • Look at this example about blindness in diabetic
    patients in a 5-year-period
  • If the risk for blindness is 2 in 100 (2) in a
    group of patients treated conventionally and 1 in
    100 (1) in patients treated with a new drug, the
    absolute difference is derived by simply
    subtracting the two risks 2 1 1.
  • Expressed as an absolute difference, the new drug
    reduces the 5-year risk for blindness by 1.
  • The relative difference is the ratio of the two
    risks. Given the data above, the relative
    difference is 1 2 50
  • Expressed as a relative difference, the new drug
    reduces the risk for blindness by half.
  • Data are accurate in both cases, but if you are
    a marketing manager for the new drug, you are
    likely to only use the relative risk reduction
    if you are a journalist, you would better do your
    job by citing the rest of the story (absolute
    risk reduction) often missing in news releases
    and direct-to-consumer prescription drug ads.

49
ECRAN Tips for understanding studies Common
pitfalls
  • Number needed to treat (NNT)
  • The NNT is defined as the inverse of the absolute
    risk reduction or the number of patients who need
    to be treated to prevent one additional bad
    outcome, calculated as 1/absolute risk reduction.
  • Applied to our hypothetical diabetes blindness
    drug example
  • the risk for blindness in a patient with diabetes
    over a 5-year period is 2 in 100 (2) in a group
    of patients treated conventionally and 1 in 100
    (1) in patients treated with a new drug. So the
    absolute difference is derived by simply
    subtracting the two risks 2 1 1.
  • the number needed to treat would be 1 / 1 (or
    .01) 100.
  • so you would need to treat 100 people with
    diabetes for 5 years in order to prevent one case
    of blindness.
  • You can see that this is an important way to look
    at new claims about new drugs.

50
ECRAN Tips for understanding studies Common
pitfalls
  • Off-label use
  • Prescription drugs are officially approved by
    regulatory agencies for specific indications to
    treat specific conditions or diseases. Physicians
    may prescribe a drug for a use that is not
    described in the approved labeling if it seems
    reasonable or appropriate to them. This is whats
    called off-label use.
  • For example it is not uncommon for physicians to
    prescribe low doses of beta blocker drugs to help
    people overcome jitters before public speaking.
    Beta blockers are not formally approved for this
    use, but in such circumstances doctors are
    advised that they have the responsibility to be
    well informed about the product, to base its use
    on firm scientific rationale and on sound medical
    evidence, to inform the patients and get its
    consent for the use of the drug.
  • A study in the Archives of Internal Medicine
    found that about 21 of all estimated uses for
    commonly prescribed medications were off-label,
    and that 15 of all estimated uses lacked
    scientific evidence of any useful or beneficial
    effect. Authors of the study recommended more
    careful monitoring of off-label uses that might
    waste money and harm patients. Another problem of
    the off-label prescribing is that its cost may
    not be reimbursed by public or private health
    insurances.
  • Finally, whilst off-label use is permitted,
    marketing of a drug for an off-label use is not.

51
ECRAN Tips for understanding studies Common
pitfalls
  • Phases of clinical trials
  • Journalists who report on drugs while they are
    still in clinical trials need to understand the
    distinction between phases I, II, and III of drug
    trials. It is misleading to report bold or
    conclusive statements about how well a drug works
    when it is only in phase I trials, since the
    primary goal of phase I trials is to evaluate how
    safe a drug is, not how well it works. Many times
    journalists report on early phase drug trials as
    if all the evidence is in hand.
  • Advice for healthcare journalists
  • Give accurate portrayals of the status of
    investigational drugs, devices and procedures,
    including significant caveats and explanations of
    hurdles, unknown and potential problems. 
  • Without such caveats consumers would be better
    advised to have doubts about the accuracy and
    balance of the story.

52
ECRAN Tips for understanding studies Common
pitfalls
  • Confidence interval
  • In the interpretation of trial results scientists
    divide errors into two classes a type I error is
    the mistake of thinking something is true when it
    is not (also known as a false positive) a type
    II error is thinking something is not true when
    in fact it is (a false negative). When testing a
    specific hypothesis, scientists run statistical
    checks to work out how likely it would be for
    data which seem to support the idea to have come
    about simply by chance. If the likelihood of such
    a false positive conclusion is less than 5, they
    deem the evidence that the hypothesis is truly
    statistically significant. They are thus
    accepting that one result in 20 will be falsely
    positive but one in 20 seems a satisfactory low
    rate.

53
ECRAN EBM guide for healthcare journalists
  • Valuable papers fur further reading
  • This list has been updated at November 2013
  • To find further contents, visit ECRAN website
    www.ecranproject.eu
  • - Opportunities for research and NIH
  • http//www.sciencemag.org/content/327/5961/36.summ
    ary
  • - Demystifying trial networks and network
    meta-analysis
  • http//www.bmj.com/content/346/bmj.f2914
  • - Independent drug testing to ensure drug safety
    and efficacy
  • http//papers.ssrn.com/sol3/papers.cfm?abstract_id
    2328348
  • - Editorial policies for clinical trials and the
    continued changes in medical journalism
  • http//jama.jamanetwork.com/article.aspx?articleid
    1699471
  • - Informing the uninformed optimizing the
    consent message using a fractional factorial
    design
  • http//www.ncbi.nlm.nih.gov/pubmed/23700028

54
ECRAN EBM guide for healthcare journalists
  • - The Womens Health InitiativeA victory for
    women and their health
  • http//jama.jamanetwork.com/article.aspx?articleID
    1745653
  • - Better reporting of scientific studies why it
    matters
  • http//www.plosmedicine.org/article/infodoi/10.13
    71/journal.pmed.1001504
  • - Quantification of harms in cancer screening
    trials literature preview
  • http//www.bmj.com/content/347/bmj.f5334
  • - Risks (and benefits) in comparative
    effectiveness research trials
  • http//www.nejm.org/doi/full/10.1056/NEJMp1309322
  • - The right to participate in high-risk research
  • http//philpapers.org/rec/SHATRT-2
  • - Whos afraid of peer review?
  • http//www.sciencemag.org/content/342/6154/60.full
  • - Why we cant trust clinical guidelines
  • http//www.bmj.com/content/346/bmj.f3830
  • - Practices and impact of primary outcome
    adjustment in randomized controlled trials
    meta-epidemiologic study
  • http//www.bmj.com/content/347/bmj.f4313

55
ECRAN EBM guide for healthcare journalists
  • - US study criticized for experimentation with
    premature infants
  • http//www.bmj.com/content/347/bmj.f4198
  • - Do clinical trials work?
  • http//www.nytimes.com/2013/07/14/opinion/sunday/d
    o-clinical-trials-work.html?pagewantedall_r2
  • - Understanding clinical trials appreciating
    medical heroes
  • http//doc.mediaplanet.com/all_projects/4921.pdf
  • - Dispelling the many myths about clinical trials
  • http//www.newswise.com/articles/dispelling-the-ma
    ny-myths-about-clinical-trials
About PowerShow.com