Introduction to Regulatory Statistical Principles - PowerPoint PPT Presentation

1 / 51
About This Presentation
Title:

Introduction to Regulatory Statistical Principles

Description:

... e.g. SAS offers several definitions of percentiles, ... If you wish to license a product with one trial, you should contact the agency and discuss the case . – PowerPoint PPT presentation

Number of Views:160
Avg rating:3.0/5.0
Slides: 52
Provided by: PeterLac
Category:

less

Transcript and Presenter's Notes

Title: Introduction to Regulatory Statistical Principles


1
Introduction to Regulatory Statistical Principles
  • Peter A. Lachenbruch
  • Oregon State University
  • College of Public Health and Human Sciences (Ret.)

2
Conclusion
  • There can be few areas where the discipline of
    statistics is conducted with greater discipline.
    Pharmaceutical statisticians may be engaged in
    work that may sometimes involve routine
    calculations, but the regular application of
    statistical principles to produce high quality
    experiments, data and analysis promotes a
    professionalism that can itself be a source of
    satisfaction.
  • Stephen Senn (2000 The Statistician)

3
Research vs. Development
  • Planned development vs. Exploratory analyses
  • Implications regarding significance tests
  • Journal Publication or Licensing
  • Understanding a biological process or
    demonstrating a therapeutic benefit
  • Assess efficacy and Safety
  • Submit data for review by regulators

4
The Cast of Characters
  • Sponsor the entity that will market the
    product, be responsible for writing the marketing
    application (NDA, BLA etc.)
  • Usually have many skills - Biology,
    Biochemistry, Medicine, Regulatory Affairs,
    Biostatistics, etc.
  • These people have roles that change over time
    biologists in pre-clinical times, biostatistics
    in clinical phases and pre-clinical, etc.
  • May be drug company, university, government
    agency, etc.

5
Characters (2)
  • Investigators the scientists who conduct the
    trials. Must demonstrate their qualifications.
    May be physicians, biochemists, etc. Employees
    of sponsor or contracted to sponsor (e.g., a
    University scientist), CRO (contract research
    organization)
  • May also be independent scientists e.g., many
    statisticians serve as consultants to drug
    companies

6
Characters (3)
  • Regulatory agency in US the FDA plays this
    role. Has 6 centers CBER, CDER, CDRH, CVM,
    CFSAN, NCTR. Most of what we will talk of here
    is related to the first three (Biologics, Drugs
    and Devices)
  • Requirements are governed by the authorizing laws
    and regulations that have evolved over many
    years. See the Code of Federal Regulations (CFR)

7
Two trials needed
  • For many years, the FDA required two adequate
    and well-controlled trials that showed clinical
    benefit to license a drug or biologic. The sole
    exception was vaccines which usually were studied
    in large (ngt5000) trials.
  • Recently, the agency has adopted a more flexible
    approach, especially in rare diseases and
    conditions in which obtaining enough patients for
    two trials would be difficult.
  • If you wish to license a product with one trial,
    you should contact the agency and discuss the
    case .

8
ICH
  • The International Conference on Harmonization
    (ICH) is an effort to require the same standard
    of evidence for licensure in various regions of
    the world.
  • Joint by US, Europe, and Japan regulatory
    agencies and by the pharmaceutical industry
    organizations in those regions.
  • You can find links to these at www.ich.org

9
Phases of Drug Development
  • Pre-clinical
  • Prior to the first human study, the sponsor must
    show evidence that the product does not have
    apparent unacceptable risks for humans
  • Done in animals rats, mice
  • Carcinogenicity, teratogenicity
  • Stability, shelf-life, potency, contaminant
    detection
  • Studies of Maximum Tolerated Dose, route of
    administration, frequency of administration (also
    done in phase 1 studies
  • ICH S series (safety) and Q series (quality)

10
Phase 1 Studies
  • Must have a valid Investigational New Drug
    authorization allows sponsor to conduct studies
    in human populations
  • Find MTD
  • Show a therapeutic response in a wide enough dose
    range that the product can be used safely
  • Primarily safety studies. May be in non-diseased
    population or a diseased population. Depends on
    the drug.
  • Often not randomized

11
Phase 2 Studies
  • Learn more about proper dose, schedule of
    administration, route of administration (oral,
    subcutaneous, intravenous)
  • Vaccines are frequently single dose, sometimes a
    booster dose is given
  • Therapeutic drugs may be given multiple times
  • Some products are given by multiple routes (at
    physician discretion)
  • Pain medications usually given until relief is
    obtained, so concern about cumulative dose
  • Chronic medication e.g., cholesterol lowering
    medication, cardiovascular meds, diabetes
    medication may be given for long periods
    lifetime. Must monitor for problems.

12
Phase 2, continued
  • Must pre-specify response/outcome
  • Cant collect many outcomes and select one that
    seems to work well. This is exploratory analysis
    and is never accepted by regulators.
  • Analysis plan must be specified
  • Report to regulatory agency of results, including
    safety, efficacy, other data.
  • It is wise to study several doses, routes of
    administration, and schedules of administration
  • Sometimes there is information from products of a
    similar class that will assist in these studies

13
Phase 3 Studies
  • Must show superiority of the product
  • Two trials usually needed
  • Prefer there to be two different populations
    e.g. tertiary care and primary care centers
  • Need reports for each trial often detailed
    information on patients is given.
  • The statistical analysis plan (SAP) must be
    specified and shouldnt be changed unless there
    is new evidence of changes in the endpoints.
    Changes after the data have been locked are
    highly suspicious.

14
Working with Regulators (with reference to FDA)
  • Goal of both sponsor and FDA is to bring safe,
    effective products to market
  • Sponsors want to present their product in the
    most favorable light (sometimes downplaying the
    bad things)
  • Regulators (FDA especially) are cautious and
    dont want to approve a product that is
    ineffective or has a poor balance of benefit and
    risk.

15
Regulatory work (2)
  • Note that regulatory agencies are not monoliths
  • Advice may differ based on which center,
    division, or branch you deal with sometimes
    reviewers will affect the advice
  • Advice you get may depend on what the favored
    research paradigm is, approach, custom of
    division/branch e.g., how missing values are
    treated
  • Note that you can appeal the decision if you
    differ from the reviewer

16
Regulatory work (3)
  • Evidence standard
  • Two convincing trials p-value below 0.05
  • What if the two convincing trials happen after 10
    attempts?
  • Can use a negative binomial to suggest this isnt
    persuasive. I find the probability of 2
    successes by the 10th trial is 0.0186 under the
    null hypothesis
  • If p(success ) is 0.8, the probability of 2
    successful trials by 4 trials is 0.97
  • Usually need to show superiority, rather than
    non-inferiority although safer products can be
    persuasive

17
Regulatory work (4)
  • Single trial can be accepted if
  • A very large trial such as a vaccine trial (tens
    of thousands of observations)
  • A very serious and rare disease is being studied
  • Speak to the regulators! Many sponsors are
    reluctant to discuss their concerns with the FDA
    since they seem to fear that if they tell some
    problem, the FDA will focus on it.
  • This is a recipe for disasters. Its better to
    avoid problems in interpretation early than have
    a fight later on.

18
Some suggestions
  • Know your reviewers
  • Perspective
  • Listen to any advice they offer and make
    counter-proposals if appropriate reviewers will
    listen. They may not agree
  • Reviewers may change during your study its
    important to have a record of any agreements
    dont rely on memory.
  • Dont rely on biological arguments clinical
    trials may contradict the biological argument

19
Types of Paperwork
  • IND or IDE
  • The application to conduct a clinical trial. No
    trial can proceed without one. Can be amended
  • Outline studies to be conducted, modifications to
    ongoing studies, where the study will be
    conducted, when it will be, etc.
  • What drugs, dosage, schedules, route of
    administration will be examined, etc.
  • Specify statistical analysis plan at an early
    amendment
  • Non-standard analyses can be proposed should be
    justified
  • Any IND can be amended studies are not cast in
    stone.

20
Paperwork (2)
  • One-arm studies (no control) are usually
    unacceptable
  • FDA has seen too many trials that have the single
    arm later shown to be inferior to a control arm
  • Non-randomized studies are notorious for having
    bias the investigator knows what drug is given,
    rates the patients more highly than in a
    controlled trial.

21
Paperwork (3)
  • A famous study (Moertel) extracted results from
    many oncology studies and found the
    non-randomized ones always showed high response
    rates, while the randomized ones were much
    poorer. Some studies of this type have been
    accepted when the population is small and the
    natural history of the disease is well known.

22
Paperwork (4)
  • Make use of ICH guidelines and FDA guidelines.
  • These are not mandatory, but represent best
    current thinking in their areas. Alternatives
    can be proposed.
  • Most important for clinical trials are E3, E9,
    E10
  • Generalizing to a population
  • Clinical trials are conducted on a small subset
    of a population. In order for a product to be
    approved for marketing, FDA and other agencies
    want to be convinced that the results apply to
    the broader population. For this reason,
    sponsors either use broad entry criteria, or
    have multiple studies on different populations
  • This also includes patients or subjects of
    different ages, different physical conditions, or
    different hospitals.

23
Paperwork (5) Marketing applications
  • NDA and BLA are applications to market a new
    product. NDAs are used by CDER (center for
    drugs) and BLAs are used by CBER (center for
    biologics)
  • The BLA requires close scrutiny of the
    manufacturing process and facilities, because of
    the greater variability of biological products.
  • The application will include complete reports on
    all studies including patient listings, analysis
    according to the original statistical analysis
    protocol (SAP) as well as any exploratory
    analyses.

24
Paperwork (6)Marketing Applications
  • Full safety analyses
  • Listing of all adverse events
  • By organ system
  • By frequency
  • Since neither FDA nor sponsors can define what
    events will occur a priori, such analyses tend to
    be exploratory. Some direction may be found in
    other drugs of the same class (e.g., a new statin
    or bet blocker, a new vaccine, etc.)

25
Regulators needs/wants
  • Well-written good grammar, concise,
    well-organized
  • Correct relevant tables, proper statistics
    (e.g. no women in prostate cancer denominators,
    etc.) no adults in tables referring to pediatric
    applications
  • Proper number of digits (no spurious precision)
  • Data set noted as source for each table/graph or
    analysis

26
Regulators needs (2)
  • Justify any statements data should demonstrate
    this.
  • Be open and honest about the data
  • If there are problems, discuss them show the
    warts as well as the beauty.
  • Do not lie about the data if an analysis is
    determined after the data has been locked, it can
    be treated as exploratory. FDA will usually not
    accept discovered hypotheses.

27
Research paper vs. Marketing Application
  • In research, the goal is to have a good paper
    published in a good journal. The author(s) may
    contact the editor to ascertain interest in the
    topic. The paper itself will rarely contain the
    data and the analysis will not be reproduced by
    referees.
  • For a marketing application, the sponsor will
    have been working with the regulatory agency for
    years and will submit all data. The analysis
    will be reproduced by the reviewers.

28
Research vs. Marketing (2)
  • For a marketing application, FDA and sponsor may
    have agreed on some points. These generally
    arent binding on either party. It is similar
    to the journal in either case, the journal may
    decide it isnt appropriate for the journal the
    regulatory agency may decide it isnt adequate
    for licensure if the data dont show convincing
    evidence of efficacy
  • The ultimate goal in regulation is to write a
    clear, good label for the product that the
    prescriber and public can understand

29
Reasons a marketing application may fail to be
approved
  • The application is not reviewable data arent
    adequate, application is incomplete (sections
    missing)
  • Data do not show evidence of efficacy
  • There are safety concerns even if efficacy is
    demonstrated in one case I know of, there were
    serious safety concerns but a blinded review of
    the data indicated that these were not the
    problem the investigators thought they were.

30
Options if Marketing Application is not Approved
  • Respond to criticisms
  • Appeal to Office Director or Center Director
  • Appeal to FDA Ombudsman
  • Have advocacy group place pressure on FDA
  • Have congressman put pressure
  • The last two are rarely effective since science
    usually has left the debate
  • Conduct a new study

31
Understanding FDA language
  • Must is directive no options to do otherwise
  • No means no
  • Recommend is not directive you should discuss
    your plans in detail
  • Should is strong, but less directive than must.
  • Generally, FDA will not comment on something
    without reviewing the data
  • Show clinical benefit usually means they are not
    interested in means.

32
FDA language(2)
  • Time issues
  • What happens at k weeks is different from what
    happens by k weeks or within k weeks
  • If confused, ask!
  • In reporting, define what you mean.
  • clinically meaningful improvement must be
    carefully defined and not rely on a physicians
    impression
  • Be careful about interpreting FDA words wanting
    a sensitivity analysis means FDA wants to be sure
    small changes in assumptions dont affect the
    conclusions.

33
FDA language (3)
  • FDA will often look at subgroups to ensure the
    findings are robust. This is not doing lots of
    subgroup analyses to find something it is more
    looking at various groups to ensure a finding is
    real.
  • In one study, a sponsor had found an effect, but
    FDA did some subgroup analysis and found 3 of 4
    sites had no effect and one had a major effect.
    Subsequent investigation found that the study
    coordinator for the site in question had
    unblinded the randomization and gave the
    treatment to patients she thought would have a
    better chance of benefit.

34
Interacting with Regulators
  • Dont ignore advice the regulators have likely
    seen similar products and know pitfalls
  • Make your submissions clear ensure your
    analysis answers the questions fully. If they
    differ from advice, explain why
  • Ensure your data are complete and consistent
  • Have a plan for dealing with missing values
  • Always present the pre-specified analyses and any
    others you have done.
  • Analyses not pre-specified will be considered
    exploratory

35
Interacting with Regulators (2)
  • In one example, I had been working on a 2 part
    model theory and a sponsor had a study that I
    thought was appropriate for the application.
  • I suggested it and the regulatory affairs person
    said well do it!
  • I responded that the sponsor should check it out
    to see if it was appropriate.
  • A week or two later, the statisticians had run a
    small simulation study and decided that a
    Mantel-Haenszel test would work better.

36
Interactions with Regulators (3)
  • Communicate with regulators
  • Some sponsors ban such communications unless a
    regulatory affairs representative is present or
    on the phone can result in delays
  • Often the regulatory affairs person does not
    understand the statistical details being
    discussed
  • This is a message for the regulatory affairs
    people dont delay work to maintain your
    control trust your statisticians and other
    staff.

37
Interactions with Regulators (4)
  • Working within a system
  • Concurrent documentation of procedures and
    studies
  • Include data cleaning and editing procedures
  • Data analysis methods and formulas
  • SOPs for lab tests and all measurements of
    patients
  • Normal ranges for key variables
  • How various labs were standardized

38
Interactions with Regulators (5)
  • Definitions
  • Treatments including dose, schedule, route
  • Population being studied
  • Null and alternative hypotheses
  • Tests to be conducted
  • Size of tests
  • Power of tests at alternatives
  • Guidances and Regulations, ICH

39
Contents of IND
  • For a phase 3 trial
  • Design of study often involves two group
    comparisons superiority vs. non-inferiority
  • Inclusion and exclusion criteria
  • Outcomes (pre-specified!), how measured, how
    standardized across investigators
  • Covariables (not too many remember label must
    be fairly general) phase 3 is NOT the time for
    variable selection
  • The covariates should be predictive of outcome,
    not of imbalance.

40
IND (2)
  • Sample size computation often its good to give
    several alternatives for different size, power
    and effect sizes. Give details
  • Distinguish between non-inferiority and
    superiority trials/analysis

41
IND (3)
  • Assumptions involved in analysis
  • How will missing values be handled (maybe have
    several methods)
  • I prefer not using LOCF
  • Multiple imputation
  • Model based methods likelihood, mixed models,
    GEE
  • Details of randomization within site, balancing
    (minimization)
  • Proposed analysis equations, references,
    justification of effect size based on pilot
    studies, the literature, etc.

42
IND (5)
  • Modifications of a study
  • Its almost always done as details change e.g.,
    recruiting is slow, so inclusion criteria may be
    broadened (age, disease inclusion, extending
    duration of recruitment)
  • SAP can be changed as long as data are still
    blinded to sponsor.
  • Changing endpoints is possible early on, but
    after data are complete, changing becomes suspect
    even if sponsor swears they havent peeked.

43
IND (5)
  • Documenting work- YES
  • Data cleaning and editing steps if programs are
    used, give them regulator may not examine, but
    can be useful
  • Acceptable ranges for covariables and endpoints
  • Handling if out of range (go to site and check?)
  • Handling inconsistencies in data
  • Software used some differences due to different
    algorithms e.g. SAS offers several definitions
    of percentiles, Stata does not
  • Distinguish between exploratory and confirmatory
    generally not helpful to have p-values with
    exploratory analyses

44
Meeting the Regulators
  • Prepare a briefing book describing issues to be
    discussed
  • Dont bring up new results at the meeting as the
    FDA will say to submit the information and they
    will respond. Extreme example of a consultant
    presenting new, un-reviewed data at an advisory
    committee meeting.
  • FDA will answer questions, often close to the
    meeting date making it difficult to reply to
    their concerns

45
Meetings (2)
  • Useless question
  • Does the FDA agree that these results support
    licensure?
  • Better to ask what additional analyses are
    needed to support licensure?
  • More helpful discussions related to schedule of
    submissions, etc.

46
Meetings (3)
  • Type A meetings when drug development is
    stalled and sponsor cant proceed without FDA
    input e.g., clinical hold (need to know what
    is needed to remove the hold), or to resolve a
    dispute between sponsor and FDA, or for a Special
    Protocol Assessment.
  • FDA will schedule a type A meeting within 30 days
    of receipt of a written request

47
Meetings (4)
  • Type B meetings occur at natural junctures during
    the course of drug development e.g. pre-IND,
    end of phase 1, end of phase 2-pre phase 3,
    pre-NDA/BLA
  • Pose clear questions based on interaction with
    FDA, problems observed during study
  • Type C meetings are all other meetings regarding
    the development and review of a product.

48
Meetings (5)
  • Regulators want to approve products but only on
    the basis of convincing data.
  • Sponsors will discuss issues before the meeting
    (dress rehearsals)
  • The FDA review team will also discus issues in
    each of the areas safety, efficacy, quality
  • Note that FDA reviewers are privy to information
    from other studies in the same class and comments
    may be related to problems observed in other
    products. The FDA cant disclose specifics, but
    sponsors should be aware that issues mentioned
    may be relevant to them.

49
A Few Statistical Issues
  • Recent developments and methodological research
    of importance to FDA
  • Incorporating Bayesian methods into drug approval
    informativeness of priors. CDRH has accepted
    these for years.
  • Genomics large number of variables, relatively
    few patients. What standards should be used for
    approval? License a process or a product? Seems
    to suggest using theory rather than clinical data

50
A Few Statistical Issues (2)
  • Selecting the margin for non-inferiority studies
    (Im not sure where this is at present)
  • Robust methods for complex statistical models
    (e.g., hierarchical models for non-normal data)
  • Follow on Biologics
  • Biologics equivalent to generics since
    Biologics are much more variable than drugs, many
    tough issues
  • Adaptive Methods

51
Conclusion
  • There can be few areas where the discipline of
    statistics is conducted with greater discipline.
    Pharmaceutical statisticians may be engaged in
    work that may sometimes involve routine
    calculations, but the regular application of
    statistical principles to produce high quality
    experiments, data and analysis promotes a
    professionalism that can itself be a source of
    satisfaction.
  • Stephen Senn (2000 The Statistician)
Write a Comment
User Comments (0)
About PowerShow.com