Is that an ADaM dataset on the Janus wall? The Humpty-Dumpty challenge of modeling study data with HL7-RIM

1
Is that an ADaM dataset on the Janus
wall?The Humpty-Dumpty challenge of modeling
study data with HL7-RIM
or SDTM V

• Dana Soloff
• Director, Statistical Programming, Genzyme
• September 25, 2009
• BACUN Boston Area CDISC Users Network

2
Caveats

• This presentation represents my thoughts, and not
  necessarily those of Genzyme.

3
Outline

• Background
• Our ADaM-HL7 Pilot
• What we did
• Our motivation
• What we learned
• What do we (pharma) do next?
• Our final analysis

4
CDISC-HL7 is coming!

• The FDA has embraced the HL7-RIM
• We envision the CDISC content to
• be sent to FDA as XML messages based
• on the HL7-RIM
• SDTM will evolve from a
• submission standard to an analysis view
• BEHRMAN 2008

5
(No Transcript)
6

When is it coming?!!
balloted 9/2009
PDUFA Prescription Drug User Fee Act. Oliva,
March 2009.
7
Genzyme ADaM-HL7 Pilot

• We hired consultants!
• We trained on HL7 V3!
• We tried to model ADaM to HL7!
• We gave up!
• (But Genzyme is trying again!)

8
What are the CDISC-HL7 Messages?

• Study Design
• What will be done?
• Study Participation
• Who is involved?
• Subject Data
• What was observed?
• Includes analysis data
• Isnt a message anymore

Has published Domain Analyisis Model (DAM)
(Think CDISC I.G.) and passed DSTU ballot 9/09.
9
Study Data Now CDANo DAM BRIDG?
It just says Document
10
Why? We were curious.

• What is CDISC-HL7?
• Why is the FDA doing this?
• Will this impact statistical reviews?
• How will we get our SDTM and ADaM data into
  CDISC-HL7?
• Does this change our vision for data standards
  architecture?

11
SDTM ADaM Traditional datasets

• Two dimensional rows and columns
• Keys relate datasets
• Requires human readable metadata
• Relationships between values often implicit

12
What is the impact of implicit relationships?

• Analysis Dataset
• One observation row
• Value for concomitant medication
• Value for adverse event
• Did the conmed cause the stroke?
• Or was the conmed administered because of the
  stroke?

13
Specifications Match Structure of Datasets
SDTMIG 3.1.2
SDTM 3.1.2
14
CDISC-HL7

• Break each dataset up by variables and values
• Elements floating in multi-dimensional space
• Wrapped in little pods of metadata called
  attributes
• Explicitly modeled relationships

15
Protocol Representation Model
BRIDG 2.2
16
What does it mean to map SDTM to BRIDG?
(apologies to Diane Wold)
17
What is CDISC-HL7?

• Its not just SDTM reformatted
• Its a very big change
• We have a lot to learn
• The standards require further development
• Complexity and our inexperience constrains our
  effective participation

18
Our questions

• What is CDISC-HL7?
• Why is the FDA doing this?
• Will this impact statistical reviews?
• How will we get our SDTM and ADaM data into
  CDISC-HL7?
• Does this change our vision for data standards
  architecture?

19
The FDA wants more information on relationships
between data

• This adverse event was the result of this
  concomitant medication administered by this
  investigator on this date in response to this lab
  value
• Great for medical review!
• Can be modeled in web browser and no limit to
  instant clicking around to understand
  relationships!

20
(No Transcript)
21
Pharma needs to join healthcare

• EHR (electronic health record) is HL7 based
• Efficiencies, reduced development time using same
  source data
• Potential to combine sponsor clinical trial data
  with subjects healthcare record data
• Personalized Medicine Genzyme!

22
Why analysis data in HL7?

• Combine statistical results with point of care
  statistical results? No
• Combine complex study-specific derived values
  across sponsors? No
• Statistical analysis is performed on groups of
  observations
• Healthcare is performed on individuals
• Linking the two is tough!

23
What is the main reason provided in the HL7
subject data use cases? Transparency.

• Reviewers want more transparency between
  collected data and results
• They want derived data and collected data
  together
• They want to be able to easily identify which
  observations we imputed, excluded, etc.

24
Why not put analysis data on SDTM?Theres no
place to put it.
Response Mean weekly lab gt 5 units, no rescue
therapies, no adverse events of interest over the
evaluation period
25
HL7 could theoretically solve this.

• Response

Findings
Events
Interventions
26
Except there are interim calculations

• Data handling and algorithms applied at every
  step
• Imputations, selected observations based on
  values or time windows
• Often comparisons to other variables before
  choosing or calculating value
• Last follow up date could come from AE, EOS, LB,
  etc.

27
RESPONSE
Rate of change
Adverse Events Selected
ATC Codes
Transfusion
P
Mean Lab
A
P
28
The FDA also wants to compare actual to planned!

• Protocol was amended four times!
• A lot of unplanned things happened
• New drugs came on the market
• Sick people didnt make it to scheduled visits
• Trials werent executed perfectly
• Bizarre data values happened
• Some samples were incorrectly analyzed

29
And then we get busy

• Perform same calculations on different
  populations
• ITT, Per Protocol
• And by different imputation methods
• LOCF, WOCF
• We may plan to use an observation in one analyses
  and not another

30
Analysis data is different than collected data!

• Real surgery doesnt have do-overs!
• ITT, Per Protocol, Safety
• More complexity and diversity in modeling
  statistics
• Entities, acts, etc. dont always make sense

31
Can we model ADaM in HL7?

• Certainly not today!
• It might be possible in the future
• There will always be considerable room for error
• Is there too MUCH information?
• Is HL7 the best way to provide more transparency
  to reviewers?
• Is the cost-benefit ratio acceptable?

32
Our questions

• What is CDISC-HL7?
• Why is the FDA doing this?
• Will this impact statistical reviews?
• How will we get our SDTM and ADaM into CDISC-HL7?
• Does this change our vision for data standards
  architecture?

33
The data are submitted and the fun begins!

• FDA receives HL7 messages
• Janus generates
• views of SDTM and ADaM that match ours
• additional analysis views with both collected and
  derived data

34
Would the FDAs views of SDTM and ADaM match ours?

• SDTM and ADaM allow flexibility in modeling
• How can one model from the specific to the
  general without a human or rules?
• One will never have standard messages defined to
  cover all cases

35
Will the FDA reviewer use our datasets or theirs?

• SDTM is a collected data standard
• Original error was assuming that SDTM could ever
  be basis for statistical review
• If reviewers are unhappy that there is no
  analysis data on SDTM
• And sponsors are required to model data in HL7
  because SDTM is inadequate
• And reviewers have access to another view than
  SDTM that includes analysis data created from
  HL7
• Why would they use SDTM?
• Other than for WebSDM, iReview

36
Will views be reassembled correctly? The
Humpty-Dumpty Problem!

• The data and the metadata are in pieces!
• Some is part of HL7 attributes
• The rest is in our black box
• How will they put together an accurate view of
  the analysis data?
• Our metadata define.xml wont document their
  view
• Will derived variables be used incorrectly when
  used out of dataset context?

37
There will be challenges for the FDA and sponsor
communication

• If we have different input datasets
• Or the Janus generated views are not accurate
• How will this promote transparency with regard to
  statistical review?
• It might help if FDA reviewers provide sponsors
  with their analysis data views
• We need define.xml!
• And ODM format! ?

38
Is all that really better than this?
Selection criteria described in define.xml
39
Our questions

• What is CDISC-HL7?
• Why is the FDA doing this?
• Will this impact statistical reviews?
• How will we get our SDTM and ADaM data into
  CDISC-HL7?
• Does this change our vision for data standards
  architecture?

40
Who prepares the submission?
Study, Data Analysis SME
CDISC-HL7 SME
41
Understanding trial, data content and analyses
key to correct modeling

• Relationships between observed and calculated not
  all captured as data
• Until we have structured protocol SAP, a fairly
  complete set of robust messages, and maybe even
  then
• We need SMEs to model data
• They dont have the HL7 expertise

42
How do we QC the result?

• Is this double work?
• Complex specs for ADaM datasets
• Complex specs for HL7
• Complex specs to reassemble ADaM from HL7
• Double-program pre-HL7 ADaM with post-HL7?

43
Our questions

• What is CDISC-HL7?
• Why is the FDA doing this?
• Will this impact statistical reviews?
• How will we get our SDTM and ADaM data into
  CDISC-HL7?
• Does this change our vision for data standards
  architecture?

44
Almost everything weve done has been valuable!
All are plans are usable!

• Governance
• End-to-end metadata driven data standards roadmap
• Metadata Repository
• Structured Protocol
• Central Lab Standard
• CDASH based collection standards

45
Where should we go from here?ADaM may be better
than HL7 in providing transparency

• Be realistic about what SDTM can do
• Its fine as a collected data standard
• But not as a base for FDA analysis review
• Implement ADaM 2.1 and ADaMIG 1.0!
• Improve our metadata (define.xml)
• Err on the side of traceability!
• Inclusion of SDTM data a priority
• Intermediate datasets when helpful
• Provide FDA multiple views of the same data
• Provide FDA helper variables for analysis

46
What can we learn from our SDTM experience?

• Less time spent developing model
• More time testing actual data!
• Engage FDA to understand and develop joint vision
• Collaborate with each other on tool development
  and share costs
• Pitch into fund to hire HL7 technical lobbyist

47
What else should sponsors do?

• Buy a really big color printer!
• Buy really big paper!
• Buy a really big magnifying glass!
• Buy a big HL7 warehouse! (eventually)

48
CDISC-HL7 Current sentiment heard around town

• HL7 may make sense for collected data - but we
  dont like it!
• - and we still need SDTM as a base for ADaM
• ODM makes more sense for analysis data
• - until we have proof that HL7 satisfies the use
  cases

49
Is HL7 TOO MUCH information?

• The world is round and we do need a jet for
  collected data
• But are you sure we should take our jet to the
  ADaM grocery store?
• Lets give pharma a chance to upgrade to more
  robust ADaM!

50
HL7 is coming!
This isnt right!
51
HL7 is Coming!
Nor is this.
52
There are open questions.We all must work
together to succeed
Open-minded, science-based, problem-solving
Learn UML
Ignore it
Repudiation
Get HL7 Tattoo
Blind Faith
Make dartboard out of UML
HL7 response meter
53
Thank you!

• Questions?