Data Quality and Data Accreditation

1
Data Quality and Data Accreditation

• Ensuring NHS Information is Fit for Purpose

Andy Gill Data Accreditation Development Manager
2
Themes of presentation

• Why is Data Quality an issue?
• What is Data Accreditation?
• Where we are in the process
• Actual data quality improvements achieved so far.
• Current and Future Developments

3
Data Quality - Foundations

• Think of a beautiful building you know well
• What do its foundations look like?
• Foundations are unseen and un-regarded, until or
  unless things go wrong
• And the costs of remedial action are very high!

4
Why is data quality important?

• Poor quality data
• Undermines confidence in the NHS
• Impairs decision making
• Increases inequity and inequalities
• Increases the risk of adverse events
• Adds to the cost of the NHS

5
The Bristol Enquiry

• Concerns over data quality were raised in the
  Bristol enquiry
• Clearly there is a consistent and on-going
  pattern of poor outcomes (from the CSR data) but
  it is difficult to know what weight should have
  been put on these data at the time with there
  being questions over the data quality 1
• and
• Steps should be taken nationally and locally to
  build the confidence of clinicians in the data
  recorded in the Patient Administration Systems in
  trusts 2
• Kennedy Report - Learning from Bristol
• 1 Chapter 20 section 23
• 2 Recommendation 149

6
Why do we need standards and audit/assessment?

• Because to quote
• Quality is never an accident it is always the
  result of intelligent effort.
• John Ruskin Theologian, Art Critic and Social
  Commentator
• (1819 - 1900)

(no rowlocks)
7
Data AccreditationDelivery

• Data Quality, Data Accreditation

8
What is Data Accreditation?

• It is a tool detailing minimum standards and good
  practice guidance to improve data quality in the
  NHS.
• It is designed to enable NHS organisations to
  establish good quality information through clear
  policies and procedures
• It is also a monitoring tool for organisations to
  demonstrate that they have the correct
  infrastructure and systems to promote data
  quality.

9
Features of Data Accreditation

• It supports a fundamental review of procedures
  and systems
• Promotes process redesign through clear minimum
  Standards and good-practice guidelines
• Provides a measure of information management
  performance
• Three stage incremental process
• Makes use of external assessments
• It leads to a nationally recognised award

10
Achieving Data Accreditation
11
The Ten Data Accreditation Criteria

• Security and Confidentiality
• Coverage
• Validation and Quality Assurance
• Training
• Accountability
• Health Records Management
• Communications
• Completeness and Validity
• Timeliness
• Accuracy

12
The Checklist Review

• It is a brief, high-level review, including
  questionnaire for key collectors, managers and
  data users.
• Purpose - to assess current data management
  performance and data quality it will provide a
  basis from which improvements can be planned.

13
The Process Review

• It is
• A detailed review of processes, policies and
  procedures
• Establishes if practice matches procedures
• Establishes if good practice is being observed.
• Its Purpose is
• To assess data management practices against
  Quality Criteria standards, and
• to identify remedial actions needed.

14
The Compliance Matrix Vs 1
15
The Outputs Review

• Also a two-part stage comprising
• Internal Review of the three Data Quality
  Standards
• External Review by an outside assessor to ensure
  conformance
• Leads to Certification of each Data Group.
• Based on a minimum one month sample.
• Remember that full Data Accreditation can only be
  given when both Processes and Outputs are
  Certificated.

16
The Three Data Quality Criteria at Outputs Review

• Completeness and validity
• Timeliness
• Accuracy.

17
Completeness and Validity

• One months data assessed by computer analysis
  taken from provider system or download of CDS
  extract
• Looking for
• Incomplete fields
• Empty fields
• Invalid codes
• Duplicate records
• Further testing for
• Dump codes (not known)
• Abnormally high frequency of invalid codes
  (especially postcodes or diagnosis codes)
• Cross-reference validation (men on Gynae wards)
• Reconciling databases.

18
Timeliness

• Is information complete in time to support
  internal, local and national processes?
• Is there a clear process map for the production
  of information within agreed timescales and
  analysis of the key points in the chain?

19
Accuracy

• Historically tested by measuring electronic data
  against casenotes
• Casenotes no longer a primary source of
  information in many trusts
• How to measure the accuracy of a truly electronic
  record?
• Check with Patients?

20
Where we are in the process

• Stage One The Checklist Review.
• Stage Two The Review of Management Processes
• Internal Assessment
• External Assessment
• Stage Three The Review of Outputs for
  Inpatients, Waiting List, Patients Charter and
  Outpatients
• Internal Assessment
• External Assessment

21
(No Transcript)
22
Quality Improvements, so far

• Documented Policies and Procedures
• Differences between assumed and actual practice
• Involvement of staff in process redesign
• Greater emphasis given to supervision and
  checking of data quality input
• Feedback on poor quality linked to (re-) training
• Improved education of staff (why we do things,
  terminology explained, understanding of data
  chains, etc)
• Actual improvements in collection of data (ethnic
  codes)
• Some anecdotes
• A Data Quality Structure with defined
  responsibilities
• Data Quality Leads nominated (LIS Toolkit)
• Data Quality Policy
• Audit Commission and CHI Audits are made easier!

23
Data AccreditationDevelopment

• Making Data Accreditation Fit for Purpose

24
The Challenge

• To develop an accreditation system that gives
  everyone involved in the provision and
  interpretation of patient information confidence
  that NHS data is reliable
• and
• where it isnt, that this is recognised and
  standards for improvement are set and readily
  accessible
• and
• that the system is easy to understand and apply
• and
• that eventually all nationally recognised NHS
  datasets will be covered by the process.

25
The Solution work underway

• Work is already underway to revise the current
  Data Accreditation documentation for Acute Trust
  Data Outputs in the light of changes in NHS
  guidance etc. New draft guidance has been
  circulated and has been disseminated to users for
  Quality Review.
• Work is also underway to consult with clinicians
  and managers on the development of the Data
  Accreditation Checklist Review and Data
  Management Process Review for PCTs, Mental Health
  Trusts and GP practices.
• We have also started work with the Cancer
  Datasets Team (specific Checklist Review already
  produced) and the Royal College of Physicians to
  look at how the Data Accreditation Process can
  give them reassurances about the quality of data
  they receive.

26
The Solution work underway(2)

• Data Accreditation has become part of the DoHs
  Information Governance Programme. (see later)
• Working with the Modernisation Agency to apply
  the Data Accreditation methodology to the Review
  of Waiting and Booking Information (RoWBI).
• Working with HES and DQRS to ensure that the Data
  Accreditation process can make effective use of
  the reports on data quality that trusts already
  receive.
• Links with Audit Commission established
• Links developing with other organisations such as
  CASPE, UKAF.

27
Data Accreditation Toolkit

• In the longer term, it is planned to develop a
  Data Accreditation Toolkit that can be applied to
  clinical and administrative dataset modules.
• These will build on the Data Accreditation
  principles, but will allow those developing the
  datasets to tailor some of the standards to their
  specific requirements - (i.e. Security standards,
  health records standards, general training etc
  would be uniform, but validation methodology,
  timeliness requirements, completeness of fields
  etc could be set individually for different
  datasets).
• Potentially in the form of a core document with
  core standards and a series of modules covering
  all data items in standardised format, but
  specific to individual datasets. Thus when
  updates to datasets or standards occur, these
  modules can be updated, without the need for
  wholesale changes.

28
Potential Data Accreditation Modules Acute
Core Data
Patient Demographics
Core Processes
Core Outputs APC WL OP AE
Other datasets
Security Conf. Health Records Accountability
Training Communications Coverage
(a) Validation QA (a)
Coverage (b) Validation QA (b) Completeness
Validity Timeliness Accuracy
Diabetes NSF
CHD NSF
Cancer Audit
Cancer Waiting Times
29
Possible Data Accreditation Modules - PCT
Core Data
Patient Demographics
Core Processes
Core Outputs Registration History Contacts
Other datasets
Security Conf. Health Records Accountability
Training (a) Communications (a) Coverage
(a) Validation QA (a)
Training (b) Communications (b) Coverage
(b) Validation QA (b) Completeness
Validity Timeliness Accuracy
Community Hospitals
Referrals and Requests
Podiatry
Prescribing
30
Possible Data Accreditation Modules M.H.
Core Data
Patient Demographics
Core Processes
Core Outputs Registration CPA MDS
Other datasets
Security Conf. Health Records Accountability
Training (a) Communications (a) Coverage
(a) Validation QA (a)
Training (b) Communications (b) Coverage
(b) Validation QA (b) Completeness
Validity Timeliness Accuracy
Community Contacts
Prescribing
Hospital Admissions
Interventions Treatments
31
Closing quote

• Fast is fine, but accuracy is everything.
• Wyatt Earp US Marshall (1848 1929)

32
Web Addresses

• Data Accreditation website
• www.nhsia.nhs.uk/dataquality/pages/accredit.asp
• Or on NHSNet at
• nww.nhsia.nhs.uk/dataquality/pages/accredit.asp