Title: Evaluation of the Scottish National Telecare Development Programme Findings and implications for future research
1Evaluation of the Scottish National Telecare
Development ProgrammeFindings and implications
for future research
- Sophie Beale
- Senior Consultant, York Health Economics
Consortium, University of York - sjb29_at_york.ac.uk
2Overview
- What is Telecare?
- Introduction to the Telecare Development
Programme (TDP) - Overview of the evaluation of the TDP
- Summary of the findings
- Implications for the evaluation of telecare
- Conclusions
3Telecare Development Programme
- Telecare Development Programme (TDP) launched in
August 2006. - To help more people in Scotland live at home
for longer, with safety and security, by
promoting the use of telecare in Scotland,
through the provision of a development fund and
associated support - Supported by c8M in funding
- Expected to provide a foundation for telecare to
become an integral part of community care across
Scotland.
4Telecare/Telehealth/Telemedicine
- There is considerable variation in the way in
which telecare, telehealth and telemedicine are
defined - All involve the delivery of services via
telecommunications and computerised systems,
usually to individuals in their own home
5Telecare/Telehealth/Telemedicine
Tele-intervention Examples
Telecare Telecare is concerned with monitoring over time to manage the risks (which may be the concern of health and/or social care) associated with independent living. Natural gas or bottled gas detector Wearable fall detector Passive Infra Red (PIR) movement detectors Nocturnal bed monitor Opening alerts for external doors.
Telehealth Remote exchange of physiological data between a patient at home and medical staff to assist in diagnosis and monitoring Blood pressure monitoring Blood glucose monitoring Cardiac arrhythmia monitoring and Medication reminder systems
Telemedicine Use of medical information exchanged from one site to another via electronic communications Remote consultations with specialists
6Telecare Development Programme Specific
objectives
- Reduce the number of avoidable emergency
admissions and readmissions to hospital - Increase the speed of discharge from hospital
once clinical need is met - Reduce the use of care homes
- Improve the quality of life of users
- Reduce the pressure on informal carers
- Extend the range of people assisted by telecare
- Achieve efficiencies (cash releasing/time
saving) - Support effective procurement.
Majority of these have important economic
implications for health and social care
7Evaluation Overview
- Joint Improvement Team commissioned an evaluation
of the TDP - Assessment of the implementation and uptake
- Assessment of performance against the agreed
objectives - Provide an evidence base on the costs and
benefits of the Programme - Develop metrics that can become embedded in
practice.
8Evaluation Methods
- Multiple methods explored at the outset of the
project - Emphasise that the TDP was an investment
programme, not a research exercise - Pragmatic approach
- Work with existing routine data where possible
- Minimise burden of data collection on
partnerships - Fit-for-purpose study, acknowledging the
limitations.
9Evaluation Methods
- Quarterly, self-reported monitoring from
partnerships on progress and outcomes against
pre-specified objectives - E.g. number of users, number of admissions
avoided, number of delayed discharges avoided - Surveys of users and carers to understand their
perceptions - Supplemented with case studies, including
in-depth interviews with stakeholders, users and
carers
10Findings Welcome financial headlines!
- Partnerships estimated to have saved over 11.5M
as a result of the TDP investment - Reduction in admissions to care homes, equating
to over 60,000 care home days c 31 of savings - Reduction in unplanned hospital admissions,
equating to over 13,000 bed days c 30 of
savings - Improved speed of discharge, equating to over
5,000 bed days 16 of savings - TDP produced savings against 6 efficiency targets
identified at the outset of the study
11Findings Impact on Users and Carers
- 60 of users reported improvements in quality of
life - Nearly 70 reported feeling more independent
- Over 90 reported feeling safer
- Less than 5 reported feeling lonelier as a
result of telecare - Nearly 75 of informal carers felt that telecare
had reduced pressure/stress - Facilitated greater independence for users
12Findings Organisational Policy impact
- Almost 8,000 new users of TDP funded telecare
during 2007/08 - Predominantly older people (85 aged gt65)
- New users with dementia, learning disabilities or
physical disabilities - Predominantly female (62)
- Approximately half of partnerships used the
National Framework Agreement to purchase telecare - Some partnerships found cheaper or better
solutions outside of the National Framework
Agreement
13Limitations
- Impact is self-reported
- Reliant on partnerships to consider the impact
relative to no TDP grant. - Not possible to include a control arm - difficult
to know the counter-factual - Not possible to attribute any incremental effect
to TDP - some of the programmes were already
underway - Incentives for partnerships to overstate the
benefits - In addition to this, caution should be taken in
interpreting the financial outcomes - Some of the efficiency gains are unlikely to be
cash-releasing
14Limitations
- Partnerships took time to implement telecare
solutions - Not all partnerships were able to complete all
quarterly returns over the course of the
evaluation - Time-lag due to time involved in the purchase and
implementation of telecare solutions - Additional time lag associated with the need for
cultural change in users and professionals
attitudes to care - E.g. referral triggers
15Limitations
- Impossible to quantify displacement or unintended
consequences of telecare - Are additional resources required in communities
as a result of less reliance on secondary
care/care homes?
16What can we interpret from the findings?
- Pragmatic study of telecare in use assessed
against a wide range of outcome measures - Beneficial impact on health and social care
efficiencies - More importantly, beneficial impact on user
outcomes, such as independent living and
perceptions of safety - The study does not provide definitive evidence of
the effectiveness of any specific intervention - Too many interventions included to assess
effectiveness in any meaningful population
17Implications for future research RCTs
- Randomised controlled trials are the most
appropriate means of assessing the effectiveness
of individual interventions - Allow for identification of an attributable
effect by controlling other influences - However, RCTs may be of limited value in
supporting the mainstreaming of telecare - Effectiveness is context specific
- Heterogeneity in users and service environments
- Heterogeneity in access to interventions
- Practical difficulties in recruitment and
randomisation - Hawthorne effect observed in telehealth
studies?
18Implications for future research
- A number of alternative approaches to evaluation
were explored - Before and after study
- Monitor users before and after the introduction
of telecare - Problematic in progressive conditions or where
telecare has been provided after a sudden change
in health state (e.g. stroke) - Use of predictive models
- Predicts expected hospitalisations based on an
individuals characteristics - Methods not fully mature problematic in
progressive illnesses and focussed primarily on
health care - Use of control groups
- Difficulties in selecting control group what
should you control for? - Is it ethical to refuse to provide telecare to a
member of the control group?
19Implications for telecare adoption
- Widespread adoption of promising technologies
remains limited due to the limited availability
of high-quality evidence - Studies tend to be commissioned by those with an
interest in promoting telecare - Small populations and lack of follow-up lead to
limited generalisability of findings - Therefore, adoption of promising technologies
remains limited
20A model for pragmatic, evidence-based uptake of
telecare?
- A risk-sharing model may support appropriate
adoption - Purchasers and suppliers define reasonable
objectives for telecare and evaluation criteria - Promising results support further adoption
- Failure to achieve objectives explored
- Rebates in place for sub-optimal performance?
21A model for pragmatic, evidence-based uptake of
telecare?
- Risk-sharing is suited to promising technologies
characterised by uncertainty over their
effectiveness - Addresses purchasers concerns about equivocal
evidence base by providing some form of
guarantee - Addresses manufacturers concerns about the need
to continuously invest in context specific trials - Generates further evidence on the barriers and
facilitators for implementation in practice
settings - Increasingly applied to pharmaceuticals
possibly more relevant to interventions such as
telecare - Hard endpoints available from routinely collected
data
22Summary Recommendations
- The Scottish TDP programme appears to have been a
cost effective use of resources - Acknowledge the limitations of the pragmatic
evaluation - Acknowledge that many of the benefits of the
programme have yet to materialise - The TDP evaluation provides further evidence to
support the mainstream use of telecare, although
the findings are equivocal - Are RCTs the most appropriate means of supporting
mainstreaming of telecare? - An evidence based approach to the staged
widespread adoption of promising telecare
technologies might address the concerns of all
stakeholders involved.
23Acknowledgements
- Diana SandersonAssociate Senior Consultant
- Jen KrugerProject Support Officer
- Paul Trueman
Director - Joint Improvement Team
- TDP partnerships