Wildlife Health Surveillance

1
Wildlife Health Surveillance

• Evan Sergeant
• AusVet Animal Health Services

2
Welcome and Introductions

• welcome
• housekeeping
• Introductions
• Who are you, where from, organization
• Expectations from course what would you like to
  learn
• How you would expect to apply what you learn in
  your current/future work
• Specific topics of interest to you

3
Workshop outcomes

• At the end of the workshop, participants should
  be able to
• Understand and explain key epidemiological/surveil
  lance terms
• Describe the sampling methods and strategies used
  for surveillance
• Plan and implement a surveillance activity

4
Course Outline Day 1

• Introductions, Expectations and Outcomes, Course
  overview
• USB sticks epitools
• Introduction to Surveillance
• Investigating disease problems
• Patterns of disease
• Measuring and comparing disease frequency
• Exercise

5
Course Outline Day 2

• Screening and diagnosis
• Sampling populations
• Bias
• Surveillance for presence/absence
• Translocation and disease risk

6
Course Outline Day 3

• Risk-based surveillance
• Prevalence surveys
• Planning a surveillance activity
• Review
• Workshop evaluation Close

7
EpiTools and USB

• Web-based epidemiological calculator and
  utilities
• Available at http//epitools.ausvet.com.au
• Provided on USB stick
• Instructions in word document in root directory
  of stick
• USB also has Resources folder, containing
  materials for case studies and activities during
  workshop

8
Surveillance

• What is surveillance?
• Surveillance or monitoring?
• Why do we do surveillance (Reasons)?
• Types of surveillance (categorisation)?

9
Definition

• OIE
•   the systematic ongoing collection, collation,
  and analysis of information related to animal
  health and the timely dissemination of
  information to those who need to know so that
  action can be taken

10
Why do surveillance

• Improved decision-making
• Diseases that are present
• Describe disease occurrence
• Assess progress
• Diseases that are absent
• Detect disease
• Demonstrate freedom

11
Types of surveillance

• Categorising surveillance
• Disease focus
• General / Targeted
• Information source
• Active / Passive
• Representativeness
• Representative / Risk-based / targeted
• Population coverage
• comprehensive / incomplete

12

• Examples
• Wildlife carers
• Dead bird/animal surveys
• Structured surveys
• Surveys of local wildlife officers

13
Outbreak Investigation

• Patterns of disease
• Measuring disease
• Investigating disease problems

14
An exampleCholera in London, 1854

• Map shows
• cholera cases in London in September 1854 (black
  bars) and
• water pumps in the same area
• Note the cause of cholera at that time was still
  unknown
• What two things are you going to do?
• Why?

15
Epidemiology in actionCholera in London, 1854
16
Before you start

• Objectives of your investigation
• Case definition,
• Unit of interest, Population at-risk
• Possible causes/differential diagnosis
• Data that might be available
• How are you going to analyse/present it?
• What are the sort of things this might tell us?

17
What might this tell us?

• Identify possible risk factors and potential
  causes
• Biological importance vs statistical significance
  of possible risk factors
• Possible interventions to stop outbreak or for
  future cases
• Directions for further investigation

18
Analysis and presentation of information?

• Measures of disease risk
• incidence/prevalence, attack rate, group specific
  rates
• Comparison of risk
• Relative risk, attributable risk and odds ratio
• Confidence intervals
• Spatial/temporal/animal patterns of disease
• Epidemic curves, maps, diagrams, etc
• Confounding

19
Case study

• FMD outbreak investigation
• Using the data and information provided
• Undertake the sort of analyses you discussed
• What does it tell you?
• What is your working hypothesis?
• What recommendations would you make for immediate
  action and/or for further investigation?
• Report back discuss

20
Example

• Salmonella in hihi

21
(No Transcript)
22
Course Outline Day 1

• Introductions, Expectations and Outcomes, Course
  overview
• USB sticks epitools
• Introduction to Surveillance
• Investigating disease problems
• Patterns of disease
• Measuring and comparing disease frequency
• Exercise

23
Course Outline Day 2

• Screening and diagnosis
• Sampling populations
• Bias
• Disease Freedom
• Translocation risk

24
Diagnosis and screening

• What is a test?
• Why do we use tests?
• What is the difference between using tests for
  diagnosis and screening?
• How do we measure test performance?
• What is a good test?

25
Test performance

• Accuracy vs precision
• Measures of accuracy
• Sensitivity
• Specificity
• Measures of precision
• Repeatability
• Reproduceability

26
Precision

• Repeatability
• The ability of a test to give consistent results
  in repeated tests performed under conditions that
  are as constant as possible, in the one
  laboratory, by one operator using the same
  equipment over a short period of time.
• Reproducibility
• The ability of a test to give consistent results
  in repeated tests under widely varying conditions
  in different laboratories at different times by
  different operators. .

27
Assessing test precision

• Robustness
• The robustness of an analytical method is a
  measure of its capacity to remain unaffected by
  small, but deliberate variations in method
  performance parameters and provides an indication
  of its reliability during normal usage
• Codex Alimentarius Commission

28
Assessing test precision

• Coefficient of variation
• The ratio of the standard deviation of a sample
  to its mean
• Correlation coefficient
• Correlation of results of duplicate testing of
  individual samples

29
Measures of accuracy

• Definitions
• Sensitivity
• Specificity

30
Sensitivity

• The proportion of animals with the disease of
  interest who test positive.
• the probability that a test will correctly
  identify those animals that are infected (Pr
  TD)
• True Positive Rate 1 false negative rate

31
Specificity

• The proportion of animals without the disease of
  interest that test negative.
• the probability that a test will correctly
  identify those animals that are not infected (Pr
  T-D-).
• True Negative Rate 1 false positive rate

32
Exercise

• What is the sensitivity of a test in following
  situations?
• 10 infected animals tested, 9 positive
• 100 infected animals tested, 90 positive
• 75 infected, 73 positive
• What is the specificity of a test in following
  situations?
• 100 uninfected animals tested, 99 negative
• 1000 uninfected animals tested, 990 negative
• 453 uninfected, 420 negative
• How confident are you in each case?

33

• Calculate confidence intervals using epitools
• Application of diagnostic tests gt Test evaluation
  against gold standard
• Disease status reference test

34
Small sample size (10 100)  Point Estimate   Lower 95 CL  Upper 95 CL 
Sensitivity 0.9 0.555 0.9975
Specificity 0.99 0.9455 0.9997
Large sample size (100 1000)  Point Estimate   Lower 95 CL  Upper 95 CL 
Sensitivity 0.9 0.8238 0.951
Specificity 0.99 0.9817 0.9952
35
Applying Individual tests

• How do we measure test performance?
• What do we want to know when we use a test?

36
Applying Individual tests

• Testing for ??
• Testing a single animal
• You know that on average 1 of animals in the
  area are infected
• Test Se 99, Sp 95
• 2 possible results test positive or test
  negative
• For each result, do you think the animal is
  infected (or not)? Probability? Why (not)?
• What difference would it make if the animal was
  from a sub-population that you knew had a 20
  infection rate

37
PPV/NPV

• Positive predictive value
• Negative predictive value
• Scenario tree to calculate

38
Results
PPV NPV
Prior probability 1 16.7 99.99
Prior probability 20 83.2 99.7

• Se 99
• Sp 95

39
D D- Total
T a (90) b (20) 110
T- c (10) d (980) 990
Total 100 1000 1100

• Se P(TD) a/(ac) 90
• Sp P(T-D-) d/(bd) 98
• PPV P(DT) a/(ab) 81.8
• NPV P(D-T-) d/(cd) 99
• PPV and NPV assume representative sample of
  population

40
Combining tests

• What is testing in series and parallel?
• How are they interpreted?
• What effect does this have on Se and Sp overall?
• What are some examples of where we use series or
  parallel testing?
• Try and use unrelated tests

41
Series and parallel

• Calculate Se and Sp
• Test 1 Se 95, Sp 95
• Test 2 Se 60, Sp 99
• Calculate in Epitools
• application of diagnostic testsgtUsing tests in
  series and parallel
• Work out using scenario tree

42

• Series
• Se 57, Sp 1
• Parallel
• Se 98, Sp 94

43
Sampling populations

• Sample or census?
• Populations target/reference, study, population
  at risk
• Sampling units animals, ecological units, etc
• Case definition(s)
• Probability and non-probability sampling
• Sample size

44
Non-probability sampling

• Convenience,
• Haphazard,
• Purposive
• Risk-based

45
Probability sampling

• SRS,
• systematic,
• stratified,
• multi-stage cluster,
• Random Geographic Coordinates
• Probability proportional to size
• Transects

46
Other sampling issues

• Sampling with or without replacement
• Sampling in small populations

47
Sample size calculation

• Sample size to estimate prevalence
• What do you need to know?
• Sample size calculations to detect disease or
  demonstrate freedom
• What do you need to know?
• Relate sample estimates to population parameters
  (inference)
• Confidence intervals, statistical significance

48
Bias

• What is bias in epidemiological studies?
• Can we categorise types of errors according to
  their source?
• What about other (non-bias) sources of error?
• How important is bias?
• What can we do to deal with it?

49
Sources of error

• Bias
• Selection
• Misclassification/Measurement (differential vs
  non-differential)
• Recall
• diagnostic
• Confounding
• Random error

50
Confounding

• occurs when two risk factors are interrelated and
  it is incorrectly concluded that one of the
  factors is causally related to the disease in
  question
• The true risk factor is the confounder

51
Example

• ??

52
Identifying/Controlling bias

• Recognise that bias might be present
• Sample selection
• Randomisation,
• Matching,
• Exclusion/restriction
• Stratification
• Accurate measurement/classification
• calibration
• series/parallel testing
• operator training and standardisation
• During analysis (confounding)
• Stratification, Mantel-Haenszel Chi-squared
• multivariable techniques multiple regression
  analyses

53
Freedom

• How do we define freedom?
• Demonstrating freedom vs detecting disease
• Can we prove freedom?
• How do we measure freedom?
• SSe, NPV

54
Population testing

• Testing in groups (cluster or population)
• Se 70, Sp 98, population size 1000
• Scenario 1 test 20 units
• Scenario 2 test 100 units

55
Your first task

• For the scenarios provided
• What is the probability we will detect infection
  if it is present?
• What if it is not infected? What is the
  probability we will correctly identify it as
  uninfected?
• What do we call these?
• What else do we need to know for our
  calculations?
• How does this affect your conclusions?

56
Additional information

• Epitoolsgt1-stage freedom surveysgtpopulation
  sensitivity with imperfect test (assuming large
  population)
• What is design Prevalence?
• Design prevalence 2
• Cut-point number of reactors 1
• SeH 1 (1 (SeP (1 Sp)(1 P)))n
• SpH Spn

57
Results
SeH SpH
n 20 49.5 66.8
n 100 96.7 13.3
58

• Dealing with poor herd specificity
• SpH decreases rapidly with increasing sample size
  or decreasing Sp
• What can we do to overcome this?
• Dealing with poor herd sensitivity
• How can we do to overcome this?

59

• Dealing with poor herd specificity
• Change test cut-off to improve Sp
• Test in series (follow-up testing)
• Increase cut-off number of reactors
• Dealing with poor herd sensitivity
• Change test cut-off to improve Se
• Test in parallel ( on both tests to be positive)
• Decrease cut-off number of reactors
• Increase sample size

60
Population testing

• Repeat assuming perfect specificity (after
  follow-up)
• Se 70, Sp 100, population size 1000,
  design prevalence 2
• Scenario 1 test 20 units
• Scenario 2 test 100 units
• Epitoolsgt1-stage freedom surveysgtpopulation
  sensitivity assuming perfect test
• Repeat assuming unknown population size

61
Results
SeH (population 1000) SeH (unknown population) SpH
n 20 24.6 24.6 100
n 100 76.6 75.6 100
62
Confidence of freedom

• Equivalent to NPV at population level
• Probability (confidence) that the population is
  free (at design prevalence) given the negative
  surveillance data
• Need to have estimates of the prior (before the
  surveillance) probability that the population is
  free and the population-level sensitivity
  (SeH/SSe).
• Can be updated over time as more surveillance
  undertaken

63
Calculation

• Pfree Prior/(1 SSe x (1 Prior))
• What is the prior?
• How do we estimate the prior?
• 50 default value
• Subjective estimate from previous experience
• Quantitative estimate from previous analysis

64
Example

• What is the confidence of freedom for the
  following example
• SSe 0.8
• Prior 0.7 (moderately confident of freedom
  before testing)
• Use epitoolsgt1-stage surveysgtconfidence of
  freedom for a single time period

65
Using historical data

• What is surveillance time period?
• How can we utilise value of historical data
  what changes over time?
• Calculations?

66
Calculations

• Prior 1 1 Pfree Pintro ((1 Pfree) x
  Pintro)
• What is probability of introduction?
• How do we estimate this?

67
Exercise

• Use epitoolsgt1-stage surveysgt confidence of
  freedom for multiple time periods
• Data in Excel
• Accumulating confidence over time.xls
• Calculate PFree for 10 time periods using
  parameters provided
• How long does it take to reach PFree 0.95?

68
1-stage freedom surveys

• Single population level
• Cluster eg eco-system, farm, etc OR
• Population-wide
• Sample size calculation
• Sample selection and data collection
• Calculate SeH/SSe and confidence of freedom

69
Case studies
70
Translocation risk

• What is the probability the consignment is
  infected?
• What are the options to reduce risk?
• How do we quantify this?
• See epitoolsgtapplication of diagnostic
  testsgtprobability of infection in test negative
  consignment
• Translocation risk.xlsx

71
2-stage freedom surveys

• Multiple (2) population levels
• Cluster eg eco-system, farm, etc AND
• Population-wide
• Sample size calculation
• at both levels
• Sample selection and data collection
• Calculate SeH SSe and confidence of freedom

72
Case studies
73
Course Outline Day 2

• Screening and diagnosis
• Sampling populations
• Bias
• Disease Freedom
• Translocation risk

74
Course Outline Day 3

• Risk-based surveillance
• Prevalence surveys
• Planning a surveillance activity
• Review
• Workshop evaluation Close

75
Risk-based surveillance

• What is risk-based surveillance?
• Differential risk in population
• Targeting of high risk group(s)
• Targeting of sub-groups more likely to produce a
  positive test result
• Dealing with risk

76
Example

• Case studies

77
Disease prevalence

• Given specific results from 3 separate
  populations
• 3/20, 3/100, 20/100
• Se 70, Sp 95
• What is the apparent prevalence in each
  population?
• What is the true prevalence and confidence
  limits?
• What is the difference?

78
Results

• Estimated true prevalence (from epitools, with
  Normal approximation CI)
• 3/20 15.4 (-8.7 39.5)
• 3/100 lt0 (-8 2)
• 20/100 23 (11 35)

79
Issues

• True prevalence is different to apparent
  prevalence (may be higher or lower)
• CI for TP wider than for AP
• TP can be negative (Why?)
• Lower CL may be negative for normal approximation
  Blakers method generally gives better
  approximation
• CI get narrower as sample size increases (and as
  estimate approaches 0 or 1)

80
Prevalence case studies

• Sampling strategies
• Sample size calculation
• Sample selection and data collection
• Analysis true prevalence and CI
• Examples
• 1-stage
• 2-stage

81
Planning surveillance

• What are the key elements of a plan for planning
  a surveillance program?

82
Planning surveillance

• For a specified policy/disease control decision
• What is the objective of doing surveillance?
• What information do I need?
• What are possible sources of information?
• existing (availability, cost, resources,
  quality)?
• options for collecting new data?
• quality standards how good does it have to be?
• what type of surveillance to collect it?
  Representative or not, targeted or general,
  active or passive?
• What will you do?

83
Planning animal health surveillance

• Background
• Purpose/objectives SMART
• Stakeholders/responsible parties
• Nature of disease/condition of interest
• Case definition(s)
• Expected outcomes of surveillance
• Data sources

84
Planning animal health surveillance

• Reference/target population
• Population description
• Source/study population
• Sampling units
• Selection strategy sample size calculations
• Sampling methods, assumptions
• Design prevalence, confidence, precision, etc
• Tests, sensitivity and specificity, etc
• Dealing with positives
• Investigation and response procedures

85

• Data Collection
• Data management analysis
• Project management and organisation
• Whos involved and roles/responsibilities
• Logistics
• Timeline and milestones
• Performance measures
• Reporting and communication
• Budget and source of funding and supporting
  agencies

86
Reporting animal health surveillance

• Executive summary
• Background/Introduction
• Objectives
• Methods
• Results Discussion
• graphs and tables of summary results
• Conclusions
• Recommendations
• References
• Appendices
• detailed results and extended summaries

87
Planning Exercise

• In groups decide on disease/condition of
  interest
• Select an appropriate surveillance strategy and
  prepare a short presentation to describe your
  system and explain why it is appropriate.

88
Workshop close

• Review
• Evaluation
• Certificates