1 / 87

Wildlife Health Surveillance

- Evan Sergeant
- AusVet Animal Health Services

Welcome and Introductions

- welcome
- housekeeping
- Introductions
- Who are you, where from, organization
- Expectations from course what would you like to

learn - How you would expect to apply what you learn in

your current/future work - Specific topics of interest to you

Workshop outcomes

- At the end of the workshop, participants should

be able to - Understand and explain key epidemiological/surveil

lance terms - Describe the sampling methods and strategies used

for surveillance - Plan and implement a surveillance activity

Course Outline Day 1

- Introductions, Expectations and Outcomes, Course

overview - USB sticks epitools
- Introduction to Surveillance
- Investigating disease problems
- Patterns of disease
- Measuring and comparing disease frequency
- Exercise

Course Outline Day 2

- Screening and diagnosis
- Sampling populations
- Bias
- Surveillance for presence/absence
- Translocation and disease risk

Course Outline Day 3

- Risk-based surveillance
- Prevalence surveys
- Planning a surveillance activity
- Review
- Workshop evaluation Close

EpiTools and USB

- Web-based epidemiological calculator and

utilities - Available at http//epitools.ausvet.com.au
- Provided on USB stick
- Instructions in word document in root directory

of stick - USB also has Resources folder, containing

materials for case studies and activities during

workshop

Surveillance

- What is surveillance?
- Surveillance or monitoring?
- Why do we do surveillance (Reasons)?
- Types of surveillance (categorisation)?

Definition

- OIE
- the systematic ongoing collection, collation,

and analysis of information related to animal

health and the timely dissemination of

information to those who need to know so that

action can be taken

Why do surveillance

- Improved decision-making
- Diseases that are present
- Describe disease occurrence
- Assess progress
- Diseases that are absent
- Detect disease
- Demonstrate freedom

Types of surveillance

- Categorising surveillance
- Disease focus
- General / Targeted
- Information source
- Active / Passive
- Representativeness
- Representative / Risk-based / targeted
- Population coverage
- comprehensive / incomplete

- Examples
- Wildlife carers
- Dead bird/animal surveys
- Structured surveys
- Surveys of local wildlife officers

Outbreak Investigation

- Patterns of disease
- Measuring disease
- Investigating disease problems

An exampleCholera in London, 1854

- Map shows
- cholera cases in London in September 1854 (black

bars) and - water pumps in the same area
- Note the cause of cholera at that time was still

unknown - What two things are you going to do?
- Why?

Epidemiology in actionCholera in London, 1854

Before you start

- Objectives of your investigation
- Case definition,
- Unit of interest, Population at-risk
- Possible causes/differential diagnosis
- Data that might be available
- How are you going to analyse/present it?
- What are the sort of things this might tell us?

What might this tell us?

- Identify possible risk factors and potential

causes - Biological importance vs statistical significance

of possible risk factors - Possible interventions to stop outbreak or for

future cases - Directions for further investigation

Analysis and presentation of information?

- Measures of disease risk
- incidence/prevalence, attack rate, group specific

rates - Comparison of risk
- Relative risk, attributable risk and odds ratio
- Confidence intervals
- Spatial/temporal/animal patterns of disease
- Epidemic curves, maps, diagrams, etc
- Confounding

Case study

- FMD outbreak investigation
- Using the data and information provided
- Undertake the sort of analyses you discussed
- What does it tell you?
- What is your working hypothesis?
- What recommendations would you make for immediate

action and/or for further investigation? - Report back discuss

Example

- Salmonella in hihi

(No Transcript)

Course Outline Day 1

- Introductions, Expectations and Outcomes, Course

overview - USB sticks epitools
- Introduction to Surveillance
- Investigating disease problems
- Patterns of disease
- Measuring and comparing disease frequency
- Exercise

Course Outline Day 2

- Screening and diagnosis
- Sampling populations
- Bias
- Disease Freedom
- Translocation risk

Diagnosis and screening

- What is a test?
- Why do we use tests?
- What is the difference between using tests for

diagnosis and screening? - How do we measure test performance?
- What is a good test?

Test performance

- Accuracy vs precision
- Measures of accuracy
- Sensitivity
- Specificity
- Measures of precision
- Repeatability
- Reproduceability

Precision

- Repeatability
- The ability of a test to give consistent results

in repeated tests performed under conditions that

are as constant as possible, in the one

laboratory, by one operator using the same

equipment over a short period of time. - Reproducibility
- The ability of a test to give consistent results

in repeated tests under widely varying conditions

in different laboratories at different times by

different operators. .

Assessing test precision

- Robustness
- The robustness of an analytical method is a

measure of its capacity to remain unaffected by

small, but deliberate variations in method

performance parameters and provides an indication

of its reliability during normal usage - Codex Alimentarius Commission

Assessing test precision

- Coefficient of variation
- The ratio of the standard deviation of a sample

to its mean - Correlation coefficient
- Correlation of results of duplicate testing of

individual samples

Measures of accuracy

- Definitions
- Sensitivity
- Specificity

Sensitivity

- The proportion of animals with the disease of

interest who test positive. - the probability that a test will correctly

identify those animals that are infected (Pr

TD) - True Positive Rate 1 false negative rate

Specificity

- The proportion of animals without the disease of

interest that test negative. - the probability that a test will correctly

identify those animals that are not infected (Pr

T-D-). - True Negative Rate 1 false positive rate

Exercise

- What is the sensitivity of a test in following

situations? - 10 infected animals tested, 9 positive
- 100 infected animals tested, 90 positive
- 75 infected, 73 positive
- What is the specificity of a test in following

situations? - 100 uninfected animals tested, 99 negative
- 1000 uninfected animals tested, 990 negative
- 453 uninfected, 420 negative
- How confident are you in each case?

- Calculate confidence intervals using epitools
- Application of diagnostic tests gt Test evaluation

against gold standard - Disease status reference test

Small sample size (10 100) Point Estimate Lower 95 CL Upper 95 CL

Sensitivity 0.9 0.555 0.9975

Specificity 0.99 0.9455 0.9997

Large sample size (100 1000) Point Estimate Lower 95 CL Upper 95 CL

Sensitivity 0.9 0.8238 0.951

Specificity 0.99 0.9817 0.9952

Applying Individual tests

- How do we measure test performance?
- What do we want to know when we use a test?

Applying Individual tests

- Testing for ??
- Testing a single animal
- You know that on average 1 of animals in the

area are infected - Test Se 99, Sp 95
- 2 possible results test positive or test

negative - For each result, do you think the animal is

infected (or not)? Probability? Why (not)? - What difference would it make if the animal was

from a sub-population that you knew had a 20

infection rate

PPV/NPV

- Positive predictive value
- Negative predictive value
- Scenario tree to calculate

Results

PPV NPV

Prior probability 1 16.7 99.99

Prior probability 20 83.2 99.7

- Se 99
- Sp 95

D D- Total

T a (90) b (20) 110

T- c (10) d (980) 990

Total 100 1000 1100

- Se P(TD) a/(ac) 90
- Sp P(T-D-) d/(bd) 98
- PPV P(DT) a/(ab) 81.8
- NPV P(D-T-) d/(cd) 99
- PPV and NPV assume representative sample of

population

Combining tests

- What is testing in series and parallel?
- How are they interpreted?
- What effect does this have on Se and Sp overall?
- What are some examples of where we use series or

parallel testing? - Try and use unrelated tests

Series and parallel

- Calculate Se and Sp
- Test 1 Se 95, Sp 95
- Test 2 Se 60, Sp 99
- Calculate in Epitools
- application of diagnostic testsgtUsing tests in

series and parallel - Work out using scenario tree

- Series
- Se 57, Sp 1
- Parallel
- Se 98, Sp 94

Sampling populations

- Sample or census?
- Populations target/reference, study, population

at risk - Sampling units animals, ecological units, etc
- Case definition(s)
- Probability and non-probability sampling
- Sample size

Non-probability sampling

- Convenience,
- Haphazard,
- Purposive
- Risk-based

Probability sampling

- SRS,
- systematic,
- stratified,
- multi-stage cluster,
- Random Geographic Coordinates
- Probability proportional to size
- Transects

Other sampling issues

- Sampling with or without replacement
- Sampling in small populations

Sample size calculation

- Sample size to estimate prevalence
- What do you need to know?
- Sample size calculations to detect disease or

demonstrate freedom - What do you need to know?
- Relate sample estimates to population parameters

(inference) - Confidence intervals, statistical significance

Bias

- What is bias in epidemiological studies?
- Can we categorise types of errors according to

their source? - What about other (non-bias) sources of error?
- How important is bias?
- What can we do to deal with it?

Sources of error

- Bias
- Selection
- Misclassification/Measurement (differential vs

non-differential) - Recall
- diagnostic
- Confounding
- Random error

Confounding

- occurs when two risk factors are interrelated and

it is incorrectly concluded that one of the

factors is causally related to the disease in

question - The true risk factor is the confounder

Example

- ??

Identifying/Controlling bias

- Recognise that bias might be present
- Sample selection
- Randomisation,
- Matching,
- Exclusion/restriction
- Stratification
- Accurate measurement/classification
- calibration
- series/parallel testing
- operator training and standardisation
- During analysis (confounding)
- Stratification, Mantel-Haenszel Chi-squared
- multivariable techniques multiple regression

analyses

Freedom

- How do we define freedom?
- Demonstrating freedom vs detecting disease
- Can we prove freedom?
- How do we measure freedom?
- SSe, NPV

Population testing

- Testing in groups (cluster or population)
- Se 70, Sp 98, population size 1000
- Scenario 1 test 20 units
- Scenario 2 test 100 units

Your first task

- For the scenarios provided
- What is the probability we will detect infection

if it is present? - What if it is not infected? What is the

probability we will correctly identify it as

uninfected? - What do we call these?
- What else do we need to know for our

calculations? - How does this affect your conclusions?

Additional information

- Epitoolsgt1-stage freedom surveysgtpopulation

sensitivity with imperfect test (assuming large

population) - What is design Prevalence?
- Design prevalence 2
- Cut-point number of reactors 1
- SeH 1 (1 (SeP (1 Sp)(1 P)))n
- SpH Spn

Results

SeH SpH

n 20 49.5 66.8

n 100 96.7 13.3

- Dealing with poor herd specificity
- SpH decreases rapidly with increasing sample size

or decreasing Sp - What can we do to overcome this?
- Dealing with poor herd sensitivity
- How can we do to overcome this?

- Dealing with poor herd specificity
- Change test cut-off to improve Sp
- Test in series (follow-up testing)
- Increase cut-off number of reactors
- Dealing with poor herd sensitivity
- Change test cut-off to improve Se
- Test in parallel ( on both tests to be positive)
- Decrease cut-off number of reactors
- Increase sample size

Population testing

- Repeat assuming perfect specificity (after

follow-up) - Se 70, Sp 100, population size 1000,

design prevalence 2 - Scenario 1 test 20 units
- Scenario 2 test 100 units
- Epitoolsgt1-stage freedom surveysgtpopulation

sensitivity assuming perfect test - Repeat assuming unknown population size

Results

SeH (population 1000) SeH (unknown population) SpH

n 20 24.6 24.6 100

n 100 76.6 75.6 100

Confidence of freedom

- Equivalent to NPV at population level
- Probability (confidence) that the population is

free (at design prevalence) given the negative

surveillance data - Need to have estimates of the prior (before the

surveillance) probability that the population is

free and the population-level sensitivity

(SeH/SSe). - Can be updated over time as more surveillance

undertaken

Calculation

- Pfree Prior/(1 SSe x (1 Prior))
- What is the prior?
- How do we estimate the prior?
- 50 default value
- Subjective estimate from previous experience
- Quantitative estimate from previous analysis

Example

- What is the confidence of freedom for the

following example - SSe 0.8
- Prior 0.7 (moderately confident of freedom

before testing) - Use epitoolsgt1-stage surveysgtconfidence of

freedom for a single time period

Using historical data

- What is surveillance time period?
- How can we utilise value of historical data

what changes over time? - Calculations?

Calculations

- Prior 1 1 Pfree Pintro ((1 Pfree) x

Pintro) - What is probability of introduction?
- How do we estimate this?

Exercise

- Use epitoolsgt1-stage surveysgt confidence of

freedom for multiple time periods - Data in Excel
- Accumulating confidence over time.xls
- Calculate PFree for 10 time periods using

parameters provided - How long does it take to reach PFree 0.95?

1-stage freedom surveys

- Single population level
- Cluster eg eco-system, farm, etc OR
- Population-wide
- Sample size calculation
- Sample selection and data collection
- Calculate SeH/SSe and confidence of freedom

Case studies

Translocation risk

- What is the probability the consignment is

infected? - What are the options to reduce risk?
- How do we quantify this?
- See epitoolsgtapplication of diagnostic

testsgtprobability of infection in test negative

consignment - Translocation risk.xlsx

2-stage freedom surveys

- Multiple (2) population levels
- Cluster eg eco-system, farm, etc AND
- Population-wide
- Sample size calculation
- at both levels
- Sample selection and data collection
- Calculate SeH SSe and confidence of freedom

Case studies

Course Outline Day 2

- Screening and diagnosis
- Sampling populations
- Bias
- Disease Freedom
- Translocation risk

Course Outline Day 3

- Risk-based surveillance
- Prevalence surveys
- Planning a surveillance activity
- Review
- Workshop evaluation Close

Risk-based surveillance

- What is risk-based surveillance?
- Differential risk in population
- Targeting of high risk group(s)
- Targeting of sub-groups more likely to produce a

positive test result - Dealing with risk

Example

- Case studies

Disease prevalence

- Given specific results from 3 separate

populations - 3/20, 3/100, 20/100
- Se 70, Sp 95
- What is the apparent prevalence in each

population? - What is the true prevalence and confidence

limits? - What is the difference?

Results

- Estimated true prevalence (from epitools, with

Normal approximation CI) - 3/20 15.4 (-8.7 39.5)
- 3/100 lt0 (-8 2)
- 20/100 23 (11 35)

Issues

- True prevalence is different to apparent

prevalence (may be higher or lower) - CI for TP wider than for AP
- TP can be negative (Why?)
- Lower CL may be negative for normal approximation

Blakers method generally gives better

approximation - CI get narrower as sample size increases (and as

estimate approaches 0 or 1)

Prevalence case studies

- Sampling strategies
- Sample size calculation
- Sample selection and data collection
- Analysis true prevalence and CI
- Examples
- 1-stage
- 2-stage

Planning surveillance

- What are the key elements of a plan for planning

a surveillance program?

Planning surveillance

- For a specified policy/disease control decision
- What is the objective of doing surveillance?
- What information do I need?
- What are possible sources of information?
- existing (availability, cost, resources,

quality)? - options for collecting new data?
- quality standards how good does it have to be?
- what type of surveillance to collect it?

Representative or not, targeted or general,

active or passive? - What will you do?

Planning animal health surveillance

- Background
- Purpose/objectives SMART
- Stakeholders/responsible parties
- Nature of disease/condition of interest
- Case definition(s)
- Expected outcomes of surveillance
- Data sources

Planning animal health surveillance

- Reference/target population
- Population description
- Source/study population
- Sampling units
- Selection strategy sample size calculations
- Sampling methods, assumptions
- Design prevalence, confidence, precision, etc
- Tests, sensitivity and specificity, etc
- Dealing with positives
- Investigation and response procedures

- Data Collection
- Data management analysis
- Project management and organisation
- Whos involved and roles/responsibilities
- Logistics
- Timeline and milestones
- Performance measures
- Reporting and communication
- Budget and source of funding and supporting

agencies

Reporting animal health surveillance

- Executive summary
- Background/Introduction
- Objectives
- Methods
- Results Discussion
- graphs and tables of summary results
- Conclusions
- Recommendations
- References
- Appendices
- detailed results and extended summaries

Planning Exercise

- In groups decide on disease/condition of

interest - Select an appropriate surveillance strategy and

prepare a short presentation to describe your

system and explain why it is appropriate.

Workshop close

- Review
- Evaluation
- Certificates