WE PROBABLY COULD HAVE MORE FUN TALKING ABOUT THESE TRAFFIC STOPPERS

1
WE PROBABLY COULD HAVE MORE FUN TALKING ABOUT
THESE TRAFFIC STOPPERS
2
WHO CLEARLY HAVE THE RIGHT OF WAY!
BUT
3
DESIGNING SURVEYS OVER TIME (PANEL
SURVEYS)VARIANCE, POWER and RELATED TOPICS

• N. Scott Urquhart
• Senior Research Scientist
• Department of Statistics
• Colorado State University
• Fort Collins, CO 80527-1877

4
BRIEF COMMENTS ON MONITORING

• Monitoring is a long-term endeavor for most
  (living) natural resources
• Think of it as a legacy for your grandchildren
• Many of you have tried to use solid data
  someone else gathered, but documented poorly.
• Much good data dies when the person who
  gathered it dies or retires!
• Monitoring data requires three things to retain
  its value
• Metadata
• Storage in a retrievable, maintained, data
  system
• Backed up safe from fires, floods, or
  earthquakes

5
SHORT OUTLINE OF THE REST OF THIS TALK

• Inference Perspectives monitoring
• Trend, Variance Structures, and Power to Detect
  Trend
• Panels and Panel Structures
• Putting it all together Power Curves and
  standard errors of estimated current status
• Status Estimates from Data Over Years

6
INFERENCE PERSPECTIVES

• Design Based
• Inferences rest on the probability
  structure incorporated in the sampling plan
• Completely defensible very minimal assumptions
• Limiting relative to using auxiliary information
• Model Assisted
• Uses models to complement underlying sampling
  structure
• Has opportunities for use of auxiliary
  information
• Model Based (eg spatial statistics)
• Ignores sampling plan
• Defensibility lies in defense of model

7
APPROACH OF THIS PRESENTATION

• Use tools from the arena of
• Model-assisted and
• Model-based analyses
• To study the performance of
• Design based
• Model-assisted analyses
• WHY?
• Without models,
• performance evaluations need simulation
  (Steve Garmans topic)
• Before substantial data have been gathered
• Minimal basis for values to enter into simulation
  studies

8
STATUS TRENDS OVER TIME IN ECOLOGICAL
RESOURCES OF A REGIONMAJOR POINTS

• Regional trend ¹ site trend
• Detection of trend requires substantial elapsed
  time
• Regional OR intensive site
• Almost all indicators have substantial patterns
  in their variability
• Design to capitalize on this dont fight it.
• Minimize effect of site variability with
  planned revisits specific plans will be
  illustrated
• Design tradeoffs TREND vs STATUS

9
REGIONAL TREND ¹ SITE TREND

• The predominant theme of ecology
• Ecological processes
• How does a specific kind of ecosystem function
• Energy flows
• Food webs
• Nutrient cycling
• Most studies of such functions must be
• Temporally intensive
• What material goes from where to where?
• Consequently spatially restrictive
• In this situation Temporal trend site trend

10
REGIONAL TREND ¹ SITE TREND( - CONTINUED)

• The predominant theme of ecology versus
• A Substantial (any) Agency Focus
• All of an ecological resource
• In an area or region
• Across all of the variability present there
• For Example, National Park Service
• All riparian areas in Olympic National Park
• All riparian areas in National Parks in the
  coastal Northwest

11
TREND ACROSS TIME - What is it?

• Any response which changes across time in a
  generally
• Increasing or
• Decreasing
• Manner shows trend
• Monotonic change is not essential.
• If trend of this sort is present, it WILL BE
  detectable as linear trend.
• This does NOT mean trend must be linear
  (examples follow)
• Any specified form is detectable
• Time years, here

12
TREND ACROSS TIME - What is it?(continued)
13
(No Transcript)
14
TREND DETECTION REQUIRES SUBSTANTIAL ELAPSED TIME

• IT IS NEARLY IMPOSSIBLE TO DETECT TREND IN LESS
  THAN FIVE YEARS. WHY?

15
VARIANCE HAS A LOT OF STUCTURE IMPORTANT
COMPONENTS OF VARIANCE

• POPULATION VARIANCE
• YEAR VARIANCE
• RESIDUAL VARIANCE

16
HOW SHOULD YOU ESTIMATE VARIANCE?

• Alternatives
• Designed-based
• Horwitz-Thompson extremely variable dont
  use
• Local Neighborhood Variance Estimator (NBH)
  Stevens
• Gives estimate of variance something like
  residual component of variance, only
• Power nearly impossible to evaluate in this
  context
• Model-assisted
• Linear models Urquhart Courbois ( Williams,
  now)
• Gives estimates of site, year and residual
  variances
• Which should you use?

17
HOW SHOULD YOU ESTIMATE VARIANCE?(Continued)

• We really arent sure, but this topic is
  under active investigation
• Don Stevens and I are good friends, so this
  topic isnt a professional conflict.
• We both want to know the answer!
• Results by Courbois (JABES, 2004) suggest this
• Unless response values and inclusion
  probabilities (p) are highly correlated, they
  (p) can be ignored.
• If this stands up, as I expect it to,
  practically, linear model estimates of components
  of variance will be fine.
• Answer expected by summer.

18
IMPORTANT COMPONENTS OF VARIANCE ( - CONTINUED)

• POPULATION VARIANCE
• Variation among values of an indicator (response)
  across all sites in a park or group of related
  parks, that is, across a population or
  subpopulation of sites

19
IMPORTANT COMPONENTS OF VARIANCE ( - CONTINUED II)

• YEAR VARIANCE
• Concordant variation among values of an indicator
  (response) across years for ALL sites in a
  regional population or subpopulation
• NOT variation in an indicator across years at a
  single site
• Detrended remainder, if trend is present
• Effectively the deviation away from the trend
  line (or other curve)

20
IMPORTANT COMPONENTS OF VARIANCE ( - CONTINUED -
III)

• Residual component of variance
• Has several contributors
• YearSite interaction
• This contains most of what ecologists would call
  year to year variation, i.e. the site specific
  part
• Index variation
• Measurement error
• Crew-to-crew variation (minimize with well
  documented protocols and training)
• Local spatial protocol variation
• Short term temporal variation

21
SOURCE OF DATA FOR ESTIMATES OF COMPONENTS OF
VARIANCE

• EMAP Surface Waters Northeast Lakes Pilot
  1991 - 1994
• About 450 observations
• Over four years
• Including about 350 distinct lakes
• Design allowed estimation of several residual
  components
• Lakes illustrate what is generally referred to
  in this presentation as sites.
• Similar patterns appear in other data sets
• Both aquatic and terrestrial

22
(No Transcript)
23
ALL VARIABILITY IS OF INTEREST

• The site component of variance is one of the
  major descriptors of the regional population
• The year component of variance often is small,
  too small to estimate. It is a major enemy
  for detecting trend over time.
• If it has even a moderate size, sample size
  reverts to the number of years.
• In this case, the number of visits and/or number
  of sites has no practical effect.

24
ALL VARIABILITY IS OF INTEREST( - CONTINUED)

• Residual variance characterizes the
  inherent variation in the response or
  indicator.
• But some of its subcomponents may contain
  useful management information
• CREW EFFECTS gt training
• VISIT EFFECTS gt need to reexamine definition
  of index (time) window or evaluation protocol
• MEASUREMENT ERROR gt work on laboratory/measurem
  ent problems

25
DESIGN TRADE-OFFS TREND vs STATUS

• How do we detect trend in spite of all of
  this variation?
• Recall two old statistical friends.
• Variance of a mean, and
• Blocking

26
DESIGN TRADE-OFFS TREND vs STATUS( - CONTINUED)

• VARIANCE OF A MEAN
• Where m members of the associated population
  have been randomly selected and their response
  values averaged.
• Here the mean is a regional average slope, so
  "s2" refers to the variance of an estimated
  slope ---

27
DESIGN TRADE-OFFS TREND vs STATUS( - CONTINUED
- II)

• Consequently
• Becomes
• Note that the regional averaging of slopes has
  the same effect as continuing to monitor at one
  site for a much longer time period.

28
DESIGN TRADE-OFFS TREND vs STATUS( - CONTINUED
- III)

• Now, s2, in total, frequently is large.
• If we take one regional sample of sites at one
  time, and another at a subsequent time, the
  site component of variance is included in s2.
• Enter the concept of blocking, familiar
  from experimental design.
• Regard a site like a block
• Periodically revisit a site
• The site component of variance vanishes from
  the variance of a slope.

29
PANEL DESIGNS

• Question What kind of temporal design
  should you use for National Parks?
• A Panel is a Set of Sites which have the same
  Revisit Schedule
• Each panel ordinarily should have as good a
  spatial coverage as possible (GRTS)
• You have many usable and defensible
  temporal designs
• Choose one which fits your needs and resources
• Evaluation tools are available, and demonstrated
  here

30
A SINGLE PANEL
NUM OBS YEAR YEAR YEAR YEAR YEAR YEAR YEAR YEAR YEAR YEAR YEAR YEAR YEAR YEAR YEAR YEAR YEAR YEAR YEAR YEAR YEAR
NUM OBS 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21
11 X X X X X X X X X X X X X X X X X X X X X

• Conventional for trend detection
• Not very good for status
• (Call this design 1)

31
AN AUGMENTED PANEL PLAN (Design 2)
NUM OBS YEARS YEARS YEARS YEARS YEARS YEARS YEARS YEARS YEARS YEARS YEARS YEARS YEARS YEARS YEARS YEARS YEARS YEARS YEARS YEARS
NUM OBS 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19
5 X X X X X X X X X X X X X X X X X X X

• Add sites to the above annual revisit plane, as

32
AN AUGMENTED PANEL PLAN (Design 2)
NUM OBS YEARS YEARS YEARS YEARS YEARS YEARS YEARS YEARS YEARS YEARS YEARS YEARS YEARS YEARS YEARS YEARS YEARS YEARS YEARS YEARS
NUM OBS 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19
5 X X X X X X X X X X X X X X X X X X X
6 X
6 X
6 X
6 X
6 X
6 X
6 X
6 X
6 X
6 X
6 X
6 X
6 X
6 X
6 X
6 X

33
A POSSIBLE NPS PANEL PLAN (Design 3)
NUM OBS YEARS YEARS YEARS YEARS YEARS YEARS YEARS YEARS YEARS YEARS YEARS YEARS YEARS YEARS YEARS YEARS YEARS YEARS YEARS YEARS
NUM OBS 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19
5 X X
5 X
5 X
5 X
34
A POSSIBLE NPS PANEL PLAN (Design 3)
NUM OBS YEARS YEARS YEARS YEARS YEARS YEARS YEARS YEARS YEARS YEARS YEARS YEARS YEARS YEARS YEARS YEARS YEARS YEARS YEARS YEARS
NUM OBS 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19
5 X X
5 X
5 X
5 X
4 X X X X X
1 X X X X X
1 X X X X X
1 X X X X
35
A POSSIBLE NPS PANEL PLAN (Design 3)
NUM OBS YEARS YEARS YEARS YEARS YEARS YEARS YEARS YEARS YEARS YEARS YEARS YEARS YEARS YEARS YEARS YEARS YEARS YEARS YEARS YEARS
NUM OBS 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19
5 X X
5 X
5 X
5 X
4 X X X X X
1 X X X X X
1 X X X X X
1 X X X X
2 X X X X
1 X X X X
1 X X X X
1 X X
2 X X
1 X X
1 X X
1 X X
2 X X

36
WHY LOOK AT POWER?

• Power provides a tool for comparing various
  designs
• Looking at it does not imply that we have to
  conduct tests of hypotheses
• The computations displayed here use
• A components of variance statistical model
• To evaluate the underlying variances of estimated
  slopes
• And the temporal design being considered
• These computations are much more complex,
  and suitable to your problems, than any of the
  web-available tools
• Use of those tools here would commit sins of
  pseudoreplication!

37
POWER FOR DESIGNS 1 3, 2A WITH PARAMETER
CONDITIONS BILLY GAVElarge is good
38
POWER FOR DESIGNS 1 3, 2A WITH PARAMETER
CONDITIONS BILLY GAVElarge is good
39
POWER FOR DESIGNS 1 3, 2A WITH PARAMETER
CONDITIONS BILLY GAVElarge is good
40
POWER FOR DESIGNS 1 3, 2A WITH PARAMETER
CONDITIONS BILLY GAVElarge is good
41
STANDARD ERROR OF STATUS FOR DESIGNS 1 3,
2AWITH PARAMETER CONDITIONS BILLY GAVEsmall is
good
42
STANDARD ERROR OF STATUS FOR DESIGNS 1 3,
2AWITH PARAMETER CONDITIONS BILLY GAVEsmall is
good
43
STANDARD ERROR OF STATUS FOR DESIGNS 1 3,
2AWITH PARAMETER CONDITIONS BILLY GAVEsmall is
good
44
STANDARD ERROR OF STATUS FOR DESIGNS 1 3,
2AWITH PARAMETER CONDITIONS BILLY GAVEsmall is
good
45
DESIGN 3 A PROPOSED NPS TEMPORAL DESIGN??

• Defensible design, given restrictions of
• Field resources sites visited per year
• Need to split field resources across two type of
  aquatic systems
• Streams
• Wetlands

46
STATUS ESTIMATES FROM DATA OVER YEARS

• Idea
• Estimate year effects, and
• Adjust all sites to the latest year
• Idea very similar to adjusted treatment means
  in the analysis of covariance
• For example, if a line approximates the trend,
• The array of Sitei (yearnow
  yearinitial)slope
• Describes current status, accounting for
  documented trend
• This approach extends to models other than
  lines.
• Linkage of site revisits (connectedness) is
  necessary to allow estimation of all differences
  in year effects

47
FUNDING ACKNOWLEDGEMENT
The work reported here today was developed under
the STAR Research Assistance Agreement CR-829095
awarded by the U.S. Environmental Protection
Agency (EPA) to Colorado State University. This
presentation has not been formally reviewed by
EPA.  The views expressed here are solely those
of presenter and STARMAP, the Program he
represents. EPA does not endorse any products or
commercial services mentioned in this
presentation.
48
RELATED INFORMATION ON THE WEB

• Web-available information from a graduate
  course in environmental Sampling (ST571) I
  taught at OSU
• http//oregonstate.edu/instruct/st571/urquhart/ind
  ex.html
• Environmental Sampling
• Anatomy
• Variable Probability Sampling
• Cost Effective Resource Allocation
• Sampling Macroinvertebrates
• Maps Grids
• Spatial Sampling
• Support Regions
• Statistical Power - Concepts
• Power to Detect Trend in Ecological Resources
• Representative Sampling
• Statistical Aspects of Taxonomic Richness
• Sample Size to Estimate Taxonomic Richness
• Evaluating a Protocol for "Measuring" Physical
  Habitat
• Also see http//www.stat.colostate.edu/starmap/
  
• Also see NPS IM site, from Port Angeles
  meeting, 2003

49
VISUALIZING LINES AND YEAR EFFECTS

• N. Scott Urquhart
• STARMAP
• Colorado State University
• Fort Collins, Co 80523-1877

50
WHAT WE SEE NO LINES NO DECOMPOSITION
51
A LINE
52
A LINE WITH YEAR EFFECTS
53
A LINE WITH YEAR EFFECTS ANNUAL RESIDUALS
54
A LINE WITH ONLY ANNUAL RESIDUALS SHOWN
55
A LINE WITH ANNUAL RESIDUALS END MARKERS
56
A LINE WITH YEAR EFFECTS, JIGGERED RESIDUALS
END MARKERS
57
A REALITY

• With a single site
• The year effect
• And residual effect
• Can not be separated
• But with several sites
• These effects can be separated
• Following figures show the patterns

58
TWO LINES
59
TWO LINES WITH YEAR EFFECTS
60
TWO LINES WITH YEAR EFFECTS JIGGERED RESIDUALS
61
TWO LINES WITH YEAR EFFECTS JIGGERED
RESIDUALS ONLY SOME YEARS
62
WHAT WE SEE NO LINES NO DECOMPOSITION