Angi Hotz, M'S' and Howard C' Bailey, Ph'D', R'P'Bio'

1
EVALUATION OF TEST ACCEPTABILITY CRITERIA FOR THE
GIANT KELP TOXICITY TEST
Angi Hotz, M.S. and Howard C. Bailey, Ph.D.,
R.P.Bio. Nautilus Environmental, San Diego, CA
Abstract
Methods
Results
Results
Results
Results
Each laboratory was assigned a number (1-6) and
all results were analyzed and are presented
anonymously. For each laboratory, the
percentages of test failures due to each of the
acceptability criteria were calculated, and then
summed for a total within-laboratory failure
rate. The failure rates from the individual
laboratories were then averaged to calculate an
overall failure rate for the giant kelp test. In
order to evaluate the robustness of the
acceptability criteria, data from each laboratory
for each endpoint were compared using the
Kruskal-Wallis test with Bonferronis multiple
comparison post-test. Robustness was determined
according to the following rules
The giant kelp toxicity test was originally
developed by the Marine Bioassay Project (MBP) in
1990, and adopted by USEPA in 1995 as part of a
suite of chronic toxicity tests using west coast
species. Acceptability criteria for this test
were largely based on data collected during the
original protocol development by MBP, and have
not been evaluated since. This study evaluated
reference toxicant test data from five
laboratories in Southern California and one in
Northern California in order to determine whether
the criteria are sufficiently robust to be
applied across a range of testing laboratories.
Comparisons and conclusions are presented.
NOEC NOEC data for the growth endpoint were
evaluated. The among-laboratory mean was 27 µg/L
copper with a standard deviation of 24 µg/L
copper (Figure 4). Only one laboratory (Lab 5)
exhibited a mean NOEC value above the criterion
of lt 35 µg/L copper. However, an additional
three laboratories (Labs 1, 2 and 3) exhibited
means just below the criterion (Figure 3).
Results from this analysis indicate that the NOEC
criterion is not robust. MSD Evaluation of MSD
data among the six laboratories showed that there
were no significant differences among labs for
the germination endpoint (Figure 4a). However,
Lab 3 exhibited significantly higher growth data
MSDs than all other laboratories (Figure 4b).
This result was expected, since this laboratory
exhibited the largest percentage of failures due
to this criterion among all of the laboratories
evaluated (Figure 1). Generally, laboratories
were able to meet the MSD criteria, and variation
in MSD values among four of the six labs was
relatively low. However, the overall means and
standard deviations for both endpoints were below
the criteria, indicating that the MSD criteria
are robust.
The second variable of interest was kelp
sporophyll desiccation period. The test protocol
specifies only that the kelp should be used for
testing within 24 hours of collection (USEPA
1995). Therefore, depending on both laboratory
location and staff judgment, desiccation period
varied among laboratories. Since kelp collection
location had an effect on germination EC50s, data
were separated by collection location and
desiccation period, and then compared using
Mann-Whitney U tests. For the germination EC50
endpoint, significant differences were detected
between Southern California fresh and overnight
kelp, and also between Northern and Southern
California overnight kelp (Figure 6a). No
significant differences in growth EC25s were
detected among all groups tested (Figure 6b).
Data ranges for fresh kelp were wider than those
for overnight kelp for both collection locations
and endpoints, suggesting that overnight
desiccation reduces variability of test results.
The final variable examined for this study was
season. This variable was also evaluated due to
the regional differences in test sensitivity
observed. Data were grouped according to kelp
collection location and season and analyzed using
Mann-Whitney U tests. Seasons were determined by
examination of sea surface temperature data
(National Oceanic and Atmospheric Administration
(NOAA) 2002). Two distinct temperature modes were
observed one from December to May, and one from
June to November. A seasonal effect was
observed for kelp collected in San Diego for both
germination and growth endpoints, where the test
was less sensitive to copper during the second
half of the year (Figure 7). Interestingly,
although no statistical seasonal differences were
observed for Monterey kelp, the trend was the
opposite of that observed for San Diego kelp
(Figure 7).

Figure 3. Mean growth NOEC data across
laboratories. Data are presented 1 SD (n20
per laboratory). Dashed lines indicate overall
means and the acceptability criterion. Solid
lines indicate 1 SD for the overall mean.
Introduction
1) For the control germination and germ-tube
length acceptability criteria, if the overall
(among-laboratory) mean was greater than the
applicable criterion, and the mean minus one
standard deviation did not overlap the criterion,
the criterion was determined to be robust. 2)
For the NOEC and MSD criteria, if the
among-laboratory mean was less than the
applicable criterion, and the mean plus one
standard deviation did not overlap the criterion,
the criterion was determined to be robust.
Figure 7. Estimated mean effect concentrations (
1 SD) for (a) germination, and (b) growth. Data
were grouped according to collection location and
season. Dashed lines represent overall means for
each kelp population, and asterisks indicated
statistically significant differences (p lt 0.05).
Scientists at Nautilus Environmental (Nautilus)
located in San Diego, CA have collectively been
conducting the giant kelp (Macrocystis pyrifera)
germination and growth toxicity test since its
development by the Marine Bioassay Project (MBP)
in the early 1990s. Despite the years of
experience in conducting the test, the laboratory
still exhibits a significant number of test
failures, most
Conclusions
Based on this evaluation of giant kelp reference
toxicant test data from six laboratories in
California, it appears that the NOEC criterion is
not robust. Lines of evidence include the
following 1) The among-laboratory mean NOEC
value exhibited a standard deviation range that
overlapped the criterion 2) Four of the six
laboratories evaluated exhibited mean and/or
standard deviations that exceeded the criterion
3) Kelp collected in San Diego was less sensitive
to copper during the second half of the year and
4) The shift in Macrocystis copper sensitivity
likely contributed to the number of failures due
to the NOEC criterion because most of the
observed failures occurred with tests conducted
with San Diego kelp during the second half of the
year. In addition to the potential need to
adjust the NOEC acceptability criterion, a
procedural change may be warranted as well. Kelp
desiccation period was a significant source of
variation in test sensitivity germination EC50s
and growth EC25s occurred across a wider range of
concentrations when fresh kelp was used for
testing as opposed to kelp that was desiccated
overnight. In addition, the laboratory with the
greatest percentage of failures due to the NOEC
criterion (Lab 5), had the greatest number of
tests conducted using fresh kelp (18 of 20)
compared to the other laboratories with mixed
datasets (10 of 20 tests each). Therefore, to
reduce variability (and possibly reduce the
number of test failures), laboratories should
consider solely using kelp that is desiccated
overnight.
frequently due to the no observed effect
concentration (NOEC) criterion for the test.
During the MBP study, conducted at the Marine
Pollution Studies Laboratory Granite Canyon
(Monterey, CA) regional and temporal differences
in kelp spore germination and growth were
evaluated. Anderson et al. (1990) found that
there were no regional differences in copper
sensitivity, but there was significant temporal
variability. Confidence in conclusions about
temporal variation was high however, conclusions
about regional differences in Macrocystis
sensitivity to copper were presented with the
qualifier that multiple kelp collection locations
within California were evaluated only once. They
recommended that the tests be repeated several
times over the course of the year to confirm
their results. Regardless, the acceptability
criteria for the Macrocystis test were
incorporated into the USEPA guidelines based on
results from that single study. In response to
the multiple in-house invalid reference toxicant
tests due to the NOEC criterion, and the manner
in which the criterion was assigned, other
laboratories that conduct the giant kelp test
were contacted to determine if they were having
similar problems meeting the test acceptability
criteria. Six laboratories (one in Northern
California, and five in Southern California)
agreed to provide data from their last twenty
reference toxicant tests (valid and invalid) for
inclusion in an evaluation of all four
acceptability criteria.
Once the robustness of each criterion was
determined, variations in test sensitivity among
laboratories were evaluated using germination
EC50, and growth EC25 data from each laboratory.
Potential sources of variation included 1)
Kelp Source (Monterey vs. San Diego) 2) Kelp
Desiccation Period (fresh vs. overnight) and 3)
Season (winter/spring, summer/fall).
Figure 1. Giant kelp reference toxicant test
within- and among-laboratory failure rates (n20
per laboratory).
Figure 4. Mean MSDp values across laboratories
for (a) germination, and (b) growth. Data are
presented 1 SD (n20 per laboratory). Dashed
lines indicate overall means and acceptability
criteria. Solid lines indicate 1 SD for the
overall mean.
Results
Test Sensitivity NOEC evaluation results
demonstrated that test sensitivity varied widely
among laboratories, and that the greatest
percentage of test failures was due to the NOEC
criterion. Therefore, an investigation into the
possible cause(s) of the variation in sensitivity
was undertaken. Although the only
sensitivity-related acceptability criterion is
for the growth endpoint, germination sensitivity
was evaluated for comparison purposes as well.
The first variable examined was kelp source.
Laboratories generally used kelp collected from
one of two locations, San Diego, CA (Labs 2, 3,
4, 5, and 6) or Monterey, CA (Labs 1, 2, and 4).
Germination EC50 and Growth EC25 data were
grouped according to kelp source location and
analyzed using a Mann-Whitney U test.
Germination EC50s were significantly different
between the two locations, while growth EC25s
were not (Figure 5). Although the Southern
California mean germination EC50 was lower than
that for Northern California, the range of values
was wider, as indicated by the larger standard
deviation (Figure 5). For the growth endpoint,
means, ranges, and standard deviations for the
Northern and Southern California datasets were
nearly the same (Figure 5).
Failure Rate The overall (among-laboratory)
failure rate for the giant kelp test was 28
percent. Three laboratories exhibited failure
rates above this mean, and three below.
Regardless of whether the within-laboratory
failure rate was above or below the overall
failure rate, the largest percentage of observed
failures was due to the NOEC criterion. The
percentages of failures due to the minimum
germination and growth criteria for test controls
were negligible for all laboratories. Failures
due to the MSD criteria were negligible for all
laboratories save one (Lab 3) (Figure 1).
Germination Growth Mean control germination
and germ-tube length were typically well above
the acceptability criteria for all laboratories
(Figure 2). Within-laboratory variability was
low, causing small differences in means to be
statistically significant. However, since all
means were above the criteria (and therefore
likely to be irrelevant), the statistical
differences are not depicted in the figure.
Based on these results, the minimum control
germination and germ-tube length criteria are
robust.
Figure 5. Estimated mean effect concentrations (
1 SD) for Northern (n45) and Southern (n75)
California kelp populations. Dashed lines
represent overall means for each endpoint, and
asterisks indicated statistically significant
differences (p lt 0.05).
Acknowledgments
Methods
We would like to thank the participating
laboratories and acknowledge the effort they put
into compiling and providing their reference
toxicant test data. Data from the following
laboratories were included with Nautilus data in
this study City of Los Angeles, Environmental
Monitoring Division (Playa del Rey, CA) City of
San Diego, Metropolitan Wastewater Division (San
Diego, CA) County Sanitation Districts of Los
Angeles, San Jose Creek Water Quality Laboratory
(Whittier, CA) Marine Pollution Studies
Laboratory Granite Canyon (Monterey, CA) Weston
Solutions, Inc. (Carlsbad, CA)
There are four acceptability criteria for the
giant kelp toxicity test 1) Mean control
germination of 70 percent 2) Mean control
germ-tube length of 10 µm 3) Growth NOEC value
of lt 35 µg/L copper in the reference toxicant
test and 4) Germination and growth MSD values of
lt 20 percent in the reference toxicant test.

Figure 6. Estimated mean effect concentrations (
1 SD) for (a) germination, and (b) growth. Data
were grouped according to collection location and
desiccation period. Dashed lines represent
overall means for each endpoint, and asterisks
indicated the number of statistically significant
differences (p lt 0.05).
Figure 2. Mean control responses across
laboratories for (a) germination, and (b) growth.
Data are presented 1 SD (n20 per laboratory).
Dashed lines indicate overall means and
acceptability criteria. Solid lines indicate 1
SD for the overall mean.
References 1) Anderson, B.S., J.W. Hunt, S.L.
Turpen, A.R. Coulon, and M. Martin. 1990.
Copper toxicity to microscopic stages of giant
kelp Macrocystis pyrifera interpopulation
comparisons and temporal variability. Marine
Ecology Progress Series 68 147-156. 2) NOAA.
2002. Climatic Diagnostics Center,
http//www.cdc.noaa.gov. and 3) USEPA. 1995.
Short-Term Methods for Estimating the Chronic
Toxicity of Effluents and Receiving Waters to
West Coast Marine and Estuarine Organisms (EPA
/600/R-95/136). Office of Research and
Development, Washington DC.