Charles Francis James Stuart Paul Geissler Charles Francis
From: "Charles M. Francis" <firstname.lastname@example.org>
Date: 29 Feb 96 14:11:32 EST
Subject: Monitoring Protocol
I would like to point out that the "someone from LPBO" referred to in Sam's recent comments was actually Christine Bishop, who is not from LPBO, although LPBO is assisting with running one of the 3 Ontario surveys (the Marsh Monitoring Program). I would like to remind people to sign their comments, because some people's mail software does not shown them all of the header information, including the name and E-mail of the sender.
A while ago I prepared a few comments on some aspects of the proposed monitoring protocols. I thought some of these might be of interest to subscribers to this list:
One approach listed in the standard protocol document suggests randomly selecting routes along rural roads, then selecting wetlands along those routes using an unspecified sampling protocol, and stopping at or near those wetlands to listen for amphibians. I see several problems with this subsampling protocol. If, as I understand is being suggested, stops are picked based on the distribution of wetlands at the start of the survey, and then one stick with those stops in future years, even if they later disappear, this could lead to a serious bias in the results if there are any fluctuations in numbers of wetlands present. The sample frame would then be "all wetlands present along roadsides at the start of the survey", rather than "all wetlands present along roadsides", which is what we would prefer (of course the ideal is all wetlands, but that is usually unattainable). The potential bias can be illustrated by an example. Let us suppose that a pond included in a survey dries up, because of a change in drainage (e.g. a beaver dam breaks), but a new wetland appears nearby, possibly because of the same event. It is quite possible that many amphibians will move from one site to the other, or at least that amphibians lost in one pond will be balanced by addition of amphibians in the other pond. However, the survey will indicate a decline, because the loss of the pond will be noted as a loss of amphibians, but unless the newly created pond is included in the survey, the increase in amphibians in the other pond will go unrecorded. In any area where water bodies are even partly ephemeral, this will lead to an apparent decline in overall amphibian numbers, even if true populations were stable.
There are two solutions to this. One, would be to re-select all wetlands every year, but this could be difficult to implement because of lack of information and/or because it is too time consuming. I also see problems with this selection process in the first place (even in the first year of a survey), because it is hard to construct a random sample of wetlands along a route unless they are all discrete (e.g. lots of little round ponds, which are not connected), which rarely happens. The document as outlined, provides two alternatives for picking wetlands (volunteers pre-surveying routes or survey designers using existing maps), but I see bias to both approaches. Maps will only show larger, less ephemeral wetlands, while volunteers may miss many wetlands that are in hearing range, but not visual range, of the road (perhaps because vegetation obscures the boundaries). This could cause significant bias. Also, the document does not specify how these wetlands are to be selected, apart from a minimum distance, which I think would cause problems with non-randomness.
A better alternative for a roadside survey would be to stop every 500m or 800m irrespective of habitat. In that way, the route would truly reflect habitat changes along roadsides and would be unbiased. It would also be much better with respect to sampling amphibians in very ephemeral habitats such as seasonal ponds or puddles that would not show up on wetland inventory maps, or may get missed by surveyors, or may be in different places each year. This is the approach presently being used in the Ontario roadside surveys (albeit not always, at least in the past, with random starting points).
If surveyors can judge habitat accurately from the roadside at night (something I would be dubious about), it might be possible to improve the efficiency of the survey by allowing them to stop for only a few seconds at inappropriate sites (long enough to judge it) and then move on, and only stopping for the full 3-minutes at sites with water. However, I doubt that would be viable except in really obvious cases (e.g. new shopping malls, although even then, amphibians may breed in the drainage ditches).
I realize that in some parts of North America, some areas may have so few wetlands that there will be serious problems with routes that only have one or two points suitable for amphibians. If that is the case, then an entirely different sampling protocol may be appropriate. For one thing, nothing precludes stratification in the initial route selection. It is both possible and appropriate, if wetland abundance varies geographically, to stratify areas prior to the random selection of starting points--e.g. to restrict route selection to rural areas with at least some wetlands, or to pick routes separately within different types of habitat (e.g. major forest strata or soil types, or prairie versus desert versus river basin, etc.). However, care is required in doing this, to ensure the population is clearly defined. The only other alternative for areas with very sparse wetlands might be to use the second approach in the document.
The other approach listed in the document (#2), consists of selecting wetlands to be sampled (rather than routes). This is also a viable approach, but only for well-defined (i.e. discrete) permanent wetlands. It is sampling a different population from a roadside survey (e.g. amphibians living in permanent wetlands that show up on inventory maps), and the conclusions that can be drawn would be quite different as a result. It would be quite viable to develop a good sampling protocol on this basis, though I am not at all certain about the best way to sub-sample (e.g. selection of survey points in a wetland). An approach such as this could be very suitable for sampling urban wetlands, which are generally very discrete (e.g. marshes surrounded by paved roads), and which can be sampled independently. Sampling based on wetlands is the approach being used by the Marsh Monitoring Program here at Long Point (although I fully agree that we have some problems with our sampling protocol, regarding lack of randomness).
It should be emphasized that there is no harm in using multiple sampling protocols in different areas of North America. The advantage of using a similar protocol over a large region (e.g. eastern North America) is that estimates of local trends can be combined with suitable weighting to estimate regional trends (as is done with the Breeding Bird Survey). However, if different states or provinces select different protocols, owing to differences in habitats, I do not see a problem with this. Generally, one tries to identify trends within habitats anyway. Use the technique appropriate to the region. Other protocols could also be developed, as long as the approach is taken of first selecting the sampling frame (the population of wetlands, roadsides or whatever that you are really interested in surveying, and that can be sampled), and then choosing a suitably randomized sampling protocol.
Obviously, of course, amphibian monitoring projects may have many different objectives besides monitoring long-term trends. As has been pointed out in recent discussions on this server, as well as elsewhere, education, public awareness, basic biology studies (e.g. studies of seasonal timing of calling) and so on can all be very important, and may not require any sort of random components to the sampling.
Also, I would like to point out that we criticize non-random samples as having potential biases,
because they may represent areas that are particularly rich or otherwise in certain species of
amphibians. However, I would like to remind everybody that random sampling based on roadsides
will not be any better if roadsides are not representative of the rest of the countryside. I am not an
expert in amphibian ecology, but I anticipate that in many parts of Canada or the north-east U.S.,
if a major change was implemented in the salts used for de-icing roads in winter, if any of these
salts had an impact on amphibians, the changes on roadsides could bear no resemblance to those
on marshes away from roads. On a less obvious line, major new developments have a tendency to
occur near roads, and I would anticipate quite different rates of habitat loss near existing roads
from more remote areas. In consequence, even a poorly randomized (due to difficulties of access
if not poor planning) sample of marshes away from roads, especially if selected independently of
known amphibian abundance, could provide a very valuable check on trends based on roadside
Charles M. Francis
Senior Scientist, Long Point Bird Observatory
From: "James N. Stuart" <email@example.com>
Date: Thu, 29 Feb 1996 14:40:33 -0700 (MST)
Subject: Re: Monitoring Protocol
RE: Charles M. Francis' comments on randomized surveying protocol...
I think several valid points are raised (or raised again, I suppose) concerning establishment of
routes. They are particularly important when looking at desert anurans. Here in the desert
Southwest, I would be very suspect of any survey results based on fixed sampling stations.
Breeding sites here are very ephemeral within a summer and are inconsistent from year to year.
Certainly there are natural basins that are more likely to hold water and attract anurans than
surrounding areas, but even these fill inconsistently depending on local rainfall patterns. Timing
of sampling is critical because you may only have a brief window of breeding opportunity, then
nothing for the rest of the year.
I certainly don't have the answer to how best to set up a protocol, but I do know that *flexibility*
in where and when you look is necessary even to locate many of our desert species, let alone to
monitor their status (chasing late afternoon thunderstorms is a popular past time for toad hunters
out here). I realize things are somewhat different in the more mesic areas of the continent. As was
suggested, protocols tailored to local environments are probably the way to go. --- jns
James N. Stuart E-mail: firstname.lastname@example.org
Museum of Southwestern Biology
University of New Mexico Phone: (505) 277-5130
Albuquerque, NM 87131 FAX: 277-0304
From Paul_Geissler@usgs.gov Mon Mar 4 08:32:46 1996
Subject: Re: Monitoring Protocol
Charles Francis posted interesting comments on alternate approaches to selecting points for
counting amphibians along randomly selected rural routes. My views of their advantages and
Identify potential wetland areas at the start of the survey and locate permanent points in
these areas to be measured each year.
Here the intent is to exclude areas that are not possible habitats for amphibians. It is wasteful to
ask volunteers to count amphibians where they could not possibly breed. It also has a bad effect
on volunteer morale and causes them to question our sanity.
Charles correctly points out that the trends would be biased toward declines if some wetlands at
the start of the survey disappear and are replaced by wetland in areas that were not classified as
potential habitat at the start of the survey. The bias results from our ability to only detect
decreases on observed wetlands but not corresponding increases in new wetlands. The bias
would only occur if the initial classification of potential wetlands missed areas that had the
potential of supporting amphibians. It is important to note that the suggestion was to identify
*potential* wetlands and not to limit sampling to areas that happen to support amphibians in a
Charles also points out problems in identifying wetlands because maps will only show larger, less
ephemeral wetlands, while volunteers may miss many wetlands that are in hearing range, but not
visible. I think the problem is in trying to locate specific wetlands, rather than broader areas of
potential amphibian habitat. If ephemeral wetlands are likely in an area, I think that sample points
should be located in that area. The intent was only to exclude areas where amphibian breeding is
extremely unlikely in any year. It is also reasonable to exclude very marginal habitats, because
loss of amphibians in these areas would have a negligible effect on the population.
Alternative 1 is an attempt at a compromise between:
Using volunteer selected routes or sites. They will probably select the best areas, resulting in the bias that Charles describes, because observed decreases would not be balanced by corresponding increases in areas that were less than optimum at the start of the survey.
Blindly selecting sites irrespective of the habitat, and counting at sites which could not
possibly support amphibians.
2. Identify wetlands each year and select new sample points each year.
I agree with Francis that this would be difficult and time consuming. It would also introduce
another component to the variance (the between site component), making it more difficult to
detect trends. However, in desert areas the pattern of wet areas may vary so much in both space
and time that an adaptive approach to identifying wet areas each year will be required, in my
opinion. I agree with James Stuart's comments about sampling desert anuran. That is a very
different and more difficult sampling problem, in my view. I am confident that some flexible
sampling protocol can be developed. However, it is probably going to be very difficult to catch
Select sample points irrespective of habitat.
This approach will work well in some states and provinces were almost all areas are potential
amphibian habitat. However in other regions, observers may waste time counting in areas that
cannot support amphibians. If one blindly lays down sample points without considering the
habitat, one will count amphibians at shopping centers, corn fields, jackpine barrens, and other
areas that everyone knows can't support amphibians. In alternative 1, we are accepting the
judgement of the person in the field that amphibians could not breed at certain sites in any year
and, in effect, enter a zero count without asking a volunteer to count at impossible sites. Not
only is it bad f or volunteer morale to ask them to count where we know there are no amphibians,
it is inefficient because constant zero counts provide no information on trends.
Charles notes that in some parts of North America, come areas may have so few wetlands that there will be serious problems. He suggests using another protocol in those areas. However, if alternative 1 is used (excluding areas that are not potential amphibian habitat), the same sampling design can be used in almost all states or provinces. This avoids problems with aggregating estimates from dissimilar surveys to obtain regional or continental estimates.
Charles has raised a number of excellent points. However in my view, consideration of the
alternatives brings us back to the original suggestion (1) as the best alternative. I appreciate his
comments because they serve to clarify points that may not have been clear in the earlier
I agree with Charles on the value of monitoring wetlands away from roads. Repeated annual
observations on a few sites would help determine if other populations have the same trends as
those near roads. However, wetlands near roads are likely to be the most impacted by
disturbance, resulting in conservative estimates. In many areas of the continent, essentially all
wetlands are near roads.
Paul H. Geissler
National Ecological Surveys Team
Office of Inventory and Monitoring
National Biological Service
12100 Beech Forest Road
Laurel, MD, USA 20708-4038
Tel. 301-497-5780, FAX 301-497-5784
From: "Charles M. Francis" <email@example.com>
Date: 04 Mar 96 18:30:47 EST
Subject: Sub-sampling protocol
Further ideas/issues in response to Paul Geissler's comments:
I understand the desirability of not wasting effort sampling completely useless sites. However, I am still uncertain how you are proposing to do the subsampling. . The protocol in the most recent version of the document that I have (dated 02/02/96) suggests that "wetlands along the roadside segment would be identified without prior knowledge of their capacity as amphibian breeding sites." Apart from specifying a minimum distance, it does not indicate how they should be selected. There is also a difference between "potentially suitable habitat" as described by Paul Geissler in his message, and "wetlands", as they would show up on any inventory map. I think it is important that survey points be selected either systematically, or randomly, neither of which is specified in the protocol. One possibility is to try to set up points at a fixed distance, but to extend the distance if the habitat is deemed unsuitable (this is quite different from identifying wetlands ahead of time). Then one has to decide how far to extend. For example, if there is a stretch of 3km of unsuitable habitat followed by 500m of woods with some damp area, which could support amphibians at certain times, followed by a marsh, should one place the survey point just inside the damp ground, or wait until the marsh is reached? Also, I'm not sure that it is always clear what constitutes unsuitable habitat. For example, corn fields that extend several km might be clearly unsuitable, but is a small corn field surrounded by trees necessarily unsuitable, given that farmers do change their crops, or abandon fields over time, and, at least in some areas, fields could have drainage ditches around them that might support amphibians?
Basically, I am in agreement with the concept of avoiding unsuitable areas--as was correctly pointed out, counting in useless habitat wastes the time of volunteers, and contributes no useful data. However, I still see a number of problems in the implementation of the concepts. Perhaps these are details that will vary from region to region, and do not need to be fully specified in a generalized document such as this one, but I think it could still be very helpful to people trying to establish survey programs to have some more detailed guidelines on the subsampling protocol, perhaps listing a few alternatives.
Another comment about the proposed minimum distance. If amphibians can be heard up to 500m away, then the spacing must be 1 km to avoid any overlap. Assuming that only a few frogs can be heard at that distance, the 800m suggested under approach #2 might be adequate, but 500m would not be. Also, under Approach #2, I assume the "no more than 500m apart on a walking survey" should read "no less than...."? I also assume one is prepared to accept greater overlap on a walking survey than a road-based survey?
Another issue is how far away from the initially selected random route starting point one should seek before giving up. If there are no wetlands, or very few wetlands, within 20km of the selected random point, should one abandon the route and select another? What criteria should be used?
I hope nobody objects to rehashing ideas, if some of these have been raised before, but I think we
would all agree that protracted exchanging of ideas prior to establishing surveys can save a great
deal more time trying to rectify problems afterwards.
Charles M. Francis
Long Point Bird Observatory (=LPBO for those who haven't guessed)