The Population Index and a Discussion of the Role Detectability Plays in Interpreting Population Counts

We monitor animals so that we can understand how their populations are changing and react to the resulting information. To understand these changes we must determine how many animals are in the area of concern across a series of moments in time. Unfortunately, rare is the circumstance that we have complete information. Animals hide from us, are sometimes too numerous to count, and we make mistakes: misidentifications, missing some individuals during our counts, and counting others more than once. Consequently, given these most imperfect circumstances we must find alternatives to getting a complete count.

Creating a population index is almost always necessary as a surrogate to a complete census. An index is an incomplete count believed to be proportional to the population size (e.g., the number of birds heard at a point, number of frogs counted within a given plot, the number of salamanders found along a stream transect, counts of tracks). An index also makes no effort to determine the relative detectability of the animals being counted nor does it estimate the missing fraction. As you will see, the main problem with use of an index is the assumption of a constant ratio between the index and the true population.

For example, if you are interested developing a way to monitor the trends in the number of red-backed salamanders (Plethodon cinereus) in your woodlands, a species which spends most of its time underground, you could place cut sections of boards (which salamanders like to hang out under) throughout the woods, periodically turn them over, and count the numbers of salamanders revealed under them. Given that your sampling frame and protocol were correctly established, you now have a population index to the number of salamanders present in your woodlands…the total or average number of salamanders found under boards each year. In analyzing and presenting this data (that is, numbers of salamanders found under boards each year) you have assumed that the number you found under boards has a consistent relation to the aCTUAL number of salamanders in the population. If moisture conditions (or some other unmeasured factor, for that matter) both varied from year to year and affected the number of salamanders under the boards, one would not know if changes in counts among years were due to a population change or to a moisture change.

In all the cases that we can think of, we know the above assumption regarding consistency (that you count the same proportion of animals each time) to be false. In the example above, you would intuitively expect that if you went and turned over your boards during 3 consecutive weeks you would find 3 different numbers of salamanders each time. Is it likely that the real number of salamanders in the area had changed during those time periods? No. But it is clear that your counting technique is imperfect, that is, it does not measure, without error, the ratio of the real number of salamanders to the number you found under boards.

So, therefore an index is a reflection of both the real numbers of salamanders present AND your ability to detect them. The consequences for monitoring are two-fold. First this change in detectability each time you go out to count will add further variation to your counts (this is often termed measurement error). The consequence of this added variation to a monitoring program is that your precision is decreased, thus while you still have the ability to detect changes, it may take you longer or you may need to sample more often to detect a change of a given size. Second, if your ability to detect salamanders, or whatever you are counting, changes OVER TIME (that is, you see proportionally more or fewer) then you have created a bias in your counts and your index is no longer an accurate reflection of changes in that population.

Some examples may help make this clear. One great impact on detectability that almost any survey faces is the impact that weather has on the number of animals counted. Bad weather lowers counts. Consequently, the savvy surveyor knows to eliminate counts taken during bad weather (or better yet, not to have gone out in the first place) as it will decrease their ability to detect trends.

The above is one common example, now let’s say that during the first years of your survey you had an exceptionally keen observer collect your data and then when they were no longer available you assigned another, less keen, biologist to do the job. If that second biologist’s skill is such that they misidentify, have poor eyesight, or simply miss more of the things being counted than the previous person due to their lack of experience or motivation, then (unless statistically corrected for) the resulting counts will be biased and you will see false declines in the counts. These declines would not be due to real declines in the population, but declines due to changes in the two observer’s skills. Clearly this is a not what you would want.

One of the most overlooked biases has to do with changes in habitat over time. Usually this problem occurs when open landscapes become less so due to the growth of trees or other tall vegetation. Imagine that you had begun counting ducks in a lake that was 90% open water. Over time the small original cattail rim at the edge of the lake grew wider, filling more and more of the lake and that a healthy growth of pickerelweed, spatterdock and waterlilies filled the remainder. Your ability to detect the ducks and coots that live on the lake is now very different from what you had originally (unless you had x-ray vision!). While there may be exactly the same number of ducks living on that lake, you would be counting a much smaller fraction because of the thick vegetation that blocked your view. The result…you falsely conclude that ducks have declined, when they have not.

Changes in technology can also create problems. If you were counting shorebirds with binoculars over a large mud flat and then some years later bought a high quality spotting scope, the result will be that your ability to detect and identify shorebirds will have increased and your counts will be biased upward compared to the earlier counts. Similar issues could come up with the use of tape recorders, computers, palm pilots or other devices that can change your ability to detect and/or identify the animals that you are counting. While there are ways that some of these changes can be accounted for with the right statistical corrections, these corrections are imperfect and there are many times when they are inappropriate. Be aware then of ANY change in how a survey is run, no matter what the technique, analytical approach, or species involved. Those changes may corrupt your ability to detect the very trends in animal populations that you had set out to determine.

The problem of unmeasurable and uncorrectable biases is the primary drawback of using an index as a means of monitoring populations. Alternatives to using an index are to count everything (usually termed a census rather than a survey and almost always impossible to achieve), to give up completely on the idea of monitoring trends (the head-in-the-sand approach), or to use a population estimation approach designed to correct for bias. We know that the complete count is near impossible and giving up is not often a good thing, so that leaves only estimation procedures as an alternative. Estimators provide a count of relative abundance or an estimate of density, but importantly also calculate the probability of detecting a species, thus permitting a correction for bias. Estimation procedures are explained more thoroughly on our pages that talk about estimating the percentage of area occupied and estimating population size, but their basic aim is to estimate the real number of animals that are out there by determining the proportion of animals missed on your counts.

The question arises then as to why you would ever want to use an index; because you never can be sure that bias might be creeping into your surveys. To answer that question you must weigh the alternatives. With both an index and an estimation procedure you take the risk that bias will alter your count in a way that could lead to false conclusions about population trends. In both approaches you take the risk that you may have mis-specified the model that relates counts to the true population and that you have otherwise violated that model’s assumptions. Tests are sometimes available that indicate whether a model’s assumptions are met or not, but often you must verify the assumption yourself, either through additional independent tests or by taking the risk that the model’s assumptions have been met.

If the amount of bias within an index is demonstrated to be low, an index will usually require fewer resources and fewer samples than an estimator.

In general, it requires a great deal less time to collect, store, and analyze index data than estimation data. The are several reasons for this. Estimation procedures often require multiple visits to the same plot, may require time-consuming marking techniques, and almost always have greater information requirements. Data entry and storage are also increased, as is analysis time.

While estimators often take a great deal of extra time, they have important advantages over indices. They allow you to estimate density and, at times, other interesting demographic facets, such as immigation, emigration, births, deaths, and turnover. Additionally, when bias is unavoidable, large, and likely to change over time, the estimator provides the necessary counter resulting in a more accurate trend value.

Bias in an index has the greatest impact on trend when it changes greatly OVER TIME, whether due to changes in observers, habitat growth, or other uncontrollable factors. Bias is less of a problem if the types of trends you are trying to detect are long-term (10+ years) and large (>50% change). For variables other than trend (e.g., comparisons among species, or between habitats) bias can play an even larger role.

As an aside you will find that both indices and estimators will use the same counting techniques, but they differ in some of the details of how the data are collected, analyzed, and how many times the plots are revisited.

The use of indices is a controversial topic. The fear is that unmeasured biasing factors may corrupt an index so much that changes go undetected or false changes are concluded. The better alternative is to use an estimator if your budget can support the cost. In many situations, however, those costs are so prohibitive that estimators cannot be used. An alternative approach would be to use some of the estimation techniques as a pilot or as an ongoing fraction of the counts to estimate the amount of bias that may be occurring. If the bias is small in relationship to the standard error of the counts then use of a straight index may be justified. Hopefully over the upcoming decades many such studies will be conducted and we will be able to post estimates of bias on this web site. For small monitoring programs on individual refuges, parks, and protected areas, the technical author of this web site (Sam Droege) feels that in many cases index approaches are the most practical ones and should not be shied away from. If issues of weather, time of year, and observer bias are controlled for, then presence of long-term bias is likely to be small when compared with the magnitude of trends that are likely to be detected.