Summary of BBS Symposium -- 4/22/2005
Symposium: Considerations for Improving the North American Breeding Bird Survey Sampling Frame
Introduction – K. Pardieck
The purpose of this symposium is to serve as a starting point to discuss and prioritize the efficient use of BBS resources to increase sample size, improve coverage, and improve species trend estimates.
Presentations (See Appendix A for abstracts):
1) REFLECTIONS ON 40 YEARS OF BREEDING BIRD SURVEY. – C. Robbins (C. Robbins ill; presented by B. Peterjohn)
2) THE BBS IN
3) PRACTICAL BBS SAMPLING CONSIDERATIONS WHEN USING VOLUNTEERS. – B. Ortego
4) THE NORTH AMERICAN BREEDING BIRD SURVEY: CREDIBLE, OR NOT? – J. Sauer
5) OPTIMISING SAMPLING EFFORT ON REMOTE ROUTES. – B. Collins
6) SAMPLE SIZE GOALS FOR MONITORING NORTH AMERICAN NONGAME BIRDS. – J. Bart
7) SPATIAL COVERAGE AND INFERENCE: TRADE-OFFS BETWEEN SURVEY DESIGN
AND MODEL ASSUMPTIONS IN THE NORTH AMERICAN BREEDING BIRD SURVEY. --
Facilitator -- Ken Rosenberg,
The preceding talks served well to outline some of the contemporary challenges facing the BBS. The intention of this session was to discuss and explore future approaches and possible solutions to the aforementioned issues.
Discussion Summary (See appendix 1 for meeting notes.):
- Noise, traffic, and route retirement/replacement:
- New route additions
Post-meeting discussion identified the need to conduct a BBS Strategic Planning meeting to more clearly define the 5-year goals of the BBS, prioritize action items, and identify resource needs for accomplishing them. The meeting is currently scheduled to take place in Laurel, Md. November 15-16, 2005.
APPENDIX A. Presentation abstracts in order of appearance.
Reflections on 40 Years of Breeding Bird Survey.
After reviewing the history of monitoring in
2) THE BBS IN
1966, the BBS in
BBS SAMPLING CONSIDERATIONS WHEN USING VOLUNTEERS. Brent Ortego,
Although rigorously designed and recognized as needed by many ornithologists, the BBS lacked popular support within the wildlife management community during its early years. As decades passed, data were shown to be closely related to actual population trends for many species and the BBS was accepted as a very useful survey of breeding birds across landscapes. As we turned the corner on year 2000, the BBS was regularly used by Partners in Flight, conservation organizations, and public agencies for conservation planning. The national BBS office has assisted greatly with the use of BBS by providing greater data availability and a variety of analytical tools to interpret data, but at the same time the national office needs to strengthen its leadership role in implementing and refining the BBS. For example, a programmatic review of the BBS is needed to determine if adequate sample sizes are conducted -- within strata, by species, or by habitat. Cost effective studies are needed to determine what magnitude of effort it will take to successfully monitor select species. Many state agencies are developing plans for all birds, they need information on adequate sample sizes, and whether it is warranted to increase samples via paid staff or increased volunteer efforts to obtain data needed.
A survey was conducted this spring of state BBS coordinators asking questions on how well does the BBS, the survey as well as the infrastructure/coordination supporting the survey, meet each state’s monitoring conservation needs. Results from the survey will be summarized and discussed.
4) THE NORTH AMERICAN BREEDING BIRD SURVEY: CREDIBLE, OR NOT? John R. Sauer*, William A. Link, James D. Nichols, and J. Andrew Royle, USGS Patuxent Wildlife Research Center, Laurel, MD 20708.
A schism exists in how the BBS is viewed within the conservation community. Some see the limitations of the survey as primarily logistical, while others reject the survey out of hand due to perceived limitations of the design. This schism must be resolved if the BBS is to be of use in the often contentious field of conservation biology. In this talk, I discuss a few of the issues associated with design and analysis of the BBS, and suggest some approaches to enhancing the credibility of the survey. One issue of particular relevance for discussion of the future of the survey is that of goals; modern survey uses require much more information than that provided by long-term trend estimates, and the presentation of survey results at temporal and spatial scales used in bird conservation places particular constraints on the survey. Field data indicate that detectability issues and roadside sampling invalidate the use of the BBS for population estimation and other goals relevant to conservation. Modifications to survey design can, and should, be implemented to enhance the value of the survey. Model-based approaches to survey analysis enhance the quality of the results by accommodating some features that likely bias survey estimates, permit better integration with modeling for management of habitat features, and help identify lingering uncertainties with survey results that may require modification of the design of the survey.
5) OPTIMISING SAMPLING EFFORT
ON REMOTE ROUTES. B.T. Collins,
In remote areas there is often only a small roster of volunteers who can be assigned to BBS surveys but the target survey area may be extensive. In this situation there is need to design the survey to be as efficient as possible and there is an intuitive concern that a small number of sample sites may not adequately represent the target population. In this study, the potential to increase the surveyed area through running individual routes intermittently is examined through a simulation study. In the simulations, population change through either a change in density or a change in extent are examined to determine if the design would be suitable in both scenarios. However, designs with intermittent visits may be more susceptible to problems caused by missing observations or changing observers and this is also investigated.
6) SAMPLE SIZE GOALS FOR MONITORING NORTH AMERICAN NONGAME BIRDS. Jonathan Bart, USGS Forest and Rangeland Ecosystem Science Center, Snake River Field Station, 970 Lusk St., Boise, ID 83706.
Reliable estimates of trend in
population size for birds are critical for effective wildlife management, but
they require long-term commitments and substantial resources. It is thus important to establish
quantitative targets for monitoring so that scarce resources can be allocated
efficiently. Work on quantitative goals
for long-term, large-scale surveys for nongame birds
began several years ago when the landbird, shorebird,
and waterbird initiatives established quantitative
goals for monitoring abundance. A year
ago, an analysis of Breeding Bird Survey routes was published which included
the number of BBS routes needed - both with and without measures to reduce
potential bias - to achieve the accuracy target for the majority of birds
well-surveyed by the BBS. This analysis
restricted consideration to the BBS even though many species are well-surveyed
by other programs too. As a result, a
new analysis is in progress that will include estimates of the accuracy of
trend estimates from multiple sources of information and the predicted overall
power that will result from combining the different trend estimates. This work will place the BBS estimates in
context with other, well-designed surveys and will provide, for the first time,
a comprehensive approach for monitoring the abundance of non-game bird
7) SPATIAL COVERAGE AND INFERENCE: TRADE-OFFS BETWEEN SURVEY DESIGN
AND MODEL ASSUMPTIONS IN THE NORTH AMERICAN BREEDING BIRD SURVEY.
J.A. Royle* and J.R. Sauer,
Route selection in the North American Breeding Bird Survey is based on a quasi-stratified random sampling design motivated (in part) by the desire to achieve unbiased estimates of trends and other summaries of avian population status. In practice, some departure from design intentions is realized because active routes become concentrated around urban areas, and this yields unbalanced sampling with respect to habitat and land use patterns, and temporal changes in land use. The need to consider potential biases induced by factors not controlled for (or uncontrollable) by design has motivated the development of a model-based framework for conducting inference about population status and trend assessments from BBS data. The present modeling framework is sufficiently generic to allow consideration of designs that deviate from random sampling. Thus, for example, redundant information that results from clustering routes around urban areas, or targeted sampling to assess specific hypotheses (e.g., about the effect of land-use patterns on population status), can be viewed not as deficiencies in the design, but as features that necessitate extension of existing models used for assessment. In this paper, we consider whether the sampling design is relevant to conducting inference about population status and trends, and we provide a framework for addressing potential biases induced by an imbalance in spatial coverage of sampled routes.
Appendix B. Notes from meeting.
(An attempt was made to capture everything said at the meeting but as you will note from the flow of statements and questions, some comments were missed.)
Keith Pardieck - Any questions about the presentations?
Fred Fallon – “John, looking at your data…any resistance from higher ups?”
John Sauer – Well, if they wanted to find fault, they could.
**** Group moved to adjoining room for facilitated discussion ****
Keith P. – OK, here’s Ken Rosenberg. He will be facilitating our discussion on potential approaches to improving the BBS sampling methodology. What will be addressing are some widely known problems many of which were introduced and summarized by our speakers.
Unidentified individual -- I want to leave with positive feelings and be gung-ho, but it sounds to me like we should just scrap it and restart.
John S. – Chan didn’t add the end of the British effort. The BBS totally replaced and rectified the common bird census problems. The BBS it’s in a format that could be corrected because it is structurally sound.
Unidentified individual - It’s astounding that this is so important and useful and so under funded, couldn’t we just find more funding?
Unidentified individual - Yes, maybe if you paid folks for some routes you could get better coverage.
Ken R. -- OK, let me start so that we don’t jump around and we can have a format that Keith P. can record and I can facilitate. The talks were great in outlining the problem but few solutions offered. Let’s get deeper into some proposed solutions like Andy Royle’s degree block analysis. A main question seems like – should we add new routes?
Dan Brauning – Who makes these decisions? Who’s in charge?
Keith P. – There are no written protocols on this, it’s just pretty much been passed down verbally. When routes are full in a state and the state coordinator feels there are enough participants to attempt to fill another set of routes then you add another set per degree block.
Dan B. – So it seems like no one’s in charge.
– It’s very much a consensus between the
? – someone voiced a question about when a route should be discontinued.
Keith P. – You just do what you can to keep it going. If the route gets too busy (traffic noise) and you can’t hear or becomes unsafe we discontinue and start a new one.
Ken R. – Ok, let’s stay on that point; route noise.
Bruce P. – Well, traffic noise is getting worse.
John S.? – Dove survey has measured car noise and we could use that.
Ken R. – OK, so we do collect car #’s
Bruce P. – Problem is bigger than # of vehicles; it gets more down to measuring background noise.
Ken R. – Is there a need for specific guidance and to when we should drop routes?
Keith P. – We have some routes that we can’t fill because people don’t want to do them. Well, it’s in instructions about 3-4 cars per stop and requesting discontinuing route but it’s not enforced by us. [Editorial comment: this statement is incorrect. There is no direct guidance in the instructions on when to change a route. Historically, traffic has generally been considered to reach nuisance levels when 3-5 cars passed by on average during a 3-min count. KP]
Ken R. – Should we do this [set rigorous and specific guidelines for discontinuing a route] (addressing John S. and Andy Royle)?
John S. – From an analytical point of view we don’t see stopped routes in the metadata.
Unidentified individual – What is metadata?
Unidentified individuals – Information (or data) describing the data.
Ken R. -- To some extent we need to know what of this metadata we need to collect as Connie said there are some routes we need to drop, it’s inevitable (safety, noise…). Just a question of when and how much bias does it introduce. OK, so seems like we need a list for management and research.
Keith P. – We now keep data on when route ends (since 1997) but I didn’t know you needed that or I would have let you know.
Connie Downes – Observer has asked to reopen routes after highway moves and noise decrease so I don’t know what stuff like that introduces. Noise is subjective; some observers complain more than others. Maybe we need multiple state coordinators to evaluate routes on the finer level.
Ken R. – I’m hearing dropping data is minimum metadata that analysts need. Keep all route drop data. The noise thing has to get a point where you’re recording cars not birds it needs to stop. So, there’s a process for discontinuing routes that needs to be matured. OK, so there seems too much control in hands of observers since if an observer doesn’t want to run it on Sunday then it can be discontinued.
Connie D. – There’s no hard and fast way, it’s really a judgment call.
Fred F. – It’s not totally at the observer’s whim, the office reviews maps.
Connie D. – It’s not just dropping because we also justify routes and change stops which go into route history. Maybe we need to encourage observers to come to state coordinators more.
One of the state coordinators -- I think state coordinators are willing but we need guidelines.
Keith P. – Well, putting it down on the state level means more work for the State Coordinator.
Ken R. – OK, well, the big issue is coming back to adding routes and Keith P.’s group needs to hear this.
Unidentified individual -- There are areas that need routes filled so adding routes seems premature.
Ken R. -- Yes, but that has been laid out in every opportunity. Let’s assume that’s being taken care of through so let’s get back to adding routes.
Unidentified individual -- So, is detection a bias? I’d like to know what the hearing standard is – maybe we need that.
Ken R. – Variability in obs. adds variation but not necessarily bias (because east isn’t better then west). Can we explore that? But the general issue of getting at obsv. ability, how important is that?
Jon Bart – I think it is important but I think it would be relatively easy to examine average ability in observers by using online education and testing program that will keep a record.
Fred F. – I think it is important. I’ve found in doing these observation studies that many people don’t recognize something and once they do, problem is solved life long.
Ken R. – Improving volunteer ability introduces a bias, No?
Fred F. – Maybe it would but the BBS will persist in time and better now is good.
Unidentified individual -- We discussed yesterday assessment, certification and education. But assessment would get at this and old observers could use this.
John S. – Some birders are just naturally better so we have to accept that when we review analysis.
Ken R. – But that’s what were saying – how much of a solution do we need?
John S. – Obsv. can’t get better always but they could for our first 40 years.
Unidentified individual - I think the bigger problem is recruiting skilled observers and keeping them. I think people drop routes because if their species are going down they don’t see a reason to keep them.
Ken R. – So, you’re saying better back and forth about what they’re doing this for and their benefit.
Tom Fox - I think having assessment info in any way is important for measuring bias. I’m hearing all 3 goals are important in volunteer reward and can be done in one package – no brainer.
Unidentified individual - I think motivation is important. But observers leaving is important so maybe we need mentoring to maintain continuity. I think this is what we need.
Ken R. – How many people use an assistant? I wonder if that introduces a bias?
Brief discussion – using a recorder allows you to focus on the surroundings instead of the data sheet. But calling out detections can also detract from aural detections.
Ken R.– OK, let’s get back to route additions only.
Fred F. – Should we add nocturnal routes?
Ken R. – That’s a good question.
-I think it would be very difficult.
[Brief discussion ]
Brent Ortego – What we need is an evaluation of the problem.
Ken – That’s been done and it’s published and available.
- It’s a question for the biometricians.
Ken – (Pressures analysts)
Andy Royal – I need to know your goals and objectives. But lets just say you can’t lose by putting more obsvservers out there so lets just do that.
Ken – lays out the PIF goal
Charles Francis – I don’t think we can come up with a quantifiable answer in a room. A few people sitting together in a room with more data on assumptions may be able to come up with this. Right now I don’t think we have all assumption data we need right now.
Ken R.– If we don’t want this public than what is our action plan to get at this? – where do we go from here?
Rex Sallabanks – (direct to biometricians)
Do we have the data that will allow you to evaluate route addition scenarios? We can determine a lot of things about what we want and what we can do but we can’t do the analyses to see if it works.
Brian Collins – Recruitment, we know we need that – that’s so important to the simulations we’ve run.
Rex S. – Yes, but that doesn’t tell us where to put the ones we have.
Fred F. – We randomly place in a degree block but maybe we need to get away from that. Maybe we can do better by getting away from random.
Charles F. – Right now we’ve independent of habitat because we don’t know, but if we start doing that then we’re getting into a different thing.
1) What is BBS best way to increase our effort?
2)? If we can increase # in a state but unevenly how does that compare to an even but lower increase?
Jon Bart – We definitely can do that, but it is complicated. It takes substantial programming. We did the best we could with what we had and it’ll be published soon. Keith P., you should do it with a group but it’s not going to be a two week plan.
Ken R. – Can we do this, Keith P.? What is the plan? Whatever’s done, it has to come out of the BBS office. We need to know if we can get the resources and what it is.
John S. – What is it? Banders send in tons of data not really helping but thinking they are. But what we’re back to again it sounds as goals. So, what’s your goal?
Ken R.– We’re all in agreement – range-wide we need to detect which birds are in trouble. It’s a nested goal. Really –
* What do we need to do to improve BBS?
*We need to know if our efforts to get at regional information are sufficient.
Can those best be met by changes in the BBS? And what do we need to do?
Charles F. - What John’s saying is that he needs something quantifiable, I mean, what is in trouble?
John S. – I think setting generic trend based goals at a regional level will not get what we want. They’re not the species we want (only the well surveyed ones), etc.
Pam Hunt – Should we maybe focus on a species basis?
Ken R. – But, we’ve laid out a set of species that could be covered by BBS. We want to know what we have to do to get there. That’s a question, a goal. Let’s take Ceruleans.
John S. – Problem is there are places with no BBS routes and habitat (ridged forests) not near roads. The models then come in so BBS can only go so far before field work or modeling needs to pick up.
Rex S. – Well, the question the stats guys need to address is if we could put routes near Cerulean warblers? Is that OK for analysis? Is it compatible with the survey?
-It does come down to goals
– Of the three that you (Ken) listed, the second (w/BCR’s) applies to BBS but others seem mutually exclusive. I’m convinced that on a managed landscape level BBS is useless and BBS is not at getting at some species.
Ken R. – I think the first goal (range-wide pops – what BBS is meant to do) is the one we should focus on.
Charels F. – BBS puts your local management into context (data from surrounding areas).
Ken R. – Whole other area we can’t resolve.
Charles F. –Clearly people need to be able to believe BBS
-Repeat J. Bart’s analysis seems very important
*Assess the range wide status of as many spp. as possible
-being able to detect which are declining over 50% of range wide.
*Performing (above) at BCR scale.
*Finer scale stat data come up with more habit scale analyses, etc.