Summary of BBS Symposium -- 4/22/2005


Symposium: Considerations for Improving the North American Breeding Bird Survey Sampling Frame


Introduction – K. Pardieck

The purpose of this symposium is to serve as a starting point to discuss and prioritize the efficient use of BBS resources to increase sample size, improve coverage, and improve species trend estimates.


Presentations (See Appendix A for abstracts):

1) REFLECTIONS ON 40 YEARS OF BREEDING BIRD SURVEY. – C. Robbins (C. Robbins ill; presented by B. Peterjohn)

2) THE BBS IN CANADA. – C. Downes







A. Royle


Facilitated discussion:

Facilitator -- Ken Rosenberg, Cornell University


     The preceding talks served well to outline some of the contemporary challenges facing the BBS.  The intention of this session was to discuss and explore future approaches and possible solutions to the aforementioned issues.


Discussion Summary (See appendix 1 for meeting notes.):


- Noise, traffic, and route retirement/replacement:


  • Route noise is an intensifying problem.  While the problem is documented in part through vehicle counts the situation stems additionally from a perceived increase in overall background noise.
  • State coordinators occasionally remark that there are routes in their state for which they cannot attract observers owing to the much decreased habitat and species diversity now found on those routes.
  • Observers play a key role in determining when routes are retired since they are the measurer of noise and roadway conditions in the field.  ‘Noise’ and ‘hazard’ are subjective parameters and the perception of what constitutes upper acceptable limits varies amongst observers.  Further, an observer’s willingness to run a route on Sunday likely affects his/her resolve towards either modifying or replacing the route.
  • It is in the best interest of the BBS to resolve route problems in a manner that satisfies observers, however, frequent modification and/or premature retirement of routes could potentially introduce a significant source of bias in the trend analysis.
  • As a cautionary practice, the consistent course of action within the BBS office has been to replace routes only when they are too busy to permit safe sampling or too noisy to hear birds.  There are no guidelines within the BBS protocol that rigidly state when a route should be modified or replaced.  This has allowed the BBS staff flexibility to extend the life of ‘problem’ routes via whatever case specific options are available (e.g., designating a route a “Sunday Route”, modifying several selected stops, etc.).
  • The issue of route retirement/replacement is further complicated by requests to re-open retired routes after noise or safety concerns have been mediated (e.g., traffic volume was redirected onto a new nearby highway).
  • From an analytical standpoint, it is unclear as to what role route retirement/replacement plays in introducing bias into the trend analysis.  Currently the analysis does not account for route retirements.  This information was not easily accessed prior to 1997, existing only in paper record, but since then has been included as a primary metadata element in the database.
  • Further evaluation of the analytical impact of alternative route retirement/replacement strategies is needed.


- New route additions


  • State coordinators regularly comment that, though vacant routes may remain in low populated areas of their state, they have many observers willing to take new routes in areas where existing routes are already filled.
  • The practice of the BBS office is to add new routes to states in sets such that each degree block receives an equal number of routes only after the majority of the state’s existing routes have already been filled.  This practice was adopted to prevent a potential bias from clustering of routes.
  • While it is without question that observer recruitment is very important to the BBS, it remains unclear whether a high but unequal gain in observers statewide represents a more improved situation than lower but equal statewide gains.  This question is of high importance at this time.
  • A key in coming up with a quantifiable answer is the establishment of clearly defined goals and objectives.  Ken suggested we already have them from PIF (?) and that’s what we will stick with: 1) Assess the range wide status of as many spp. as possible being able to detect which are declining 50% range wide, 2) Performing part 1 immediately above but at the BCR scale, and 3) finer scale data to come up with more habitat scale analyses.
  • Further evaluation of the analytical impact of alternative route addition strategies is needed.  The task is complicated and requires substantial programming.  It is best handled by a small group of BBS analysts working together through a series of closed meetings.


- Detection


  • Some analysts believe that variability in the species detection abilities of observers may be an important analytical consideration.
  • Others believe that variation between observers is inevitable given the nature of both hearing and learning and that this variation does not necessarily introduce bias since it is expected and accounted for by reviewers.
  • Some participants expressed concern that improving volunteer ability through training might introduce a new bias.  Response from some others to that was that, in terms of BBS longevity, that risk is acceptable given the potential improvement in data quality.
  • Recruiting good observers and holding them is the more important consideration.  This quickly led to the suggestion of mentoring as the answer.  Again, I chose not to include it but let me know if you think it’s worth while and we can look at my notes.



Post-meeting discussion identified the need to conduct a BBS Strategic Planning meeting to more clearly define the 5-year goals of the BBS, prioritize action items, and identify resource needs for accomplishing them.  The meeting is currently scheduled to take place in Laurel, Md. November 15-16, 2005.

APPENDIX A. Presentation abstracts in order of appearance.


1) Reflections on 40 Years of Breeding Bird Survey.  Chandler S. Robbins, USGS Patuxent Wildlife Research Center, Laurel, Maryland, USA 20708-4000

            After reviewing the history of monitoring in North America, I discuss the origin of the Breeding Bird Survey (BBS), including the need for such a program and field testing of methodology to determine optimum starting time, length of stop, number of stops per route, and acceptable weather conditions.  Before initiating the survey I ran sample routes in Maryland, New Hampshire, South Dakota, Colorado, and Alaska to become familiar with environmental factors affecting roadside counts.  I also ran the same route multiple times throughout a season to measure progressive changes in detectability, and did simultaneous double sampling to study observer effects.  The BBS was field-tested with 50 routes throughout Maryland and ten in Delaware in 1965, then launched in the states and provinces east of the Mississippi River in 1966, and in the rest of the continent in 1967 and 1968.  For quality control, all participants in 1965 were required, in addition to their assigned 50-stop route, to also run a 50-stop route that I had run the same year.  In the first few years the results were keypunched into cards, and the cards were sorted various ways to prepare summaries. The electronics age has completely changed input and analysis procedures and ways of displaying results, but the field procedures are still the same as forty years ago.  Thoughts on improving the BBS are shared.


2) THE BBS IN CANADA. C.M. Downes, Environment Canada, Canadian Wildlife Service, 1125 Colonel By Drive, Ottawa, Ontario, K1A 0H3.

In 1966, the BBS in Canada was started by Tony Erskine, Canadian Wildlife Service, in cooperation with Chan Robbins, USGS. Thirty-six routes were run in the Maritimes and Quebec.  Currently, some 450 routes are run each year throughout all provinces and territories.  The sampling goal in Canada is a minimum of one route per degree block.  Sampling intensity is governed mainly by the availability of qualified observers and, especially in the north, reliable roads.  Sampling intensity varies throughout the country with up to four routes per degree block in some areas especially where the population and road distribution is dense.  The analytical method used by the Canadian Wildlife Service weights the number of routes per degree block in order to reduce the bias of uneven sampling.  In 1996 a Grassland Bird Monitoring pilot project designed 31 routes using BBS protocol in areas with high concentrations of native grassland.  These routes are not yet included in the main BBS database and discussions on whether or not they introduce a bias are ongoing.


3) PRACTICAL BBS SAMPLING CONSIDERATIONS WHEN USING VOLUNTEERS.  Brent Ortego, Texas Parks and Wildlife Department, Victoria, TX 77905.

Although rigorously designed and recognized as needed by many ornithologists, the BBS lacked popular support within the wildlife management community during its early years.  As decades passed, data were shown to be closely related to actual population trends for many species and the BBS was accepted as a very useful survey of breeding birds across landscapes.  As we turned the corner on year 2000, the BBS was regularly used by Partners in Flight, conservation organizations, and public agencies for conservation planning.  The national BBS office has assisted greatly with the use of BBS by providing greater data availability and a variety of analytical tools to interpret data, but at the same time the national office needs to strengthen its leadership role in implementing and refining the BBS.  For example, a programmatic review of the BBS is needed to determine if adequate sample sizes are conducted -- within strata, by species, or by habitat.  Cost effective studies are needed to determine what magnitude of effort it will take to successfully monitor select species.  Many state agencies are developing plans for all birds, they need information on adequate sample sizes, and whether it is warranted to increase samples via paid staff or increased volunteer efforts to obtain data needed.

      A survey was conducted this spring of state BBS coordinators asking questions on how well does the BBS, the survey as well as the infrastructure/coordination supporting the survey, meet each state’s monitoring conservation needs.  Results from the survey will be summarized and discussed.


4) THE NORTH AMERICAN BREEDING BIRD SURVEY:  CREDIBLE, OR NOT?  John R. Sauer*, William A. Link, James D. Nichols, and J. Andrew Royle, USGS Patuxent Wildlife Research Center, Laurel, MD 20708.

A schism exists in how the BBS is viewed within the conservation community.  Some see the limitations of the survey as primarily logistical, while others reject the survey out of hand due to perceived limitations of the design.   This schism must be resolved if the BBS is to be of use in the often contentious field of conservation biology.  In this talk, I discuss a few of the issues associated with design and analysis of the BBS, and suggest some approaches to enhancing the credibility of the survey. One issue of particular relevance for discussion of the future of the survey is that of goals; modern survey uses require much more information than that provided by long-term trend estimates, and the presentation of survey results at temporal and spatial scales used in bird conservation places particular constraints on the survey.    Field data indicate that detectability issues and roadside sampling invalidate the use of the BBS for population estimation and other goals relevant to conservation.  Modifications to survey design can, and should, be implemented to enhance the value of the survey.  Model-based approaches to survey analysis enhance the quality of the results by accommodating some features that likely bias survey estimates, permit better integration with modeling for management of habitat features, and help identify lingering uncertainties with survey results that may require modification of the design of the survey.


5) OPTIMISING SAMPLING EFFORT ON REMOTE ROUTES.  B.T. Collins, Environment Canada, Canadian Wildlife Service, 1125 Colonel By Drive, Ottawa, Ontario, K1A 0H3.

            In remote areas there is often only a small roster of volunteers who can be assigned to BBS surveys but the target survey area may be extensive. In this situation there is need to design the survey to be as efficient as possible and there is an intuitive concern that a small number of sample sites may not adequately represent the target population.  In this study, the potential to increase the surveyed area through running individual routes intermittently is examined through a simulation study. In the simulations, population change through either a change in density or a change in extent are examined to determine if the design would be suitable in both scenarios. However, designs with intermittent visits may be more susceptible to problems caused by missing observations or changing observers and this is also investigated.


6) SAMPLE SIZE GOALS FOR MONITORING NORTH AMERICAN NONGAME BIRDS.  Jonathan Bart, USGS Forest and Rangeland Ecosystem Science Center, Snake River Field Station, 970 Lusk St., Boise, ID 83706.

            Reliable estimates of trend in population size for birds are critical for effective wildlife management, but they require long-term commitments and substantial resources.  It is thus important to establish quantitative targets for monitoring so that scarce resources can be allocated efficiently.  Work on quantitative goals for long-term, large-scale surveys for nongame birds began several years ago when the landbird, shorebird, and waterbird initiatives established quantitative goals for monitoring abundance.  A year ago, an analysis of Breeding Bird Survey routes was published which included the number of BBS routes needed - both with and without measures to reduce potential bias - to achieve the accuracy target for the majority of birds well-surveyed by the BBS.  This analysis restricted consideration to the BBS even though many species are well-surveyed by other programs too.  As a result, a new analysis is in progress that will include estimates of the accuracy of trend estimates from multiple sources of information and the predicted overall power that will result from combining the different trend estimates.  This work will place the BBS estimates in context with other, well-designed surveys and will provide, for the first time, a comprehensive approach for monitoring the abundance of non-game bird populations in North America.




J.A. Royle* and J.R. Sauer, USGS Patuxent Wildlife Research Center, Laurel, MD 20708.

            Route selection in the North American Breeding Bird Survey is based on a quasi-stratified random sampling design motivated (in part) by the desire to achieve unbiased estimates of trends and other summaries of avian population status. In practice, some departure from design intentions is realized because active routes become concentrated around urban areas, and this yields unbalanced sampling with respect to habitat and land use patterns, and temporal changes in land use.  The need to consider potential biases induced by factors not controlled for (or uncontrollable) by design has motivated the development of a model-based framework for conducting inference about population status and trend assessments from BBS data. The present modeling framework is sufficiently generic to allow consideration of designs that deviate from random sampling.  Thus, for example, redundant information that results from clustering routes around urban areas, or targeted sampling to assess specific hypotheses (e.g., about the effect of land-use patterns on population status), can be viewed not as deficiencies in the design, but as features that necessitate extension of existing models used for assessment.  In this paper, we consider whether the sampling design is relevant to conducting inference about population status and trends, and we provide a framework for addressing potential biases induced by an imbalance in spatial coverage of sampled routes.


Appendix B. Notes from meeting. 

(An attempt was made to capture everything said at the meeting but as you will note from the flow of statements and questions, some comments were missed.)


Keith Pardieck - Any questions about the presentations?


Fred Fallon – “John, looking at your data…any resistance from higher ups?”


John Sauer – Well, if they wanted to find fault, they could.


**** Group moved to adjoining room for facilitated discussion ****


Keith P. – OK, here’s Ken Rosenberg. He will be facilitating our discussion on potential approaches to improving the BBS sampling methodology.  What will be addressing are some widely known problems many of which were introduced and summarized by our speakers.


 Unidentified individual -- I want to leave with positive feelings and be gung-ho, but it sounds to me like we should just scrap it and restart.


John S. – Chan didn’t add the end of the British effort.  The BBS totally replaced and rectified the common bird census problems.  The BBS it’s in a format that could be corrected because it is structurally sound.


Unidentified individual - It’s astounding that this is so important and useful and so under funded, couldn’t we just find more funding?


Unidentified individual - Yes, maybe if you paid folks for some routes you could get better coverage.


Ken R. -- OK, let me start so that we don’t jump around and we can have a format that Keith P. can record and I can facilitate.  The talks were great in outlining the problem but few solutions offered. Let’s get deeper into some proposed solutions like Andy Royle’s degree block analysis.  A main question seems like – should we add new routes?


Dan Brauning – Who makes these decisions?  Who’s in charge?


Keith P. – There are no written protocols on this, it’s just pretty much been passed down verbally.  When routes are full in a state and the state coordinator feels there are enough participants to attempt to fill another set of routes then you add another set per degree block.


Dan B. – So it seems like no one’s in charge.


Bruce Peterjohn – It’s very much a consensus between the US and Canadian National BBS Coordinators.


? – someone voiced a question about when a route should be discontinued.


Keith P. – You just do what you can to keep it going.  If the route gets too busy (traffic noise) and you can’t hear or becomes unsafe we discontinue and start a new one.


Ken R. – Ok, let’s stay on that point; route noise.


Bruce P. – Well, traffic noise is getting worse.


John S.? – Dove survey has measured car noise and we could use that.


Ken R. – OK, so we do collect car #’s


Bruce P. – Problem is bigger than # of vehicles; it gets more down to measuring background noise.


Ken R. – Is there a need for specific guidance and to when we should drop routes?


Keith P. – We have some routes that we can’t fill because people don’t want to do them. Well, it’s in instructions about 3-4 cars per stop and requesting discontinuing route but it’s not enforced by us.  [Editorial comment: this statement is incorrect.  There is no direct guidance in the instructions on when to change a route.  Historically, traffic has generally been considered to reach nuisance levels when 3-5 cars passed by on average during a 3-min count. KP]


Ken R. – Should we do this [set rigorous and specific guidelines for discontinuing a route] (addressing John S. and Andy Royle)?


John S. – From an analytical point of view we don’t see stopped routes in the metadata.


Unidentified individual – What is metadata?


Unidentified individuals – Information (or data) describing the data.


Ken R. -- To some extent we need to know what of this metadata we need to collect as Connie said there are some routes we need to drop, it’s inevitable (safety, noise…).  Just a question of when and how much bias does it introduce.  OK, so seems like we need a list for management and research.


Keith P. – We now keep data on when route ends (since 1997) but I didn’t know you needed that or I would have let you know.


Connie Downes – Observer has asked to reopen routes after highway moves and noise decrease so I don’t know what stuff like that introduces. Noise is subjective; some observers complain more than others.  Maybe we need multiple state coordinators to evaluate routes on the finer level.


Ken R. – I’m hearing dropping data is minimum metadata that analysts need.  Keep all route drop data.  The noise thing has to get a point where you’re recording cars not birds it needs to stop. So, there’s a process for discontinuing routes that needs to be matured. OK, so there seems too much control in hands of observers since if an observer doesn’t want to run it on Sunday then it can be discontinued.


Connie D. – There’s no hard and fast way, it’s really a judgment call.


Fred F. – It’s not totally at the observer’s whim, the office reviews maps.


Connie D. – It’s not just dropping because we also justify routes and change stops which go into route history.  Maybe we need to encourage observers to come to state coordinators more.


One of the state coordinators -- I think state coordinators are willing but we need guidelines.


Keith P. – Well, putting it down on the state level means more work for the State Coordinator.


Ken R. – OK, well, the big issue is coming back to adding routes and Keith P.’s group needs to hear this.


Unidentified individual -- There are areas that need routes filled so adding routes seems premature.


Ken R. -- Yes, but that has been laid out in every opportunity.  Let’s assume that’s being taken care of through so let’s get back to adding routes.


Unidentified individual -- So, is detection a bias?  I’d like to know what the hearing standard is – maybe we need that.


Ken R. – Variability in obs. adds variation but not necessarily bias (because east isn’t better then west).  Can we explore that?  But the general issue of getting at obsv. ability, how important is that?


Jon Bart – I think it is important but I think it would be relatively easy to examine average ability in observers by using online education and testing program that will keep a record.


Fred F. – I think it is important.  I’ve found in doing these observation studies that many people don’t recognize something and once they do, problem is solved life long.


Ken R. – Improving volunteer ability introduces a bias, No?


Fred F. – Maybe it would but the BBS will persist in time and better now is good.


Unidentified individual -- We discussed yesterday assessment, certification and education.  But assessment would get at this and old observers could use this.


John S. – Some birders are just naturally better so we have to accept that when we review analysis.


Ken R. – But that’s what were saying – how much of a solution do we need?


John S. – Obsv. can’t get better always but they could for our first 40 years.


Unidentified individual - I think the bigger problem is recruiting skilled observers and keeping them.  I think people drop routes because if their species are going down they don’t see a reason to keep them.


Ken R. – So, you’re saying better back and forth about what they’re doing this for and their benefit.


Tom Fox - I think having assessment info in any way is important for measuring bias.  I’m hearing all 3 goals are important in volunteer reward and can be done in one package – no brainer.


Unidentified individual - I think motivation is important.  But observers leaving is important so maybe we need mentoring to maintain continuity.  I think this is what we need.


Ken R.   How many people use an assistant?  I wonder if that introduces a bias?


Brief discussion – using a recorder allows you to focus on the surroundings instead of the data sheet.  But calling out detections can also detract from aural detections.


Ken R.– OK, let’s get back to route additions only.


Fred F. – Should we add nocturnal routes?


Ken R. – That’s a good question.

-I think it would be very difficult.


[Brief discussion ]


Brent Ortego – What we need is an evaluation of the problem.


Ken – That’s been done and it’s published and available.

- It’s a question for the biometricians.


Ken   (Pressures analysts)


Andy Royal – I need to know your goals and objectives.  But lets just say you can’t lose by putting more obsvservers out there so lets just do that.


Ken – lays out the PIF goal


Charles Francis – I don’t think we can come up with a quantifiable answer in a room.  A few people sitting together in a room with more data on assumptions may be able to come up with this.  Right now I don’t think we have all assumption data we need right now.


Ken R.– If we don’t want this public than what is our action plan to get at this? – where do we go from here?


Rex Sallabanks – (direct to biometricians)

Do we have the data that will allow you to evaluate route addition scenarios?  We can determine a lot of things about what we want and what we can do but we can’t do the analyses to see if it works.


Brian Collins – Recruitment, we know we need that – that’s so important to the simulations we’ve run.


Rex S. – Yes, but that doesn’t tell us where to put the ones we have.


Fred F. – We randomly place in a degree block but maybe we need to get away from that.  Maybe we can do better by getting away from random.


Charles F. – Right now we’ve independent of habitat because we don’t know, but if we start doing that then we’re getting into a different thing.

1) What is BBS best way to increase our effort?

2)? If we can increase # in a state but unevenly how does that compare to an even but lower increase?


Jon Bart – We definitely can do that, but it is complicated.  It takes substantial programming.  We did the best we could with what we had and it’ll be published soon.  Keith P., you should do it with a group but it’s not going to be a two week plan.


Ken R. – Can we do this, Keith P.?   What is the plan?  Whatever’s done, it has to come out of the BBS office.  We need to know if we can get the resources and what it is.


John S. – What is it?  Banders send in tons of data not really helping but thinking they are.  But what we’re back to again it sounds as goals.   So, what’s your goal?


Ken R.– We’re all in agreement – range-wide we need to detect which birds are in trouble.  It’s a nested goal. Really –

            * What do we need to do to improve BBS?

            *We need to know if our efforts to get at regional information are sufficient.


            Can those best be met by changes in the BBS? And what do we need to do?


Charles F. - What John’s saying is that he needs something quantifiable, I mean, what is in trouble?


John S. – I think setting generic trend based goals at a regional level will not get what we want.  They’re not the species we want (only the well surveyed ones), etc.


Pam Hunt – Should we maybe focus on a species basis?


Ken R. – But, we’ve laid out a set of species that could be covered by BBS.  We want to know what we have to do to get there.  That’s a question, a goal.  Let’s take Ceruleans.


John S. – Problem is there are places with no BBS routes and habitat (ridged forests) not near roads.  The models then come in so BBS can only go so far before field work or modeling needs to pick up.


Rex S. – Well, the question the stats guys need to address is if we could put routes near Cerulean warblers?  Is that OK for analysis?  Is it compatible with the survey?

-It does come down to goals

– Of the three that you (Ken) listed, the second (w/BCR’s) applies to BBS but others seem mutually exclusive.  I’m convinced that on a managed landscape level BBS is useless and BBS is not at getting at some species.


Ken R. – I think the first goal (range-wide pops – what BBS is meant to do) is the one we should focus on.


Charels F. – BBS puts your local management into context (data from surrounding areas).


Ken R. – Whole other area we can’t resolve.


Charles F. –Clearly people need to be able to believe BBS

-Repeat J. Bart’s analysis seems very important

            *Assess the range wide status of as many spp. as possible

                        -being able to detect which are declining over 50% of range wide.

            *Performing (above) at BCR scale.

            *Finer scale stat data come up with more habit scale analyses, etc.