Implementing Breeding Bird Survey Review Panel Recommendations
USGS Patuxent Wildlife Research Center
20 March 2000
In February, 2000, USGS Patuxent Wildlife Research Center (Patuxent) received the report of the Peer Review Panel evaluating Patuxent's responsibilities in the Breeding Bird Survey program (BBS). The Center Director accepted this report on behalf of the Center.
This memorandum establishes a plan for implementing recommendations identified by the Review Panel as important for improving the quality and functioning of the BBS. This implementation plan and the Panel's report will be available for examination on the Patuxent home page (http//:www.pwrc.usgs.gov). Within the Center, the Chief of the Research Program and the Chief of the Monitoring Program are assigned responsibility for organizing the implementation of the report's recommendations. They are to report monthly, in writing, to the Center Director on progress in implementation. As Co-chairs of the BBS Implementation Team, they should add people to their Implementation Team, appoint task forces, and seek advice and counsel as they deem appropriate. Close consultation with the U.S. Fish and Wildlife Service (USFWS) and Canadian Wildlife Service (CWS) is essential, and the Co-chairs should seek appointment of liaisons from these Services to the BBS Implementation Team.
The Panel dealt with a broad suite of issues, including the design and volunteer base of the survey itself, data management procedures, analysis of data, and dissemination of BBS products. Some of the recommendations are straightforward and can be implemented, or at least initiated, without further evaluation. Others are more complex, requiring careful consideration of alternative approaches and close coordination between the Research and Monitoring Programs at the Center and with partners, particularly the USFWS and CWS.
To address the complex issues, the BBS Implementation Team will appoint short-term Task Forces with memberships appropriate to the issue and representing all units directly affected by the recommendations. In some cases it may be appropriate for the CWS or the USFWS to be represented on the task forces, if they choose. The Implementation Team is to remain sensitive to the fact that the BBS Review Panel's review was of the USGS program not the CWS program. Implementation will take place in full consultation with the USFWS under the auspices of the Memorandum of Understanding establishing a Partnership for Migratory Bird Conservation, comprised of the Office of Migratory Bird Management, North American Waterfowl and Wetlands Office, and Patuxent Wildlife Research Center.
The following is the list of the Panel's recommendations. Decisions, target dates, and responsible individuals are indicated. Reports from Task Forces must identify a clear process and time line for implementing each of the recommendations under its purview and indicate the individuals responsible for implementation. As a general rule, priority attention should be given those recommendations designated ''high priority" by the Panel. In addition to addressing the recommendations and findings of the Review Panel, the BBS Implementation Team should consider the future role and functions of the BBS within the North American Bird Conservation
Initiative, both as a component of a Nationwide bird monitoring program and in the on-the- ground delivery of bird conservation within Bird Conservation Areas. The Implementation Team should prepare a written strategic document to guide the development of the BBS over the next five years.
Summary of BBS Review Panel's Recommendations
Note: Priorities assigned by the Panel are shown as *** (high), ** (medium), and * (low).
Recommendation 1. The magnitude and direction of the potential on-road bias associated with the roadside nature of the BBS counts should be expressly investigated by testing for attraction or repellent effects of roads with paired counts on and off roads within the same habitat, replicated in different habitats and regions. Alternatively, putative bias could be addressed by building off-road sampling into the BBS protocol.**
Decision: See Task Force C below.
Recommendation 2. Investigate the feasibility of conducting a designed study to systematically investigate the extent, if any, to which species trends differ on and off roads.*
Decision: See Task Force C below.
Recommendation 3. Establish a working group to develop operational methods of obtaining periodic habitat and other environmental information along BBS routes (ideally every five to ten years).
Decision: Establish a Working Group by April 30, 2000 (Peterjohn, Pardieck, et al. in consultation with Sauer et al.). Produce progress report with recommendations by September 30, 2000.
Recommendation 4. Assess the extent to which the habitats sampled by BBS routes are representative of the geographic areas for which trends are reported, at regular and frequent intervals, preferably every five to ten years.***
Decision: See Task Force C below.
Recommendation 5. Give greater attention to the potential biases introduced into the BBS dataset by the loss of routes as a consequence of traffic growth. Conduct an analysis of the possible effects of route replacement on trend estimates and develop a Standard Operating Procedure (SOP) as to the most appropriate treatment of data involving route replacement.***
Decision: See Task Force C below.
Recommendation 6. Conduct a technical evaluation of the empirical effects of spatial and temporal variation in detection probability. Initial work should examine the feasibility of the alternative approaches to the problem, followed by implementation of the selected option.***
Decision: See Task Force C below.
Recommendation 7. The BBS should continue to study the magnitude of observer variability, its effects on BBS results, and methods of addressing observer effects during data analysis.**
Decision: See Task Force C below.
Recommendation 8. Continue effort to determine the strengths and weaknesses of alternative methods of trend analysis and the circumstances under which each is appropriately used.**
Decision: See Task Force C below.
Recommendation 9. Greater effort should be devoted to achieving appropriate geographic coverage by the BBS, specifically by filling major gaps in coverage within the existing BBS coverage area in the U.S. and Canada and by continuing efforts to extend the BBS into Mexico.**
Decision: Identify areas in U.S. and Canada that would benefit from expanded BBS route coverage. Report on need, feasibility, and resources needed by February 1, 2001. Report on progress and problems and resources needed for extending BBS into Mexico by February 1, 2001 (Pardieck, Peterjohn).
Recommendation 10. Patuxent and CWS researchers should periodically and formally review the technical research needs of the BBS program, taking costs and benefits into account, and make recommendations to BBS management as to priorities for ensuring that needed research will be carried out.*
Decision: Research needs in addition to those identified by the Panel should be a regular topic of discussion in the Monitoring Forum coordinated by the Chief of Monitoring and the Chief of Research (see Recommendation 29). In addition, a solicitation of ideas for new research on aspects of the BBS will be developed and posted conspicuously on the BBS site. On an annual basis the Chief of Monitoring and Research will review the needs identified and seek resources for addressing the needed research on a priority basis.
Recommendation 11. Promote greater outreach to BBS stakeholders, especially BBS State Coordinators. Major elements in this outreach should include increased personal contact between staff and volunteers, and the conduct by BBS staff of workshops and other activities strengthening a sense of participation by stakeholders in the BBS.**
Decision: (For recommendations 11, 12, and 13) Prepare a report that reviews how BBS staff presently recruit, train, and communicate with volunteers and the role of State coordinators in this process. Identify how this suite of activities could be improved through training workshops, and what resources would be necessary. Report by September 30, 2000 (Peterjohn, Pardieck).
Recommendation 12. Begin discussions with State Coordinators and BBS participants to identify ways to increase and strengthen observer participation in the BBS in preparation for its anticipated growth in scale and intensity.**
Decision: See Recommendation 11.
Recommendation 13. Develop programs to train observers to consistent high standards, particularly to reduce variation in detection probability between observers.**
Decision: See Recommendation 11.
Recommendation 14. Develop a SOP to enforce consistency in editing and to ensure that all edits to the database are reversible and clearly documented. These standards should be applied retrospectively, documenting where they cannot be met.***
Decision: Prepare draft SOP by July 31, 2000; Patuxent Research Program and possibly other user groups complete review by, August 31, 2000; Prepare final SOP by October 31, 2000.
Recommendation 15. Assess the utility or otherwise of non-standard routes and define appropriate SOP's for the treatment of recommended types.**
Decision: Report by November 1, 2000 and, is appropriate, develop SOP(s) by April 1, 2000 (Pardeick, Peterjohn).
Recommendation 16. Discontinue the practice of processing the historical files to yield stop-level digital data but allow access to the clerical records (with appropriate precaution against loss of the originals) upon request, with the expectation that the individual researchers involved will meet the costs of processing such data for specific research projects. The possibility of Patuxent receiving an archival copy of any new or revised digital data should be discussed with the Principal Investigator and the funding agency at the time of the request for access to the original.*
Decision: Adopt recommendation immediately (Howe, Peterjohn, Pardieck).
Recommendation 17. Make timely release of each year's BBS data a high priority.***
Decision: Submit report recommending timelines, process, and criteria for database suitability by June 30, 2000. Clearly identify resources needed for same-year delivery of data and tradeoffs associated with advancing the delivery schedule (Peterjohn, Pardieck).
Recommendation 18. Publish, either via the Web or in the literature, an assessment for every species represented in the BBS database, indicating geographic, temporal, and other major limitations in sampling that affect quality or generality of trend estimates. All Patuxent web sites presenting BBS results should indicate which species have had such limitations identified (preferably linking to the specific details).**
Decision: See Task Force A below.
Recommendation 19. Develop a SOP to ensure documentation of changes to BBS Web sites, at the minimum issuing a new version number for a modified program or dataset and documenting the changes. Such version numbers should be cited in all publicly available analyses.***
Decision: See Task Force A below.
Recommendation 20. On all BBS Web pages that present bootstrap analyses, provide an option for seeding the random number generation to ensure exact replication of bootstrap sequences.*
Decision: See Task Force B below.
Recommendation 21. The Patuxent Web sites should provide the user with more options for control of Web-based analysis, including user-defined aggregation of routes, looping of analysis over multiple species, and greater conformity to the principle of user control of navigation over the Web site.**
Decision: See Task Force A below.
Recommendation 22. Web sites maintained by Patuxent in relation to BBS activities should be clearly annotated as to their intended role, with cross-reference to the sister sites for information found uniquely there.*
Decision: See Task Force A below.
Recommendation 23. Explore the viability of simple indices of annual abundance that can be calculated independently of route-regression, even if with less precision, in consultation with experts on alternative methods.**
Decision: See Task Force C below.
Recommendation 24. Complete development of a Patuxent/CWS-approved computer program that is flexible, user-friendly and well-documented, for distribution to users wishing to run their own analyses.**
Decision: See Task Force B below.
Recommendation 25. The standard BBS analysis program should be available to qualified researchers, either as raw code or as library modules compiled from a commonly-used programming language, along with full program documentation.***
Decision: See Task Force B below.
Recommendation 26. CWS and Patuxent researchers should work together to decide on the best data-selection and weighting methods, and adopt the same analysis method. CWS should only retain differences that are justifiable in the Canadian context, and CWS presentations of Canadian results should clearly state why they may differ from results derived from Patuxent analyses for Canada.***
Decision: See Task Force B below.
Recommendation 27. Encourage greater communication on BBS between statisticians and researchers within BRD and CWS, both to avoid duplication and to promote collaborative efforts on mutually‑defined goals for research on BBS and its results, and encourage greater sharing of analysis programs and other information that would promote fulfillment of respective agency goals in each country.***
Decision: See Task Force B below.
Recommendation 28. Impacts of every proposed new Monitoring Group commitment should be assessed explicitly with respect to the budgetary and staffing needs of the BBS, and any additional workload imposed on BBS staff should be offset with additional resources, as appropriate.**
Decision: The Chief of the Monitoring Program will place a high priority on implementing BBS Review Panel recommendations and develop budget proposals to fund those actions that cannot be addressed with existing discretionary funding. On an annual basis, BBS needs will be carefully weighed against discretionary initiatives before allocations are made.
Recommendation 29. Patuxent senior management should identify and clarify the respective roles of key individuals within the monitoring and research programs regarding BBS coordination, data management, data analysis, product development and refinement, user requests, and information transfer.***
Decision: The key to maximizing the effectiveness of the BBS and its value to users is to maintain a dynamic equilibrium between the research component (rigorous evaluation of methodological issues and development and improvement of analytical approaches) and the operational management and routine analysis and delivery of BBS data and data products. This will succeed in the long term only if roles of research and operational personnel are clear, if the contributions of different personnel are clearly credited in delivered products, and if active communication among involved parties is maintained.
Responsibility for management of BBS volunteers, data collection, data editing, and storage and delivery of raw data will remain with the Monitoring Program (Pardieck, Peterjohn). With the addition of a statistician/programmer in the Monitoring Program (Recommendation 30), primary responsibility for routine analyses and delivery over the web, or through other means, of secondary data products will shift from the Research to the Monitoring Program. However, the Research Program will maintain a strong advisory role in the delivery of routine products and, as desired, a strong role in the development and improvement of technology associated with web delivery of BBS products, including GIS applications (Sauer et al.).
The new statistician in the Monitoring program will be expected to maintain a close working relationship with colleagues in the Research Program. This interaction and the overall dialogue among biologists and statisticians engaged in the BBS will be enhanced by establishment of an informal Monitoring Forum, chaired by the Chiefs of Monitoring and Research, which will meet on a regular basis to review and discuss research opportunities and progress, issues, and problems of both a technical and operational nature associated with activities in the BBS and other monitoring programs. Scientists in the Research program will be encouraged to continue to play a very active role in the critical evaluation of methodology of the BBS as well as other monitoring programs and in development of new analytical tools (Sauer, Link, et al.).
Recommendation 30. Add a full‑time operational programmer/statistician whose primary duties would be to supply operational support, to conduct routine analyses, and to produce standard products. This position should be assigned to the Monitoring group.***
Decision: A position for a statistician/programmer, who will conduct and deliver data analyses for the BBS and other survey and monitoring programs and provide expertise in monitoring design for existing and new programs, will be established in the Monitoring Program. The Chief of the Monitoring Program will prepare a position description and advertise the position as soon as it is confirmed that base funding is available. Once hired, the incumbent will liaison closely with quantitative biologists and statisticians in the Research Program and with external partners involved in bird monitoring.
Recommendation 31. Patuxent should undertake an expert feasibility study of the cost-effectiveness and utility of digitizing individual route stops and of implementing GIS capability within BBS operations. Any perceived need for digitized route locations prior to completion of this assessment should be contracted out rather than conducted in-house.*
Decision: Report on an approach to conducting a feasibility study by February 1, 2001 (Peterjohn, Pardieck).
Recommendation 32. Patuxent and CWS should be proactive in expanding recognition of the value and uses of BBS data, playing a larger role in integrating BBS results with other data and research.**
Decision: Identify new ways of applying BBS data and propose a process for promoting expanded uses of the BBS by March 31, 2001(Co-chairs assign representatives from each program and invite representation from management users, e.g. USFWS, CWS, NABCI bird initiatives).
Task Forces
A. Management of the BBS Web Presence (Recommendations 18, 19, 20, 21, 22) (Chiefs of Monitoring and Research, Co-coordinators).
The Chief of Monitoring, Chief of Research, and Chief of Science Support will establish this Task Force By June 1, 2000. The Task Force shall prepare a report by August 1, 2000, evaluating the feasibility of implementing each of the recommendations assigned and proposing a timeframe for implementation not to exceed 6 months for recommendation 19, 21, and 4 months for recommendation 22, as amended here. Recommendation 20 will require coordination with Task Force B. Recommendation 22 is amended to require that there be a single access point to the BBS through the Patuxent Home Page. The Task Force will design the basic structure of the BBS site, ensuring that all components are fully integrated in a logical manner, but in a way that gives clear credit to individuals responsible in past and present for developing and delivering is various sub-pages. The BBS site will be managed in a cross-Program manner and constitute a standard topic of discussion in the Monitoring Forum (see Recommendation 29).
B. Operational Aspects of Trend Analysis (Recommendations 24, 25, 26, 27) (Chiefs of Monitoring and Research, Co-coordinators).
This Task Force will be appointed by the Chief of Research and Chief of Monitoring by June 1, 2000, and CWS will be invited to designate a representative. The Task Force shall prepare a report by August 1, 2000, evaluating the feasibility of implementing each of the recommendations assigned and proposing a timeframe for implementation, not to exceed 4 months.
C. Research Needs (Recommendations 1, 2, 4, 5, 6, 7, 8, 23) (Chiefs of Research and Monitoring)
This Task Force will be appointed by the Chief of Research and Chief of Monitoring by May 1, 2000, and CWS will be invited to designate a representative. The Task Force will be responsible for (a) assessing the technical feasibility of conducting each investigation called for by the Review Panel; (b) defining the scientific problems to be addressed, in detail sufficient to serve as pre-proposals or requests for proposals, (c) identifying scientists capable of conducting the defined research. The target date for these products is July 1, 2000. (See decision for Recommendation 10 for procedures for identifying and addressing future research needs, beyond those mentioned by the Review Panel).