BPA Fish and Wildlife FY 1998 Proposal

Section 1. Summary
Section 2. Goals
Section 3. Background
Section 4. Purpose and methods
Section 5. Planned activities
Section 6. Outcomes, monitoring and evaluation
Section 7. Relationships
Section 8. Costs and FTE

see CBFWA and BPA funding recommendations

Section 1. Summary

Title of project
PATH - Facilitation, Tech Assistance & Peer Review

BPA project number   9600600

Short description
Facilitation, technical assistance and peer review for PATH (Plan for Analyzing and Testing Hypotheses). Test alternative hypotheses underlying key salmon management decisions in the Columbia River Basin; develop improved decision support tools; design research, monitoring and adaptive management experiments.

Business name of agency, institution or organization requesting funding
ESSA Technologies Ltd.

Proposal contact person or principal investigator

Section 2. Goals

General
Adaptive management (research or M&E)

Section 3. Background

Stream area affected

Project is an office site only   Yes

History
PATH began in 1995. In 1993 and 1994, funding was provided to facilitate cooperative efforts by the Bonneville Power Administration, the Northwest Power Planning Council, the National Marine Fisheries Service and various state and tribal agencies to compare and enhance the models they use to evaluate salmon management options. Results from these model comparison activities and associated peer-review efforts showed that each modeling system has different strengths and weaknesses, several common patterns of model behavior, and some significant differences. In 1994, an independent Scientific Review Panel (SRP) (coordinated by Dr. Larry Barnthouse, then of Oakridge National Laboratory) completed an interim report in which they concluded that there were three major differences between modeling systems: 1) the distribution of survival over the life span; 2) the effect of flow on survival; and 3) the benefit of transportation. The panel felt that as long as these differences exist the models were going to give different answers in a fairly predictable fashion. This would result in conflicting advice to decision makers and would make further analysis of details of model behavior relatively unproductive. The panel concluded that it would be more fruitful to focus on describing and attempting to resolve the fundamental issues, through hypothesis formulation and testing using Bayesian approaches. NMFS concurred with the recommendation of the SRP to conduct an analysis of alternative hypotheses. The 1995 NMFS Biological Opinion on operation of the federal Columbia River Power System (pg. 124, Rec. 17) stated that "The BPA shall participate with NMFS in activities to coordinate the regional passage and life cycle models and to test the hypotheses underlying those models." NMFS noted that the emphasis should shift to analyses that test the different assumptions underlying the models, rather than refining our understanding of how the models are different. This was the genesis of the Plan for Analyzing and Testing Hypotheses (PATH), though the detailed design was developed in consultation with senior decision makers from the NPPC, CORPS, BPA, NMFS, State and Tribal fish agencies. Critical to the success of PATH are three components: 1) facilitation of the interagency scientific working groups, 2) specialized technical expertise (e.g. Bayesian statistics, decision analysis, salmon population dynamics); and 3) external, independent peer review.
In the first year and a half of its existence, PATH has already made considerable progress. Specific achievements include:
· clarification of management decisions with senior personnel in the major institutions;
· development of hypothesis frameworks and sets of alternative hypotheses relevant to management decisions;
· considerable data reconnaissance, acquisition and refinement prior to completion of retrospective analyses of specific hypotheses;
· detailed retrospective analyses for hypotheses related to hydrosystem, habitat, hatchery and harvest management decisions;
· three workshops, each involving about 30 research scientists, to plan retrospective and prospective analyses, review the results of preliminary analyses and assess their implications for management decisions;
· a series of technical meetings of task work groups to advance progress on specific retrospective analyses;
· novel development and/or application of analytical tools to assist in decision making
· three-level hypothesis framework
· decision trees for hydrosystem, habitat and hatchery management decisions
· a Bayesian maximum likelihood estimation (MLE) framework to evaluate ability of different models to predict stock-recruitment patterns
· several different statistical analyses (cluster analyses, multiple regression, analysis of variance and covariance) to assess patterns implied by spatial and temporal contrasts in stock-recruitment
· a method for evaluating survival trends in the freshwater spawning and rearing life stage
· prospective analyses for determining the required improvements in life cycle survival
· development of a Bayesian population model to be used to simulate the implications of habitat. harvest, hatchery, and hydro management actions for survival and recovery of listed Snake River spring/summer chinook stocks
· a plan for formal decision analysis to assess through a variety of performance measures the effects of different combinations of actions in each of the four H’s (hydrosystem, hatcheries, habitat, harvest]
· numerous reports (see below)
· a 30-page Conclusions Document synthesizing the major findings from the 620-page Final Report on Retrospective Analyses for FY96, including outstanding information needs necessary to resolve major uncertainties
· a set of presentations on progress by PATH participants to the Implementation Team (IT) Committee on PATH and other IT representatives; members of the NPPC and the public; meetings with the Research Review Group of the IT; and meeting with the Independent Scientific Group (now the Independent Scientific Advisory Board) to coordinate our activities.

Biological results achieved
PATH has had indirect, yet significant benefits for anadromous fish: it has developed a rigorous framework of testable hypotheses, and generated results which shed light on the potential to develop management actions to recover endangered Columbia River salmon populations. This will ultimately benefit fish populations by making the best use of limited resources for fish population expenditures. The improved understanding generated through PATH’s retrospective analyses has helped to sharpen the focus of hydrosystem management decisions, and provided quantitative tests of the hypotheses that hydro, habitat, hatchery and harvest activities have contributed to declines in Snake River spring-summer chinook populations since 1970. These analyses are currently being extended to other regions (Mid-Columbia) and stocks/species (fall chinook, steelhead). The independent Scientific Review Panel (SRP) has given two very thorough and very positive reviews of PATH’s work to date. In addition, decision makers at the NPPC and Implementation Team have found the results of our work very helpful to their discussions on future management decisions. They are impressed by the clarity of our Conclusions Document, and the fact that such a diverse group of scientists have been able to achieve consensus.

Project reports and papers
The following list does not include two draft reports on retrospective analyses.
Marmorek, D.P. and I. Parnell (eds.). 1995. Plan for Analyzing and Testing Hypotheses (PATH): Information package for Workshop 1 - Design of retrospective analyses to test key hypotheses of importance to management decisions on endangered and threatened Columbia River salmon stocks. Prepared by ESSA Technologies Ltd., Vancouver, BC with contributions from ANCOOR (Analytical Coordination Working Group) and Dr. R. Deriso, 88 pp. and appendices.
Marmorek, D.R, I, Parnell, L. Barnthouse and D.R. Bouillon. 1995. PATH - Plan for Analyzing and Testing Hypotheses. Results of a Workshop to Design Retrospective Analyses. Prepared by ESSA Technologies Ltd. Vancouver, BC for Bonneville Power Administration, Portland, 71 pp. and appendices.
Marmorek, D.R., D.R. Bouillon, and I. Parnell. 1996. Plan for Analyzing and Testing Hypotheses (PATH): Results of the Kah-Nee-Ta Workshop on Retrospective and Prospective Analyses, (April 17-19, 1996). Prepared by ESSA Technologies Ltd., Vancouver, BC for Bonneville Power Administration, Portland, OR, 52 pp. and appendices.
Marmorek, D.R. (ed.)., J.J. Anderson, L. Bashan, D. Bouillon, T. Cooney, R. Deriso, P. Dygert, L. Garrett, A. Giorgi, O.P. Langness, D. Lee, C. McConnaha, I. Parnell, C.M. Paulsen, C. Peters, C.E. Petrosky, C. Pinney, H.A. Schaller, C. Toole, E. Weber, P. Wilson, and R.W. Zabel. 1996. Plan for Analyzing and Testing Hypotheses (PATH): Final report on retrospective analyses for fiscal year 1996. Compiled and edited by ESSA Technologies Ltd., Vancouver, B.C. 620 pp. Chapters:
1. An Overview of PATH and Retrospective Analyses
2. Level 1 Hypotheses
3. Contrasts in Stock-Recruitment Patters of Snake and Columbia River Spring and Summer Chinook Populations
4. Level 2 Hypotheses
5. Retrospective Analysis of Passage Mortality of Spring Chinook on the Columbia River
6. Hydro Decision Pathway and Review of Existing Information
9. Evaluation of Productivity and Survival Rate Trends in the Freshwater Spawning and Rearing Life Stage for Spring and Summer Chinook
10. Trends in Upstream Spawning and Rearing Habitat
11. PATH Hatchery Impacts
12. Review of the Influence of Climate on Salmon
13. Contrasting Stock-Recruitment and Harvest Patterns of the Columbia River Stream-type Chinook Populations
Chapters 3, 5, and 9 are about to be submitted to Canadian Journal of Fisheries and Aquatic Sciences.
ESSA Technologies Ltd. (compl.) 1996. Plan for Analyzing and Testing Hypotheses (PATH): Scientific Review Panel (SRP) reviews: FY96 final report on retrospective analyses - Sept. 26/96. Compiled by ESSA Technologies Ltd., Vancouver, B.C.
Marmorek, D. and C. Peters (eds.). 1996. Plan for Analyzing and Testing Hypotheses (PATH): Conclusions of FY96 retrospective analyses: December 10, 1996. Prepared by ESSA Technologies Ltd., Vancouver, B.C. 28 pp.
Barnthouse, L. (ed.), J. Collie, B. Dennis, S. Saila, and C. Walters. 1996. Plan for Analyzing and Testing Hypotheses (PATH): First Scientific Review Panel Report. Prepared by ChemRisk Division, McLaren/Hart Environmental Engineering Co., Oak Ridge, TN for Bonneville Power Administration, Portland, OR, 20 pp.
Barnthouse, L. (ed.), J. Collie, B. Dennis, S. Saila, and C. Walters. 1997. Plan for Analyzing and Testing Hypotheses (PATH): Second Scientific Review Panel Report. Prepared by ChemRisk Division, McLaren/Hart Environmental Engineering Co., Oak Ridge, TN for Bonneville Power Administration, Portland, OR, 20 pp.
Peters C. and D. Marmorek. 1996. Detailed notes from PATH Workshop 3, Wenatchee, Washington (October 7-11, 1996). Prepared by ESSA Technologies Ltd., Vancouver, BC for Bonneville Power Administration, Portland, OR, 24 pp.
Peters C. and D. Marmorek. 1997. Applying decision analysis to PATH: Discussion paper. 17 pp. And Appendices. January 15, 1997.

Adaptive management implications
The hypothesis and decision frameworks we developed and applied in PATH have provided a means of harnessing a wide array of information, analytical tools and unpublished scientific knowledge towards key management decisions. The analyses have clearly confirmed patterns of spatial and temporal change in spring-summer chinook stocks, which not only elucidate the most likely causes of recent declines, but also lay the groundwork for grouping stocks for future adaptive management experiments. By bridging across different types of data sets and studies (e.g. migration corridor survival, transportation benefit, spawner-smolt survival, spawner-recruit survival, climate and ocean indicators, land use and hatchery indicators), PATH has generated a higher level understanding of how to integrate across life history stages and spheres of management action (hydro, hatchery, habitat, harvest). This provides a concrete foundation for designing adaptive management programs and coordinating research initiatives. PATH scientists have recommended several specific research, monitoring and evaluation approaches to resolving critical uncertainties (e.g. the magnitude of delayed mortality of both transported and in-river migrants).

Section 4. Purpose and methods

Specific measureable objectives
1. RETROSPECTIVE ANALYSES: Publish peer-reviewed reports and journal articles demonstrating the overall level of support for key alternative hypotheses with implications for management decisions on endangered or threatened salmon populations of the Columbia River Basin. Provide succinct guidance to management agencies (particularly NPPC and Implementation Team) based on these outputs in written format and through oral presentations. Propose other hypotheses and/or model improvements that are more consistent with the data. Develop improved models that incorporate what has been learned from the retrospective analyses. In FY96 our retrospective analyses focused mainly on spring-summer chinook. There are five goals for FY97 retrospective analyses:
2) publishing several of the FY96 spring/summer analyses in a peer-reviewed journal;
3) completing a Conclusions Document summarizing FY96 results (now finished and distributed to NPPC and Implementation Team);
4) complete follow-up retrospective analyses for spring-summer chinook (for hydrosystem, hatchery, habitat, harvest and climate hypotheses, especially improving the quantification of habitat and hatchery impacts;
5) complete data acquisition, run reconstructions and retrospective analyses for fall chinook; and
6) complete data acquisition and run reconstructions for steelhead.
In FY98 further retrospective analyses will occur for steelhead, but most work on spring-summer and fall chinook will focus on prospective analyses, as outlined below.
2. PROSPECTIVE ANALYSES: Publish peer-reviewed reports and journal articles that use a number of analytical approaches that explicitly account for uncertainty (e.g. Bayesian statistics, formal decision analysis) to project the range of possible future states of salmon populations under different management actions. A related objective is to apply these analytical approaches to assess the ability of adaptive management experiments to distinguish among competing hypotheses from future information. In generating these outputs, we will gradually evolve a quantitative adaptive management framework for development and implementation of a regional salmonid recovery program. Advise various institutions (NMFS, NPPC, BPA, USFW) on research, monitoring and adaptive management experiments which would maximize the rate of learning and clarify decisions.

Critical uncertainties
Uncertainties:
1. Litigation among the agencies in PATH can disrupt good working relationships and impair productivity. There were no major court cases in FY96, but the amount of future litigation is highly uncertain.
2. The reopening of the NPPC Fish and Wildlife Program in FY97 could require significant time investment by many agency PATH participants, slowing progress on PATH deliverables.
3. Rates of Learning: There are serious limitations on how quickly we can improve the existing data base, due to both the serious condition of the stocks (which limits the possible range of experimentation and monitoring), and the time needed to accumulate sufficient statistical power to have ‘comfortable’ levels of confidence in decision paths taken. Decisions will need to be made under considerable uncertainty; PATH can help to maximize the learning associated with chosen decision paths, though the acceptable level of risk will ultimately be a political choice, not a scientific one. The formal decision analysis and Bayesian statistical approach used by PATH is intended to explicitly account for these uncertainties.
Risks:
As PATH is not a physical project, it does not have any risks to other stocks. Risk assessment of alternative management actions is a key component of PATH.

Biological need
Salmon populations in the Columbia River Basin have been in decline since the early days of western settlement. The annual production of the Snake River spring/summer chinook during the late 1800’s was probably in excess of 1.5 million fish or 39% to 40% of all Columbia River spring/summer chinook (NMFS Biological Opinion, 1995). Today the population of Snake River spring/summer chinook is approximately 0.5% of its historic abundance, with approximately 1,800 spring/summer chinook returning to the Snake River. The story is similar for the Snake River fall chinook. From 1938, when the gates closed on the Bonneville dam, to 1950, the returns of Snake River fall chinook fell from approximately 72,000 to 29,000. Today, approximately 350 Snake River fall chinook return. Such declines have led to both races of Snake River chinook being listed under the Endangered Species Act, though both have continued to decline since listing (NMFS, Proposed Recovery Plan for Snake River Salmon, 1995). Past efforts to halt the decline have been ineffective because they have not shared common objectives, authorities, or jurisdictions and have not been adequate to the task of balancing resource allocation decisions (NMFS, Proposed Recovery Plan for Snake River Salmon, 1995). Therefore, there is an urgent need for coherent, defensible guidance to decision makers.
PATH has made very significant progress in building constructive working relationships among scientists from agencies with very different perspectives (i.e. BPA, NMFS, CORPS, State and Tribal agencies, NPPC). This is due to both the facilitation expertise of ESSA and the involvement of objective, neutral scientists from outside of the region (Deriso, Botsford, Peterman, and the Scientific Review Panel). Whereas in past years decision makers were presented with conflicting advice from scientists within different agencies, PATH has helped to define many areas of common agreement and is specifying the information or experiments needed to resolve remaining key areas of disagreement. The high quality retrospective analyses cooperatively generated by the PATH group over the past 17 months, the PATH prospective analyses currently underway, and the conceptual foundation provided by the ISG in their Return to the River report, will together deepen the scientific basis for the difficult and urgent decisions that must be made.

Hypothesis to be tested
A number of testable hypotheses have been developed and tested as part of the PATH process. For the retrospective analyses, these hypotheses fall into three groups: 1) Level 1 Hypotheses are exploratory analyses to assess patterns of change in stock indicators over space and time to identify differences in trends among species and stocks, without investigating mechanisms to explain those differences; 2) Level 2 Hypotheses attempt to explain trends in stock indicators in terms of changes in either the survival of particular life history stages, or the stresses affecting life stage survivals, thereby providing inferences on where to focus management actions; and 3) Level 3 hypotheses explain mechanisms associated with observed trends in survival of key life stages identified at Level 2, and link directly to management decisions on hydro, harvest, habitat or hatcheries.

Alternative approaches
NA

Justification for planning
PATH retrospective analyses have provided insights on the relative contributions of hydro, habitat, hatchery and harvest activities to index stock trends over the last 40 years. These results have important bearing for the types of on-the-ground efforts which are likely to yield benefits in different streams, as distinct from those which will likely be ineffective or even potentially harmful. While this should not prevent on-the ground efforts from proceeding, the work in PATH can be used to place such projects in an adaptive management context (i.e. larger scale strategic thinking, selection of controls and before/ after monitoring) to maximize the learning and benefits associated with such a project.

Methods
PATH consists of an iterative series of workshops, analytical activities and reporting steps to test key hypotheses underlying management decisions, coordinated by an interagency PATH Planning Group. (The PATH Planning Group includes the PATH facilitator, David Marmorek (ESSA Technologies); H. Schaller, ODFW (representing the State fishery agencies); J. Geiselman, BPA (representing the power system operating agencies); C. McConnaha, NPPC; E. Weber, CRITFC; and C. Toole, NMFS.) The workshops and reports force participants to complete tasks, and provide for fruitful exchange, feedback and internal peer review. Both a core set of 25 PATH participants, and an extended set of 15 - 20 occasional participants, provide input to analytical activities. Interaction with the Implementation Team for the Draft Recovery Plan helps to priorize major goals.
Iteration within the PATH process occurs as the logical framework of hypotheses is revised over time in response to improvements in both information and analytical methods. This framework is intended to:
1. compile and analyze information to assess the level of support for alternative hypotheses relevant to key management decisions, identifying knowledge and data gaps that could be filled through management experiments, research and monitoring;
2. provide guidance to the development of regional programs that would stabilize, ensure persistence, and eventually restore depressed salmon stocks to self sustaining levels; and
3. provide a structure for an adaptive learning approach to development and implementation of a regional salmonid recovery program.
The overall PATH process has five features to ensure high quality outputs: 1) excellent fisheries scientists from the participating agencies; 2) active participation of three internationally recognized independent fisheries scientists in PATH workshops and technical meetings (Drs. Peterman, Deriso and Botsford); 3) the formation of interagency work groups to address specific topics, which ensures strong internal review of all work products; 4) overall coordination , mediation and integration by the PATH facilitator; and 5) external review by the Scientific Review Panel (Drs. Walters, Collie, Saila and Dennis).
PATH activities in FY96 culminated in the completion of a Conclusions Document, which summarizes the findings of retrospective analyses. This document represents the consensus view of PATH participants on what the data and analyses completed thus far say about possible reasons for the decline in abundance of Snake River spring/summer chinook. This document has been supplemented by a series of presentations to the NPPC, the Implementation Team, and the public. Summary outputs and quarterly presentations are an integral part of the PATH process and are an important means of communication between PATH and interested groups in the region. PATH products are also available on the BPA-maintained www site.
Some of the technical methods used to date in PATH were summarized in Section 2 on historical information. PATH uses a weight-of-evidence approach to hypothesis testing, looking for consistency across all available evidence, and the sensitivity of conclusions to the weights assigned to different data sources and analytical results. The retrospective analyses provide the foundation for prospective analyses. The FY97 goals for the prospective analyses are as follows:
P1. Estimate the improvement in life cycle survival required to reach various salmon objectives (survival, recovery, rebuilding) and the uncertainty associated with these estimates, using a Bayesian modeling approach that incorporates all uncertainties. These survival improvements can be expressed as Biological Objectives, consistent with the 1994 NPPC Fish and Wildlife Program. To develop goals for rebuilding, decisions will need to be made on which stocks are included. For survival and recovery goals these are clearly related to stocks listed under the Endangered Species Act. (This task nearly complete for Snake River spring-summer chinook as of Jan. 30/97).
P2. Develop a formal decision analysis framework, which provides a common framework for incorporating alternative management action packages, alternative states of nature (with their respective posterior probabilities based on retrospective analyses), and a variety of performance measures. The decision analysis framework will permit the calculation of the expected value of various performance measures (e.g. probably of survival, probability of recovery, expected rates of learning), given a number of different hypotheses about key processes, and their associated probabilities. In some cases (e.g. hydro) these probabilities may be computed from retrospective analyses, whereas in other cases (habitat, hatcheries) they may need to be more subjectively assigned. The development of a suite of performance measures will involve interaction with the Research Review Group and the Independent Scientific Advisory Board (ISAB). Development of a set of action packages for the decision analysis will involve interaction with the Implementation Team as well as other entities. A modular set of interacting software tools is expected to evolve within this task to permit all PATH investigators to flexibly explore the implications of alternative model formulations. This modular framework would include the output from different passage models; tests of this output against both stock-recruitment, passage survival and transportation studies (using an MLE framework to estimate Bayesian posterior probabilities); and a decision analysis tool to generate expected values of different performance measures given the model output and associated probabilities.
P3. Use of the decision analysis approach and other methods to assess the rate of learning associated with alternative sets of management actions, research and monitoring activities, and adaptive management experiments. This analysis of the benefits of different management and research directions would be linked to ongoing research, monitoring and evaluation programs, to assess how existing activities could be modified to better answer key uncertainties, and also to suggest new activities which could be added to those already planned. A component of this objective is to define performance measures which could be used on an in-season basis to optimize the likelihood of reaching survival improvement objectives.
The above three tasks will be completed in FY97 for spring-summer chinook. Prospective modeling and decision analyses for chinook and steelhead will be completed in FY98, though more qualitative assessments will certainly occur in FY97.

Section 5. Planned activities

Constraints or factors that may cause schedule or budget changes
As indicated above, lawsuits and NPPC re-examination of Fish and Wildlife Program could divert the time of some key PATH participants.

Section 6. Outcomes, monitoring and evaluation

SUMMARY OF EXPECTED OUTCOMES

Information products
Improved rationale for salmon recovery decisions on transportation, drawdown, in-river measures; habitat restoration; hatchery activities and harvest
Improved research, monitoring, and adaptive management strategies.
Increased consensus among scientists in different agencies on the state of knowledge, range of possible trajectories of endangered stocks, and appropriate management actions.
Improved consolidation of decision support and decision analysis tools, and stronger links of these tools to empirical evidence.
Publication of results in peer-reviewed journals.
Presentation to decision makers and the public.

Coordination outcomes
Better integration of decision making among management agencies through a clear framework for decision analysis and adaptive management.

Provisions to monitor population status or habitat quality
PATH’s research, monitoring and adaptive management recommendations will detail the most appropriate monitoring approach. This must consider the whole life cycle of the stocks of interest, as well as life stage specific measures.

Information feed back to management decisions
Quarterly meetings with Implementation Team, NPPC, Research Review Group, ISAB, System Configuration Team

Critical uncertainties affecting project's outcomes
Securing full commitment of PATH participants from senior managers in NPPC, State and Tribal fish agencies, BPA, NMFS, CORPS. PATH’s work is quite broad in nature and strategic. More tactical issues are best addressed by others.

Evaluation
See above sections. Evaluation essentially incorporated by the clarity brought to the decision making and recovery planning process.

Incorporating new information regarding uncertainties
The PATH Planning Group has weekly conference calls during which we adapt to new information and uncertainties.

Increasing public awareness of F&W activities
Through quarterly presentations to the NPPC and public; concise summaries of conclusions; and the BPA-maintained World-Wide Web site on PATH.

Section 7. Relationships

Opportunities for cooperation
PATH currently involves cooperation among scientists from NMFS, BPA, NPPC, ODFW, IDFG, WDFW, CRITFC, USFS, CBFWA, CORPS, as well as from a number of academic and research institutions (U.Washington, Simon Fraser University, UC Davis, UBC, U. Rhode Island, U. Idaho, Inter-American Tropical Tuna Commission) and private firms (ESSA Technologies, Paulsen Environmental Research, Don Chapman Consultants). In addition, the Independent Scientific Group has participated in PATH since its inception (Phil Mundy, Jim Lichatowich, Chip McConnaha, and recently Chuck Coutant). Close cooperation with the ISG is very important to PATH.

Section 8. Costs and FTE

1997 Planned  $450,000

Other non-financial supporters
NA

How does percentage apply to direct costs
Subcontractors represent about half of the total contract. In FY97, the total subcontractor mark-up is budgeted at $19,868 (out of total contract of $450,000).