FY 2003 Mainstem/Systemwide proposal 200303600

Section 1. Administrative

Proposal titleCollaborative, Systemwide Monitoring and Evaluation Program.
Proposal ID200303600
OrganizationColumbia Basin Fish and Wildlife Authority (CBFWA)
Proposal contact person or principal investigator
NameJann Eckman
Mailing address2501 SW First Ave. Portland, OR 97201
Phone / email5032290191 / jann@cbfwf.org
Manager authorizing this projectJann Eckman
Review cycleMainstem/Systemwide
Province / SubbasinMainstem/Systemwide /
Short descriptionThis project proposes an integrated effort of state, tribal and federal fisheries managers to catalogue, make available, critically assess, and improve system-wide monitoring and evaluation for fish and ecosystem status.
Target speciesSalmon, bull trout, steelhead, lamprey.
Project location
LatitudeLongitudeDescription
Columbia River Basin
Reasonable and Prudent Alternatives (RPAs)

Sponsor-reported:

RPA
RPA 180:
185 to 190. Tier 3 studies of juvenile survival
191 to 193.
195
198
10.A.2
10.A.3

Relevant RPAs based on NMFS/BPA review:

Reviewing agencyAction #BiOp AgencyDescription
NMFS Action 180 NMFS The Action Agencies and NMFS shall work within regional prioritization and congressional appropriation processes to establish and provide the level of FCRPS funding to develop and implement a basinwide hierarchical monitoring program. This program shall be developed collaboratively with appropriate regional agencies and shall determine population and environmental status (including assessment of performance measures and standards) and allow ground-truthing of regional databases. A draft program including protocols for specific data to be collected, frequency of samples, and sampling sites shall be developed by September 2001. Implementation should begin no later than the spring of 2002 and will be fully implemented no later than 2003.
NMFS/BPA Action 180 NMFS The Action Agencies and NMFS shall work within regional prioritization and congressional appropriation processes to establish and provide the level of FCRPS funding to develop and implement a basinwide hierarchical monitoring program. This program shall be developed collaboratively with appropriate regional agencies and shall determine population and environmental status (including assessment of performance measures and standards) and allow ground-truthing of regional databases. A draft program including protocols for specific data to be collected, frequency of samples, and sampling sites shall be developed by September 2001. Implementation should begin no later than the spring of 2002 and will be fully implemented no later than 2003.

Section 2. Past accomplishments

YearAccomplishment
Not applicable.

Section 3. Relationships to other projects

Project IDTitleDescription
NWPPC Fish and Wildlife Program, and the NWPPC Information Evaluation and Planning Study. This proposal will strengthen the Planning Study by providing skilled personnel from federal, state and tribal agencies to: 1) document organize and analyze existing data.
NMFS Biological Opinion and Technical Review Teams. As described in section c, this proposal provides a mechanism for implementing the M&E recommendations of both the FCRPS BIOP and the Technical Review Teams. It does not duplicate the analyses of the TRTs.
USFWS Recovery Monitoring and Evaluation Group (RMEG). The USFWS is funding RME efforts on bull trout, and this proposal will not duplicate those efforts. Rather, it will allow the USFWS efforts to be integrated with other RME work .
Stock Assessment work by Groups Concerned with Harvest (e.g., U.S. v Oregon Technical Advisory Committee; WDFW and ODFW Status Report on Columbia River Fish Runs and Fisheries; Pacific Fisheries Management Council; and Pacific Salmon Commission.). All of these groups require consistent stock assessment information, with consistent sets of estimates regarding such critical variables as conversion rates, harvest rates, hatchery fractions, spawner escapements.
Subbasin Planning. Subbasin plans need to be consistent with province- and basin-level goals, objectives, and strategies. This proposal will help to realize a level of integration in M&E will ensure that the complete life is considered.
EDT. The data inventory, evaluation and monitoring design activities proposed in this project will help to integrate the data required to test the hypotheses embodied in EDT.
ESSA Technologies Ltd. Multi-watershed Innovative Proposal (ESSA 2002b). The ESSA proposal, if funded , will use a workshop approach over 18 months to conduct pilot tests of the effectiveness of habitat restoration activities, using existing data across different watersheds.

Section 4. Budget for Planning and Design phase

Task-based budget
ObjectiveTaskDuration in FYsEstimated 2003 costSubcontractor
1. Establish an Oversight Committee and Core Group. $0
1.1 Create and maintain Oversight Committee, CoreGroup, and project management system. ongoing $217,858 Yes
Outyear objectives-based budget
ObjectiveStarting FYEnding FYEstimated cost
1. Establish an Oversight Committee and Core Group. $0
1.1 Create and maintain Oversight Committee, CoreGroup, and project management system. 2004 2005 $435,717
Outyear budgets for Planning and Design phase
FY 2004FY 2005
$217,858$217,858

Section 5. Budget for Construction and Implementation phase

Task-based budget
ObjectiveTaskDuration in FYsEstimated 2003 costSubcontractor
2. Provide monitoring data, evaluations and improved M & E designs for Tier 1 questions. $0
2.1 Catalogue existing information relevant to Tier 1 questions. 2.1. Provide monitoring data, evaluations and improved M & E designs for Tier 2 questions. ongoing $23,025 Yes
2.2 Standardized methods to store/display Tier 1 data & metadata. ongoing $32,588 Yes
2.3 Develop and apply analytical methods to address Tier 1 questions ongoing $32,722 Yes
2.4 Recommend modified and new monitoring designs for Tier 1 ongoing $38,952 Yes
3. Provide monitoring data, evaluations and improved M & E designs for Tier 2 questions. $0
3.1 Catalogue existing information relevant to Tier 2 questions. ongoing $83,834
3.2 Standardized methods to store/display Tier 2 data & metadata. ongoing $130,606 Yes
3.3 Develop and apply analytical methods to address Tier 2 questions ongoing $123,495 Yes
3.4 Recommend modified and new monitoring designs for Tier 2 ongoing $69,345 Yes
4. Provide monitoring data, evaluations and improved M & E designs for Tier 3 questions $0 Yes
4.1 Pose hypotheses and organize data to test them ongoing $19,500 Yes
4.2 Organize, store and display data to be used for Tier 3 analyses. ongoing $94,378 Yes
4.3 Develop and apply analytical methods to address Tier 3 questions ongoing $87,267 Yes
4.4 Recommend improved management experiments and monitoring ongoing $45,193 Yes
Outyear objectives-based budget
ObjectiveStarting FYEnding FYEstimated cost
2. Provide monitoring data, evaluations and improved M & E designs for Tier 1 questions. 2004 2005 $0
2.1 Catalogue existing information relevant to Tier 1 questions. 2.1. Provide monitoring data, evaluations and improved M & E designs for Tier 2 questions. 2004 2005 $46,050
2.2 Standardized methods to store/display Tier 1 data & metadata. 2004 2005 $65,176
2.3 Develop and apply analytical methods to address Tier 1 questions 2004 2005 $65,445
2.4 Recommend modified and new monitoring designs for Tier 1 2004 2005 $77,905
3. Provide monitoring data, evaluations and improved M & E designs for Tier 2 questions. 2004 2005 $0
3.1 Catalogue existing information relevant to Tier 2 questions. 2004 2005 $167,668
3.2 Standardized methods to store/display Tier 2 data & metadata. 2004 2005 $261,213
3.3 Develop and apply analytical methods to address Tier 2 questions 2004 2005 $246,990
3.4 Recommend modified and new monitoring designs for Tier 2 2004 2005 $138,690
4. Provide monitoring data, evaluations and improved M & E designs for Tier 3 questions 2004 2005 $0
4.1 Pose hypotheses and organize data to test them 2004 2005 $39,000
4.2 Organize, store and display data to be used for Tier 3 analyses. 2004 2005 $188,757
4.3 Develop and apply analytical methods to address Tier 3 questions 2004 2005 $174,534
4.4 Recommend improved management experiments and monitoring 2004 2005 $90,387
Outyear budgets for Construction and Implementation phase
FY 2004FY 2005
$780,907$780,907

Section 6. Budget for Operations and Maintenance phase

Task-based budget
ObjectiveTaskDuration in FYsEstimated 2003 costSubcontractor
n.a. $0
Outyear objectives-based budget
ObjectiveStarting FYEnding FYEstimated cost
Outyear budgets for Operations and Maintenance phase

Section 7. Budget for Monitoring and Evaluation phase

Task-based budget
ObjectiveTaskDuration in FYsEstimated 2003 costSubcontractor
n.a. Work to be reviewed by ISAB $0
Outyear objectives-based budget
ObjectiveStarting FYEnding FYEstimated cost
Outyear budgets for Monitoring and Evaluation phase

Section 8. Estimated budget summary

Itemized budget
ItemNoteFY 2003 cost
NEPA N/A $0
Subcontractor ODFW $142,959
Subcontractor WDFW $137,093
Subcontractor CRITFC $137,093
Subcontractor Yakama Nation $40,311
Subcontractor Colville $40,311
Subcontractor Nez Perce $40,311
Subcontractor ESSA $164,491
Subcontractor IDFG $113,547
Subcontractor Eco Logical Research $58,754
Subcontractor MDFWP $123,893
$998,763
Total estimated budget
Total FY 2003 cost$998,763
Amount anticipated from previously committed BPA funds$0
Total FY 2003 budget request$998,763
FY 2003 forecast from 2002$0
% change from forecast0.0%
Cost sharing
OrganizationItem or service providedAmountCash or in-kind
USFWS .5 FTE $50,000 in-kind
NMFS 1.125 FTE $112,500 in-kind
CBFWA .75 FTE $75,000 in-kind
FPC .5 FTE $50,000 in-kind
Other budget explanation

We have only shown budget estimates for FY2003 to FY2005 because this is an application for three years of funding. If successful, it is anticipated that this collaborative M&E program will continue for more years. All subcontracts include salaries, benefits, travel and CBFWA's 12.8% contract administration fee. Task level budget are available for individual subcontractors. Additional budget refinements are expected as project coordination continues.


Reviews and recommendations

This information was not provided on the original proposals, but was generated during the review process.

Recommendation:
Fundable only if response is adequate
Date:
Aug 2, 2002

Comment:

This proposal addresses one of the major management deficiencies in the basin, the lack of a top-down basin-wide monitoring protocol. Such a protocol is of critical importance for assessing changes in stock and environmental conditions and the effectiveness of restoration and mitigation actions. Thus, this proposal is potentially a high priority for funding. Several issues, however, need to be addressed by the sponsors.

Implementation-related issues:

  1. Independent oversight of the project is needed to track progress and identify potential problems before they get out of hand. The ISRP should rigorously review the progress of this project annually if it is funded. All major work products such as the Design Plans and Data Analysis Reports should be subject to independent peer review by reviewers selected by the ISRP. The reports of the peer-review panels should be submitted to the ISRP and included in their annual review of the project. The sponsors should comprehensively respond to each concern raised in the ISRP's annual review. The ISRP's recommendations for future funding will be based in part on these annual reviews.
  2. To what extent does the proposed work overlap, duplicate, or complement other ongoing or planned monitoring and evaluation projects within the basin? If this project is funded will it replace the M&E component of other projects? If so, how will coordination be accomplished? How would this project affect the dedicated RME proposals submitted in this province? Action agencies may need to modify their current M&E protocols to conform to the recommendation of the basinwide program. Is there a firm commitment from state, federal, and tribal entities to adopt the monitoring protocols resulting from this project? The USFS and the BLM are not listed as cooperators in the proposal, yet they have management responsibility for the bulk of federal lands in the basin. How will coordination with these agencies be achieved?

Project Organization Issues:

  1. The key questions developed for each Tier should be explicitly related to general recovery goals and objectives for the basin and subbasins. For example, what are the basin-wide goals and objectives for salmonid recovery and how will addressing the key Tier 1 questions ensure progress toward meeting those goals? The same question could be asked for provinces, subbasins, and ESU's.
  2. The proposal focuses principally on development of a basinwide monitoring program but the evaluation of the data collected through monitoring is barely discussed. Evaluation is a critical component in the M&E process without which the key questions cannot be answered. The proposal needs to explicitly address how the evaluation component (i.e., analysis of the monitoring data) will be incorporated into the process. Specific methods for analysis are not required in the proposal but a general plan for the conduct of analyses is needed. Perhaps the proposal should be retitled "Collaborative, Systemwide Monitoring Program" with the monitoring data to be available to the region for evaluation?

Methodological Issues:

  1. The key questions mostly appear to address system state, for example, biological and physical habitat "condition." Physical (e.g., geomorphic and hydrologic) and biological (e.g., species interactions, habitat relationships) processes determine system state and are subject to modification by human actions. Critical ecological elements such as habitat stability and connectivity, metapopulation dynamics, and genetic and life history diversity are barely touched upon, if at all. The key questions and, in fact, the entire protocol development should be more process-based, explicitly emphasizing ecological processes and functions as much as states.
  2. Tier 3 is their "effectiveness monitoring of recovery projects" and is in direct competition with the BPA-RME program. The sponsors approach to effectiveness monitoring is an attempt to standardize research experiments from the top down, which may not be possible. Effectiveness monitoring (research) should be called for in RFPs to answer specific questions. For Tier 2 monitoring, impacts of non-native species need to be explicitly considered.
  3. The proposal needs to more clearly define "classes" of management actions (page 6).
  4. A monitoring and evaluation plan for the project itself is needed.
  5. What is meant by the statement"...Tier 1 data layers are intended to be coarse-scale assessments that do not capture interannual variation and spatial variation in covariate magnitude." Specifically what does "in covariate magnitude" mean?

In addition, the Coded Wire Tag Programs that are among the primary monitoring and evaluation programs for stock identification in the harvest, magnitude of harvest on various stocks, etc. should be brought under this integrated effort to catalogue, make available, critically assess, and improve systemwide monitoring and evaluation for fish and ecosystem status. Other projects that should be brought under this overarching project to provide system-wide monitoring and evaluation include parts of or all of projects #198810804 (StreamNet), #198712700 (Smolt Monitoring), #199008000 (PTAGIS), #199403300 (FPC), and #199602000 (CSS).

Action Agency/NMFS RME Group Comments:

PLANNING GROUP - This project is well written and has several valuable objectives and tasks that are needed by the region. However, most all of the objectives and tasks are currently underway as part of other regional processes and associated contracts or proposals such as:

  1. the NMFS Biological Opinion and the Federal Caucus' Basinwide Salmon Strategy RME Program;
  2. NMFS and USFWS TRT Recovery Planning;
  3. the NWPPC's Provincial Review Process;
  4. Data Protocols and Data Needs Assessment Contracts;
  5. Subbasin Planning;
  6. the Regional Analytical Advisory Committee;
  7. USFS , BLM, and EPA Monitoring Programs;
  8. Oregon and Washington State Monitoring Programs;
  9. the Lower Columbia River Estuary Program; and
  10. the Corps of Engineer's AFEP Program.
The NMFS and Federal Action Agencies have developed a draft RME framework that overlaps much of the needs of the Fish and Wildlife Program and other Federal and state RME programs. A regional workgroup session in September, 2002 with the formation of an RME Regional Coordination Group is already planned to provide a collaborative process for coordinating these overlapping programs. The state and tribal fishery agencies, CBFWA, USFWS, and the NWPPC will be included in this Regional Coordination Group as well as other key agencies for the RME Programs identified above. This coordination effort will include resident fish RME needs under the USFWS BiOp. The work proposed by 35033 would be redundant to these other processes and associated contracts. The proposal also appears to duplicate current CBFWA support contract objectives of coordinating the state and tribal fisheries agencies and the region. In addition, funding is proposed for federal and state employees that are already requirements under current programs and activities.

ISRP Remarks on RME Group Comments:

The RME Committee comments raise similar issues as those identified in the ISRP's question 2 under Implementation-related issues above.


Recommendation:
Core Program*
Date:
Oct 24, 2002

Comment:

This project would develop a regional M&E effort based on existing, on the ground monitoring efforts. The proposal was developed to address the Stock Status Program Summary specifically as it relates to the NMFS BiOp, as well as other aquatic resources. The proposal is the only existing proposal that addresses all the needs that have been identified by the Federal Action Agencies RME workgroup. * Failing to fund this project immediately will pose a high risk of significant and immediate negative impact on successful implementation of the Fish and Wildlife Program.
Recommendation:
Core Program*
Date:
Oct 24, 2002

Comment:

Due to the apparent lack of available funds for new start projects, the project sponsors reduced the budget by 3% across the board.
Recommendation:
Fund
Date:
Nov 5, 2002

Comment:

Fundable. High priority. Agree with CBFWA's Core Program ranking. This proposal addresses one of the major management deficiencies in the basin, namely the lack of a coordinated basinwide monitoring program. Such a program is of critical importance for assessing changes in stock and environmental conditions and the effectiveness of restoration and mitigation actions. Thus, this proposal is of urgent priority for immediate funding.

The sponsors answered all questions and addressed all concerns expressed by the ISRP in our preliminary review including the need for independent oversight and outside peer review. This project provides an urgently needed umbrella framework to 1) collaboratively develop systemwide M&E protocols and 2) coordinate data collection activities, protocols, and standards. The basic objective of the Collaborative System wide Monitoring and Evaluation Program (CSMEP) project is a coordinating mechanism for individual M&E projects rather than assuming all M&E activities into itself.

It appears to the ISRP that this proposal is in direct competition with the planned activities of the Action Agency/NMFS RME Group. Competition between the RME Group, currently funded by BPA, and this project CSMEP (#35033), proposed by CBFWA and recommended by the ISRP, is a problem to be resolved in the political arena. We emphasize that resolution of this competition and development of a coordinated Columbia Basin wide monitoring program is critical for collection of the best quantity, quality, and utility of scientific data to evaluate the efforts of the region to recover fish and wildlife habitat and populations.

Several ongoing monitoring and database projects are already under the general direction of the CBFWA or advisor committees made up of mostly CBFWA members, including parts of or all of projects #198810804 (StreamNet), #198712700 (Smolt Monitoring), #199008000 (PTAGIS), #199403300 (FPC), and #199602000 (CSS). If #35033 is funded then the functional melding of #35033 with these projects is likely assured. CBFWA as the project sponsors do not propose to formally bring other existing M&E projects under this project in the foreseeable future, but rather to coordinate activities with these other projects, and collaboratively improve the systemwide information to aid decision-making. As proposed by CBFWA, project #35033 does not propose to incorporate administration and implementation of these projects, or to dictate individual project M&E actions and protocols for existing M&E projects. However, project #35033 does propose to integrate relevant Tier 1, 2 and 3 data from component programs into a systemwide M&E program, and make recommendations for filling critical information gaps related to key management questions facing the region.

It was refreshing to see in the NMFS Proposal #35019 (Develop and Implement a Pilot Status and Trend Monitoring Program for Salmonids and their Habitat in the Wenatchee and Grande Ronde River Basins) that one of the Action Agencies agrees with the ISRP that the CBFWA proposal #35033 contains the necessary collaborative components to implement a comprehensive monitoring program basinwide. We note that NMFS is also an active member of the CBFWA. Proposals #35033 and #35019 (and by extension, parts of the other NMFS proposals #35016, #35020 and #35048) could be combined with other ongoing projects to provide a systemwide monitoring and evaluation project. The ISRP could not agree more with the statement in the NMFS proposal #35019 that "The absolutely essential elements of 35033 that the other projects lack is the basinwide perspective, both in the collaborative representation of nearly all fisheries management agencies, as well as the inclusion of fishes other than anadromous salmonids. Ultimately, the most efficient manner for the Columbia River basin to approach a comprehensive monitoring program would be in the form of integrated aquatic ecosystem health assessment. Components of the above 5 projects, plus many ongoing monitoring programs, if coordinated within a single purpose, design, and data management and evaluation framework, could produce the ideal monitoring program for the basin's aquatic natural resources." Unfortunately, there are some technical deficiencies or incomplete methods in the NMFS proposals #35016, #35020 and #35048 and the ISRP cannot give unqualified support to these proposals at this time.

The proposed Project #35033 is broader, both in scope and participation, than other M&E projects proposed in the systemwide province and, therefore, has a higher probability of success and should receive priority for immediate funding. The CSMEP project provides an environment for developing and coordinating common data collection protocols and standards. Several logistical and institutional issues remain to be resolved, but the ISRP believes that this proposal has the best potential to significantly improve the quantity, quality, and utility of scientific data for evaluation of fish and wildlife recovery efforts in the Basin.


Recommendation:
Date:
Jan 21, 2003

Comment:

Statement of Potential Biological Benefit
Indirect. Collaborative effort among agencies to document existing monitoring data, and assess the strengths and weaknesses of existing monitoring and evaluation efforts through workshops and work groups.

Comments
NMFS is included in proposal sponsors. Comment is inappropriate.

Already ESA Required?
No

Biop?
Yes


Recommendation:
Fund (Tier 1)
Date:
Jun 11, 2003

Comment:

Category:
1. Council Staff preferred projects that fit province allocation

Comments:
Check ties to overall CBFWA project (#1989-062-01) for any overlap.


Recommendation:
Date:
Aug 4, 2003

Comment:

Negotiations with CBFWA and other regional entities are ongoing, resulting in more focused project than originally proposed, nearing closure.
REVIEW:
NW Power and Conservation Council's FY 2006 Project Funding Review
Funding category:
expense
Date:
May 2005
FY05 NPCC start of year:FY06 NPCC staff preliminary:FY06 NPCC July draft start of year:
$968,802 $968,802 $968,802

Sponsor comments: See comment at Council's website