[env-trinity] Executive Summary of Independent Review of CVP/SWP OCAP BO

[Tom Stokely]

Review of the Biological Opinion 

of the Long-Term Central Valley Project 

and State Water Project Operations Criteria and Plan 

For 

Johnnie Moore 

Lead Scientist 

California Bay-Delta Authority 

December 2005 

Executive Summary 

Background and Charge 

Over the last 150 years, the Sacramento River has been engineered into a massive water delivery system, which includes various dams that have blocked access to much of the historical habitat of anadromous salmonids. Development of the basin's water resources has, in effect, unintentionally initiated a large-scale ecological experiment. The experiment examines whether the historical habitat templates, and their associated salmon and steelhead production systems, can be relocated below the migration barriers. This undertaking has, so far, put at risk of extinction three of the basin's four evolutionary significant units (ESUs): winter-run and spring-run Chinook salmon and steelhead in the Sacramento River are listed as endangered or threatened under the Federal Endangered Species Act (ESA). 

This is the report of the Technical Review Panel (Panel) convened by the CALFED Bay-Delta Program to review the Biological Opinion (BO) on the Long-Term Central Valley Project (CVP) and State Water Project (SWP) Operations Criteria and Plan (OCAP). The OCAP BO was issued by the National Marine Fisheries Service (NMFS), Southwest Region. The BO assesses the effects of the continued operations of the CVP and SWP on listed Chinook salmon and steelhead in the Sacramento River, and on coho salmon in the Trinity River. This review was initiated at the request of the National Marine Fisheries Service's Sacramento Office. 

The Panel's basic charge is to determine whether NMFS used the best available scientific and commercial information in developing the BO. According to our charge: 

"The Technical Review Panel's Charge is to evaluate and comment on the technical information, models, analyses, results and assumptions that formed the basis for the assessment of the proposed long-term water operations of the projects described. . For example, the panelists should review how NMFS assessed the individual responses of fish to certain effects (i.e., flows, water temperatures, diversions, etc.) and whether NMFS missed best available information on how fish are likely to respond to those impacts." 

Accordingly, the Panel considered whether the best available scientific information (including models and analyses) was discussed or cited in the BO, and how NMFS took this information into account in reaching its decisions. 

The charge to the Panel included two important restrictions on the content of the review. First, we were not asked whether NMFS made the proper determination regarding jeopardy. Therefore, we do not address that question in this report. 

Second, we were also not asked whether NMFS properly considered the effects of baseline conditions in assessing jeopardy. All analyses in the BO were based upon the incremental impacts due to the proposed operations of the projects, rather than the impacts of baseline plus project operations. The Panel understands that there was a rationale for this approach (i.e., baseline does not cause jeopardy; hence the incremental effects are evaluated by themselves). 

Technical Review Panel Document -12/30/2005 1 

But the Panel would feel remiss without at least mentioning that this is a critical assumption and that from a purely scientific view, populations may not respond linearly to progressively increasing losses of individuals. 

Findings 

The Panel is unanimous in its finding that the scientific information used in the BO is not the best available. As salient examples, NMFS ignored the potential effects of climate change in their analyses in the BO, and NMFS used a temperature-mortality model (LSalmon-2) that does not produce credible estimates of temperature-induced mortality. Other important factors, such as variable ocean conditions or the risks associated with hatchery-released fish, are described in parts of the BO, but how these factors were related to the conclusions regarding jeopardy were unclear to the Panel. 

The Panel identified three overarching issues, which if addressed, would improve the presentation of the analyses in the BO. Specifically, the BO would have benefited: 


    1.. 1) from a clearly articulated conceptual model 


    2.. 2) from an analytical framework (based on the conceptual model) for the various data analyses, statistical models, and analytical tools that were used 


    3.. 3) by placing its analyses in the context of an explicitly defined life cycle approach. 

In addition, the Panel identified 15 specific technical issues in the BO. These technical issues are described in detail in the body of this report; we highlight the major issues below. 


    a.. 1) Global climate change was not considered. 

The BO assumes that the climate and hydrologic regime during the last century will persist into the future. The Panel does not believe that global climate change (e.g., temperature warming), and the consequent temperature and hydrological changes, received adequate treatment in the BO. This deficiency resulted in an important uncertainty being ignored that could affect the characterization of the risk to the ESUs. 

2) Variability in ocean productivity, and its affect on fish production, was not incorporated into the analyses. 

The current status of the listed populations is, in part, an outcome of recent favorable ocean conditions. What will the status of listed populations be under less favorable conditions that may occur in the near future? By not including variability of ocean conditions in its analysis, the BO does not adequately address whether or not the listed populations are sufficiently large to survive a period of poor ocean conditions. 

3) Unknowns or uncertainty were either not adequately incorporated into the analyses, or their incorporation was not clearly explained. 

In some cases, uncertainties were simply ignored or their consideration was deferred to other future analyses or other in-progress biological opinions. For example, Table 9 

Technical Review Panel Document -12/30/2005 2 


    1.. 

in the BO (page 193) summarizes the effects of the proposed project on the listed ESUs, but Table 9 fails to list eleven additional effects mentioned in the text of the BO. Ignoring or deferring the consideration of these effects in analyses does not give the listed species the required benefit of the doubt. 

4) Some models and analyses appeared to be flawed. The application of monthly temperature models to anadromous fish studies is a point of concern. Of particular concern is the adoption, with little discussion, by NMFS of these monthly results both for assessing potential impacts and for setting thermal criteria. In addition, the data used to develop relationships between water temperature and salmon gamete, egg, and alevin mortality was not the best available. 

5) Greater consideration should be given to genetic and spatial diversity in the ESUs. Too little consideration was given to the genetic and spatial diversity aspects of the ESUs. The Central Valley Technical Recovery Team (CVTRT) noted that the "dependent" populations of spring Chinook and steelhead occupy marginally suitable habitats that either depend on migrants from the nearby streams or operate as a metapopulation in which each stream is not individually viable, but the group persists. These dependent populations are a valuable resource because they exist in marginal environments, may contain valuable genetic attributes (e.g., higher temperature tolerance), and may serve as links with other populations in ways that increase the viability and resiliency of the ESUs over long time scales. The BO did not adequately treat the genetic and spatial diversity aspects in their analysis. 

To guide us in our evaluation, NMFS posed seven questions for the review that addressed issues they felt were important. We were advised that we could reformulate these questions. We decided that the salient issues covered by the original seven questions could be covered with four questions. The four questions overlap somewhat among themselves in their coverage of the important issues. The reformulated four questions, and a summary of our responses to them, are listed below: 


    a.. 1) Are the technical data, tools, and analyses used in the BO (e.g., modeling, calculations, qualitative assessments) able to determine impacts to individuals and to populations of listed ESUs? 

NMFS' dependence on using existing off-the-shelf models, especially for quantitative analyses, resulted in less quantitative results and more qualitative-based assessments than is desirable. The Panel appreciates the constant tension between the desire to use a systematic approach versus the uneven availability of information among species and among river systems. However, we are of the opinion that, even with the time constraints and desire to be systematic, there are several areas where the data and analyses used by NMFS could have been improved. 

Technical Review Panel Document -12/30/2005 3 


    a.. 2) Were risks and uncertainties adequately considered and treated in the BO? What risks, uncertainties, and limitations were not addressed? 

Characterizing uncertainty is an important step in assessing risk, and ultimately in understanding the strengths and weaknesses of regulatory and management decisions. Uncertainty can take a number of forms (Francis and Shotton 1997). Uncertainty arises from natural variability; missing, inaccurate, or imprecise data; necessity for simplifying assumptions for analyses and models; and how agencies communicate and interact. Regardless of the form, characterizing uncertainty is an important step in assessing risk, and ultimately in properly understanding the basis of regulatory and management decisions. 

The Panel determined that more explicit treatment of uncertainty would greatly improve the scientific underpinnings of the BO. Some of the uncertainties of concern to the Panel were addressed more fully at the October 12-13, 2005 workshop at the University of California at Davis than in the BO, which suggests that uncertainties may have been considered but not documented in the BO. The Panel concluded that the BO would benefit from better documentation of key uncertainties and a clear description of how they were incorporated into the final decision analysis. Furthermore, we suggest that, when possible, the BO should treat the major uncertainties in a more quantitative manner. 


    1.. 3) In the absence of available information to establish probable responses to impacts (e.g., steelhead losses at the CWP and SVP pumps, spring-run Chinook salmon survival and reproduction above Red Bluff Diversion Dam), were reasonable scenarios developed to identify the likely effects of the proposed projects on the fish species of interest? Were comparisons made to other species with similar impacts? 

At the October 12-13, 2005 workshop, the Panel was given a handout1 that highlighted some of the guidelines used in preparing biological opinions. A key point appeared on page 10 of the handout, under the section heading entitled "Providing the Benefit of the Doubt to Listed Populations." The text stated that the Services should conduct their analyses to avoid making a Type II error (i.e., avoid making the mistake of concluding that there was no effect on listed species or their habitat when in fact there was an effect). In other words, the BO should give the listed species the benefit of the doubt. This guideline of giving the species the benefit of the doubt was used as a criterion when we evaluated the BO, especially when we addressed question (3). 

The BO ignored or understated several factors that are expected to have measurable effects on the listed salmon and steelhead populations. Where there was insufficient information to develop a quantitative estimate of an effect, the BO frequently appeared to ignore the unquantified effect, or to not clearly explain how the unquantified effects were incorporated into the overall integration and synthesis. Potentially important effects were apparently not included in the 

1A package containing pages 8-10 of material copied from another source was given to the Panel by Penny Ruvelas of NOAA Fisheries at the October 12-13 Workshop. The header on the handout was entitled "Background paper on assessment framework for jeopardy analysis." 

Technical Review Panel Document -12/30/2005 4 

synthesis, or there was no explicit explanation as to how they were included in the final analysis. For example, it was unclear to the Panel exactly how the following issues were incorporated in the analyses or final synthesis: the effect of hatcheries, mortality of subyearling Chinook on route to and within the Delta, the effects of global climate change, and the effects of Red Bluff Diversion Dam on adult and juvenile passage. The failure to explicitly explain how known or anticipated effects were incorporated into the synthesis of effects is inconsistent with the guideline to provide "the benefit of the doubt to the listed species." 


    a.. 4) Where information was limiting or unavailable (e.g., the abundance of steelhead), did the BO provide evidence or make reasonable assumptions regarding the probable responses of listed ESUs to proposed project operations? Of particular concern are the potential effects besides direct mortality (e.g., changes in fecundity and growth rates of individuals, genetic diversity, access to specific spawning and rearing areas)? 

The effect of the proposed projects focused on population abundance (numbers of individuals), and effects on life history traits and population structure were essentially ignored or simply noted in the BO. Several potential pathways for the proposed project to affect life history and population structure, which may affect the determination of jeopardy, were noted by the Panel. These pathways include: changes in temperature and water routing affecting juvenile fish growth and survival in the floodplains and the Delta; disproportionate harvest of older age classes affecting population age-structure and total egg production; supplementation of wild stocks with hatchery stocks reducing genetic fitness; and operations of diversion dams affecting the movements of fish into marginal, but evolutionary significant, habitats. Additional quantitative or qualitative analysis of proposed project effects on life history traits and population structure would strengthen the science underlying the BO. 

Technical Review Panel Document -12/30/2005 5 
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www2.dcn.org/pipermail/env-trinity/attachments/20060109/c7149cc1/attachment.html>