Mapping the Role of Models in US and EU Risk Regulation: A Legal Background Paper

Elizabeth Fisher*, Pasky Pascual** and Wendy Wagner***

TAUC Workshop 10-11 October 2006

Washington DC

DRAFT - Please do not cite or circulate beyond the TAUC group without permission.

This is a working paper to be used in conjunction with the TransAtlantic Uncertainty Colloquium, to be held Oct. 10-11 in Washington, DC.  The views expressed in this paper are those of the individual authors and do not necessarily reflect the views and policies of their respective institutions. An employee of the Unites States Environmental Protection Agency (EPA) has prepared portions of this document; those portions have been reviewed in accordance with EPA's peer and administrative review policies and approved for presentation and publication.

Table of Contents

Introduction. 3

1.    Environmental Models. 4

1.1.       Environmental Models – Definition and Sources of Uncertainty. 4

1.2.       Why the regulatory use and legal analysis of models ?. 6

2.    Environmental Models in the Administrative Context 6

2.1.       The Deliberative Problem Solving Paradigm of Environmental Models. 8

2.2.       The Sound Scientific Analysis Paradigm.. 9

2.3.       Assessing the Paradigm.. 10

3.    The United States. 12

3.1.       The Use of Models in U.S. Environmental Law: Background. 12

3.2.       The Oversight of Model Development and Use by EPA.. 16

3.3.       EPA’s Internal Processes. 17

3.3.1.          The Analytic Blueprint 17

3.3.2.          EPA’s Draft Guidance on Environmental Models. 18

3.4.       Stakeholder Oversight of Models. 19

3.4.1.          Review by the Office of Management and Budget 19

3.4.2.          Science Advisory Board Review... 20

3.4.3.          Review by Stakeholders. 20

3.4.4.          Challenges in Court 21

3.5.       Potential Problems with the Use and Oversight of Models at EPA.. 23

4.    The European Union. 25

4.1.       The development of legal frameworks governing the role of models in risk regulation  25

4.2.       The implications of models for the rulemaking process. 30

4.3.       The potential treatment of environmental models by the courts. 31

 Introduction

In this paper we map the role of models in environmental and risk regulation in the United States (US) and the European Union (EU). This exercise in mapping is not a comprehensive description of when and how models are utilised by regulators – models are used so frequently that such an enterprise would be impossible. Rather, we are concerned with mapping understandings about the role of models that are embedded in legal and administrative culture in both jurisdictions. These understandings can largely be understood to fall into one of two opposing caricatures of models. From one perspective (the one we prefer), models are understood as tools for deliberative problem solving. As such, they are not expected to be accurate depictions of reality, although it is hoped that they will be as accurate as possible, but they act as useful analytical reference points for promoting effective discourses about environmental problems among disparate groups. From another, opposing perspective, models are understood as not only essential to sound scientific analysis, but their legitimacy depends on their ability to eliminate uncertainty and provide definitive answers to pressing questions. Although this perspective purports to use models to ensure “objectivity,” in reality its strong form can create expectations that not only cannot be met by models, but can lead to inevitable judgments being obscured from view in order to meet unrealistic demands for certainty. In regulatory circles, these two characterisations, in their strong form, tend to be incommensurable and yet they are often operating simultaneously in regulatory regimes.

This paper is structured as follows. In the first section we discuss some general features of environmental models. In the second section we elaborate on these two opposing caricatures of models in US and EU risk regulation. In the third section we examine environmental models in the US context and show how the sound scientific analysis paradigm has dominated legal discourse. In the fourth section we analyse environmental models in the EU context to show how both paradigms are in operation. In the final section [not yet completed] we provide a critical comment on this state of affairs by arguing that the most legitimate way to understand the role of environmental models in risk regulation is from the deliberative problem solving perspective.

Three points should be made before starting. First, this paper is not a comprehensive analysis of the interface between environmental models, risk regulation and law. Rather, our purpose is to sketch a framework for thinking about those interfaces – interfaces which have largely been ignored by lawyers and policy makers alike. Second, this paper is primarily concerned with the use of environmental models in the administrative setting of risk regulation in which regulatory science is in operation.[1] In that setting our focus is upon law and policy due to the fact that both have an important role in regulating administrative decision-making and thus influencing understandings about the role and nature of environmental models. The final point to make is in relation to the comparative nature of this study. Comparative law cannot simply be an isolated study of the same type of laws in different jurisdictions.[2] Rather regard must be had to the role of legal and socio-political culture. In this paper, we have taken a relatively ‘thick’ understanding of the context we are studying in that we see law and policy as embedded  in social and political cultures.[3] As such, law cannot be treated in a ‘plug and play’ type manner.

1.           Environmental Models

1.1.    Environmental Models – Definition and Sources of Uncertainty

As a matter of first order, we present our definition of the term “environmental model,” as well as our reasons for focusing on model uncertainties. As we will argue, models—as distinct from other sources of scientific and non-scientific information (e.g. databases; results from single experiments or field trials; narrative testimonials from advocacy groups)—present unique opportunities and challenges that merit special attention regarding their role in developing and administering environmental policies.

By model, we refer to a computerized, mathematical representation of a system. A system—whether it be the U.S. economy, the ecology of the Rhine River, or the European Commission’s political system—is a set of phenomena that shares some common aspect(s) in which one is interested.[4] In order to understand, explore, or make predictions about a system, an analyst will often select elements from the system and formalize relationships among these elements through mathematical equations that are codified within a computer program. It is this formalization that we call a “model.” This definition seems to be generally shared across the EU and US.[5]

When we say we are interested in “environmental models,” we signify our interest in the entire panoply of models used for environmental decision-making. These include models of physical systems—such as the dispersal of gas molecules or the leaching of pesticides through soil—for which the algorithmic components may be obtained through experiments within the modeler’s tight control. They also include models of economic and social systems—such as the monetary value a community would be willing to pay for pristine waters—that codify relationships that, because they are based on observations rather than experimentation, might be influenced by confounding factors. Regardless—these models, whether used as tools for scientific investigation or for regulatory decisions, must ultimately contend with ineluctable uncertainty.

Figure 1 below shows why this must be so. As a first step, the modeler must decide which elements of a system he should focus on; uncertainty already pervades this choice, as the modeler exercises some expert judgment in determining whether he has properly identified the components driving the system he wishes to model (see Figure 1.A below). Secondly, the modeler collects data about the system, a process replete with uncertainties about whether he has chosen and properly applied the correct method to gather representative data that characterize how the system operates (see Figure 1.B below). Even when the modeler has properly executed these first two steps, it is often the case that multiple mathematical equations can describe the same data set (see Figure 1.C below). It is therefore the case that uncertainty inevitably permeates the development of environmental models.

The preceding paragraph focused on issues of uncertainty that relate to the development of a model in the abstract. When a model is used for a specific regulatory application—such as to establish discharge limits for nitrogen in an estuary—decision-makers must contend with another element of uncertainty. It is usually the case that the features of the system being modeled varies over time and space (see Figure 2 below). Because a model may have been developed based on an initial set of conditions, the model developer and user must determine whether a model is sufficiently robust to use even when these conditions have changed. This determination, too, is subject to uncertain expert judgment.

1.2.    Why the regulatory use and legal analysis of models ?

Given the uncertainties in environmental models, it is worth asking (1) why models continue to be used in developing and administering environmental policies; and (2) why they warrant the attention of legal research. Answering the latter question, models warrant legal analysis for two primary reasons. First, for reasons we detail in the succeeding paragraph, models help establish that there is a reasonable basis for regulation by pointing to a rationally-based causal relationship. What is understood as a reasonable basis will vary from jurisdiction to jurisdiction and from context to context. Secondly, because uncertainties obscure this causal relationship, models and their results are “malleable,” in the sense that they can be used to argue for competing legal interests. Models are not innately objective scientific constructs that—if done correctly—yield a single, verifiable depiction of reality. Rather, even when developed according to the best principles of science, models may yield multiple versions of reality, each of which may be perfectly valid, each of which may result in an alternative set of legal responsibilities, and therefore, each of which may lay claim to advocates who use administrative and legal procedures to contest or support competing viewpoints.  Models can thus be analogized to sharp kitchen knifes.  They can be invaluable instruments in the kitchen, but when used carelessly or by sinister people, they can become dangerous.

Despite their scientific uncertainties and the consequent malleability with which they might be applied in a legal context, models continue to be used for environmental decisions because, as we previously stated, they facilitate rational, deliberative rule-making. They do so in several ways.

First, models help forge the causal relationship between that which is regulated and that environmental bad to be avoided or environmental good to be achieved. Data alone are simply a cluster of observations—dots on a graph or points on a map. Relationships among the data must be wrested from these observations through the judicious use of hypotheses and quantitative reasoning, i.e. through modeling.

Second, models help to organize and to integrate the results of multiple scientific studies so that the “weight of scientific evidence” may be brought to bear on an environmental issue. Regulations tend not to be based on single studies; rather they are generally based on results culled from multiple sources of scientific information—monitoring data, epidemiology studies, toxicology studies, expert scientific opinion.  These results are typically pooled together through a formalized construct of how these pieces of scientific evidence related to each other, i.e. through modeling.

Third, models facilitate how information from multiple disciplines can be brought to bear on an environmental issue. In order to reach a regulatory decision, agencies will generally need to address not just how a physical system operates, but also whether a proposed regulatory approach makes social and economic sense. To do so, agencies formalize their analyses through integrated assessments that detail how a regulation might result in societal changes, how these changes affect the environment, and what the costs and benefits of these effects will be, i.e. through modeling.

2.           Environmental Models in the Administrative Context

Our primary focus in this paper is upon environmental models as they are used in the setting of environmental standards and in the authorisation of potentially risky activities. This is a regulatory activity that attempts to prevent harm and is, in both the US and EU, an administrative activity in that it is nearly always delegated to a secondary law maker by a primary law maker.[6] With that said, the administrative sphere of the US and EU are very different. In the US there is a well established Federal regime of environmental and risk regulation where significant rule-making powers have been entrusted to large administrative agencies which are regulated by a complex body of administrative law. As Kagan has noted, US legal culture is one of adversarial legalism[7] and thus legal discourse has had a predominant role to play in shaping administrative decision-making and courts significant forums for debating the legitimacy of administrative action.[8] In contrast, the EU is still emerging as a polity and its present structure does not easily accord with existing models of states, particularly due to its intergovernmental aspects.[9] The administrative sphere is an ad hoc assemblage of Community, national and sub national administrative bodies acting in inter connected ways.[10] So far in the EU,[11] litigation has not a played significant role in shaping these legal frameworks and there has also been a greater embracement of non-legal forms of decision-making.[12]

What is apparent however, is that administrative power occupies an uncomfortable and awkward space in each polity.[13] This is not surprising. In the US unelected public administration is at odds with cherished concepts of liberal democratic constitutionalism[14] and in the EU the growth of the administrative realm would seem to threaten the fragile institutional balance between Community institutions, and between Member States and the Community.[15] The history of administrative law in both jurisdictions has been a history of searching for prescriptive paradigms to explain and legitimate the exercise of administrative power.[16] Moreover the policies and laws which regulate how public administration is constituted, limited and held to account will be reflections of those paradigms and accountability mechanisms are arenas for debating what paradigm should dominate.[17] Fisher describes this phenomenon as administrative constitutionalism[18] and while not the focus of this paper is clearly an important aspect of the background.

As seen above the purpose of a model is in establishing a ‘reasonable basis’ for regulation. Looking at the US and EU risk regulation landscape we can understand models contributing to that legal basis in two distinct ways – as a tool for deliberative-problem solving paradigm and as a means of ensuring sound scientific analysis.

2.1.    The Deliberative Problem Solving Paradigm of Environmental Models

The first paradigm of an environmental model that can be identified in risk regulation is a deliberative problem solving paradigm in which the role of an environmental model is to assist in decision-making by being a tool in which information can be integrated across disciplines so as to promote communication, trust, deliberation and co-operation.[19] These models are typically iterative, and are understood to be addressing problems where the nature of that problem, and the information in relation to it, are understood to be dynamic. As representations of systems, models are akin to metaphors and analogies in that they aid understanding and thus problem solving.[20] In other words deliberative problem solving models are not so much  concerned with ‘proving that there is a relationship’ between source and effect but with exploring the nature and contours of this possible relationship in constantly evolving ways.

The deliberative problem solving paradigm dominates thinking among those that work at the coal face of regulatory science. Models for this group are tools, the utility of which depends on how well they help in the problem solving enterprise. Scientific uncertainty in modelling is important to take into account for this group because an understanding of scientific uncertainty is important in both understanding the utility of the model and in understanding the nature of the problem the model is trying to address. Indeed, for many regulatory scientists this is the only acceptable way to understand the role of models in the regulatory context.[21] This is particularly because the environmental problems which are subject to regulation are understood to be particularly complex, uncertain, and socio-politically ambiguous.[22]

From a public administration perspective, the deliberative problem solving paradigm is promoted alongside paradigms of administration which characterise good public administration as being constituted to address complex problems through an ‘analytical-deliberative’ process.[23] Public administration is expected to exercise its discretion flexibly and accountability is ensured through showing that decision-makers have addressed the complexity of problems conscientiously through an iterative mixture of analysis and deliberation. The focus is on addressing complex problems rather than on finding facts. As will be seen, the way in which this paradigm manifests itself is very different in each legal culture but in each case administrative power is being understood as something that is constituted permanently to address problems in an on going manner.

2.2.    The Sound Scientific Analysis Paradigm

The second paradigm of environmental models that can be identified in these two cultures is the sound scientific analysis paradigm. Under this paradigm the role of an environmental model is to be a value neutral description of reality and the quality of the model is dependent upon the accuracy of that description. As this is case, then models are frameworks of pure scientific reasoning which depict reality. As such, the existence of scientific uncertainty points to problems with the model. Such models are thus more concerned with ‘proving that there is a relationship’ between source and effect in factual terms than with exploring the nature and contours of this possible relationship in constantly evolving ways.

From a public administration perspective, the sound scientific analysis paradigm accords with rational-instrumental understandings of public administration.[24] From this perspective, risk regulation consists of two distinct activities – an objective process of risk assessment and a political process of risk regulation with the latter being understood as a form of interest group representation.[25] If the role of public administration is understood to be expertly assessing facts in line with a legislative mandate, then the role of an environmental model in this context is to aid in ensuring the accuracy of that assessment. A good model is one that makes decision-making more precise and the process of accountability is concerning with eliciting proof of that precision. A consequence of this understanding of environmental models is that the reality of an environmental problem must be understood as being capable of being accurately captured by a process of simplified representation. As such environmental problems are understood as both assessable and manageable.  Indeed, under a strong form of this perspective, the absence of definitive findings in a model can be read to suggest that there is no causal relationship or scientific connection that warrants regulatory intervention.

While the sound scientific analysis paradigm fits comfortably within a rational-instrumental perspective of administrative law,[26] it can sometimes, often by coincidence, provide several important regulatory actors with an attractive frame for advancing their positions or carrying out their assignments.  From the standpoint of regulated parties, for example, the sound scientific analysis perspective offers a way to justify a demand of complete proof of cause and effect as a prerequisite to regulation.  The sound science perspective also provides a vehicle for some stakeholders to delay regulation by raising minor critiques and nitpicks, including highlighting large uncertainties, each of which can devastate a model in a legal system that expects scientific perfection.   From an agency’s perspective, while the sound science perspective obviously creates daunting hurdles for agency modellers, the sound science characterization can also help insulate their modelling work from meaningful review by courts or other lay publics because the models are perceived to be solely “scientific” or technocratic.  Moreover, from the agency’s perspective, once a model actually manages to survive administrative process requirements, it is considered “final;” constant revisions and adjustments are neither expected nor tolerated.  Finally, courts may find the review of models to be far easier if they can evaluate them as they do other “facts.”  If models are instead perceived as open-ended devices that enhance deliberation, the criteria by which they must be evaluated are nebulous and difficult to apply in a consistent way. 

2.3.    Assessing the Paradigm

In Table One below we set out what we see as the differences between the two paradigms.

Table One: Two Paradigms of Models

In the next two sections we look at how these understandings of models have dominated legal and policy discourses in the US and the EU. Before doing that however it is worth highlighting a number of points. First, these understandings are prescriptive understandings and as such they influence what is understood as good decision-making. As prescriptions, however, they also have a powerful influence on how administrative practice is described. Second and following on from this, the two paradigms may be operating in relation to the same regime and a particular model may be open to competing interpretations. Thus for the purpose of an agency’s internal review a model may be understood in accordance with the deliberative problem solving paradigm while a judge engaging in judicial review may understand it in terms of the sound scientific analysis paradigm. This state of affairs is a feature of nearly all public administration and reflects the fact that decision-makers are often operating in contexts in which they are subject to multiple accountabilities which are based on different understandings about their role and nature.[28]

3.           The United States

Because most environmental law and regulation in the United States depends on scientific research to inform decision-making, regulators are increasingly relying on quantitative models to support their decisions.  Over the last decade, in fact, it has become an expectation rather than merely a hope that a large body of science will support environmental and health regulation.  The calls for “sound science” from stakeholders and from officials within Congress and the Executive Branch are the prevailing view of what is necessary for viable environmental regulation in the U.S. 

            Yet when this popular “sound science” perspective mixes with the U.S. legal system’s adversarial structure, it can create a very hostile environment for models.  In the U.S., well-accepted climate change models, risk assessments, fate and transport models, and other regulatory-based models have been attacked through formal agency or judicial processes because: they had not been validated with “real” data; used unverified coefficients; or were perceived to be too uncertain to be the basis for imposing costly regulatory obligations, despite an agency’s legislative mandate to err on the side of public health.  By contrast, the view of models as deliberative instruments that assist, but cannot determine regulatory outcomes has been relegated to a minority view in the U.S. -- a view that is now held primarily by modelers, not most politicians, judges, or powerful stakeholders.

This section discusses the United States’ approach to models.  We begin with a background and history of the use of models in the United States.  We then consider the types of oversight that apply to models and conclude with some gaps and problems that remain in the U.S. regulatory system with respect to assimilating models into environmental decisions.

3.1.    The Use of Models in U.S. Environmental Law: Background

Most environmental law and regulation in the United States originates from laws passed in the 1970’s that require the Environmental Protection Agency (EPA) to place regulatory controls on products, like pesticides and chemicals sold in commerce, and on wastes discharged into the air, on land, and into the water.  In these laws, agencies have found science to be an increasingly useful tool to satisfy their sweeping congressional commands. 

Congress appreciated that in providing agencies with broad and ambiguous legislative directions it was ceding a great deal of power to unelected officials.  In order to provide greater accountability for agencies and to ensure that they do not get the facts, including the science, wrong, Congress passed numerous laws and statutory provisions, embodied in administrative law, that impose a variety of different oversight mechanisms on agency decision-making.  For example, persons who object to an agency’s use of the available scientific information can not only submit comments to the administrative record, but they can challenge the agency regulations in court, arguing that the regulations are “arbitrary and capricious” in light of the available science.[29]  Congress and in some cases the agencies themselves have also created prominent science advisory boards to review the quality of the scientific analysis underlying an agency’s regulation.[30]  Highly controversial studies used for regulation have sometimes been subject to expensive reanalysis or verification.[31]  And on more than one occasion Congress has held oversight hearings to investigate potential problems with agency rulemakings, including their use of science in the decision-making processes.[32] 

Although these oversight processes do not contemplate the use of models in particular, models have played an increasingly influential role in the development of environmental and public health regulation.  In environmental law, most statutory directives provide the agency with very broad discretion to determine what regulatory standards or programs will “protect the public with an adequate margin of safety” from air pollutants,[33] “will not cause unreasonable adverse effects” from pesticides,[34] or ensure that all waters in the United States are “fishable and swimmable.”[35]  Although models are rarely explicitly required in these mandates,[36] their use seems inevitable, if not essential to effectuate Congress’ goals.  It is not surprising that agencies like the EPA often make use of the available models or even develop new ones in their development of these regulatory standards. 

The most established use of models in environmental law occurs in EPA’s effort to satisfy its Clean Air Act mandate to ensure that all areas of the United States do not exceed ambient standards for several major air pollutants.[37]  Since the regulatory system demands that all areas in the United States fall below these pollutants levels in order to protect the public health with “an adequate margin of safety,” models quickly became an integral part of the regulatory process.  EPA uses risk assessment models to determine the “safe” level of air pollution for sensitive subgroups in the population, and it deploys sophisticated fate and transport models to predict levels of these pollutants after emissions reductions have been made from various mobile and stationary sources.  Since states generally implement CAA programs in their own states under federal direction, states also have incentives to develop and run very precise models that will both predict and explain how the state will attain ambient standards presently and the future.  If the ambient standards are not met, there are significant economic and related consequences that both EPA and the states are eager to avoid.  Models – indeed very accurate models – have thus become a vital feature of this particular Clean Air Act program.

In other areas of pollution control, Congress initially instructed EPA to regulate by requiring industries to install the “best pollution control technology,” which effectively circumvented the need to model environmental effects.[38]  Once these primitive controls for pollution control were put in place on an industry-by-industry basis, some localized risks to health and the environment remained.  In response, both legislative amendments and regulatory refinements led EPA to depend on quantitative models to address the remaining pollution problems.  These programs include the Total Maximum Daily Load (TMDL) program of the Clean Water Act,[39] the residual risk program of the Clean Air Act (for air toxics),[40] risk assessments for the clean up of Superfund Sites and defunct hazardous waste facilities,[41] and the Maximum Contaminant Level Goal (MCLG) standards of the of the Safe Drinking Water Act.[42]  Models, particularly fate and transport and risk assessment, play an important role in these programs, particularly the TMDL program which requires states or the EPA to trace the contributions of individual point and nonpoint dischargers into surface waters and predict how water quality would improve with reductions of pollutants from individual sources.[43]

EPA also uses quantitative risk assessment models to evaluate the safety of pesticides and toxic chemicals under its licensing regulatory programs.  As scientific knowledge grows in this area, scientists have developed more sophisticated models that can be used for some classes of chemicals.  For example, pharmacokinetic (i.e., PBPK) models are playing an increasingly important role among toxicologists in evaluating risks and are slowly infiltrating the agency.[44]  Because they can be complex, assumption-laden, and provide predictive outputs that cannot be checked against the real world, EPA has resisted using these models, at least to the extent that they suggest levels that are more permissive than current risk adverse levels.[45]  Regulated parties are constantly pressuring EPA to integrate these models into its decision-making.

Models can also be used by the agency in the enforcement context – for example, to prove that a party violated a regulatory obligation (using fate and transport models, for example).[46]  Although the use of these models is often site- and context-specific, it constitutes an important use of models that is likely to grow over time.

Models are also used in less formal ways that nevertheless affect regulatory goals and requirements.  Toxic tort suits, brought by alleged victims under the common law of torts seeking compensation for harms, often depend on models.  Models can also be used by agencies or Congress more informally in assessing problems, setting priorities, or evaluating the efficiency of rules.  In these cases, the models do not create binding or direct obligations for members of the public or regulated parties, but instead provide information for “deliberating” about whether there is a problem in need of redressing, like climate change and acid rain, or whether a proposed solution is the best among alternatives.  Finally, states can use and develop models for their own programs or to fulfill their responsibilities under federal statutes, particularly the Clean Air Act NAAQS and the Clean Water Act TMDL programs. 

It is perhaps not surprising given the wide range of models used by EPA that the agency’s goals for identifying the best model are also quite variable.  The most important source of variation from the perspective of model development and evaluation is the agency’s willingness to reject models that are more certain or accurate in favor of models with greater simplicity or consistency.  In some settings, a natural system may be so complex that a cutting-edge model will provide too much unpredictability or geographical inconsistency to satisfy pressing regulatory needs; instead a simple, albeit imperfect national model may be preferred to support predictable regulatory standards. 

Simple models are sometimes also better able to satisfy EPA’s protective mandates, which instruct the agency to err on the side of public health.  Simple models that lead consistently to false positives, rather than produce a more even distribution of uncertainty, may be more faithful to the EPA’s legislative charge.  For example,  rather than model what levels of PCBs in contaminated land will be protective in any given locale, EPA instead demands that cleanups meet a national cleanup standard for PCBs.[47]  This simple, national standard provides predictability and avoids the costs of collecting data, applying models, and responding to inevitable challenges and disagreements about site-specific requirements.  At the same time, it guarantees minimal protection for public health in a way that a complex model may not.

3.2.    The Oversight of Model Development and Use by EPA

When they inform regulatory decisions, models used for regulation must be subjected to mandatory administrative processes of oversight and review.  These processes not only serve as checkpoints to ensure the reliability of models, but can serve as challenge points for affected parties to contest the models, sometimes in illegitimate ways. 

The extent of review and oversight of models depends on how they are used in regulatory decisions.  If a model is used to support binding regulatory requirements, then the models must be formally reviewed by stakeholders and often by experts.  Generally this full review also involves several levels of internal review by scientists and policymakers within the agency;[48] inter-agency review by the Office of Management and Budget;[49] formal review by an independent panel of scientific experts;[50] a notice and comment process that includes the “general” public and stakeholder input;[51] and often, a challenge brought by an unhappy stakeholder before a panel of federal appellate judges.[52]  It is important to note, however, that each of these oversight processes apply to all binding rulemakings, whether they use models, or not.           

            It may be most useful to view how the US regulatory system views model uncertainty in terms of four interrelated layers of treatment.  (See Figure 3 below.)  At its most substantive level, evaluation of model uncertainty is a complex, scientific endeavor. At this level (A), analysts use quantitative and qualitative approaches to evaluate uncertainties about whether: a model identifies the appropriate system components; the appropriate data were used, the appropriate model was selected; and the model was properly applied. US EPA relies on various mechanisms (B) to conduct this evaluation, some of which are undertaken in order to fulfill certain legal and administrative drivers (C). If, however, external stakeholders are unhappy with a model and its results, they may avail of only a subset of these drivers in order to raise challenges to these models in court (D). Even then, courts will generally base their decisions only on whether US EPA has fulfilled its procedural obligations in establishing scientific panels (FACA) and providing sufficient opportunity for public comment (APA).  Only in the most egregious cases will the courts look to the innermost nucleus of substantive science (A) in order to rule against the use of a model and its results because it does not comply with the APA’s requirement of a rational basis.  Each of these stages are discussed in turn.

3.3.    EPA’s Internal Processes

3.3.1. The Analytic Blueprint

The US EPA engages in many activities—passing regulations, issuing policy statements, developing risk assessments, drafting guidance documents, constructing models—that define the technical and operational details of environmental programs. In deciding whether it should initiate an activity and, if so, how best to implement it, the agency uses a formal, internal process that relies upon the collective judgment of offices across US EPA. To facilitate this process, if the agency deems a proposed activity to be sufficiently significant, it will prepare an internal document—a so-called “Analytic Blueprint”—to synthesize the activity’s rationale, along with any pertinent economic and scientific data and any legal-, policy-, stakeholder-, and compliance-related issues.

The Analytic Blueprint (hereinafter, “blueprint”) serves as a roadmap to facilitate the agency’s deliberations on how best to address an environmental issue—which regulatory and non-regulatory strategies to pursue; what data to collect; which analyses to undertake; how best to implement a strategy given the available time and resources. While a single office within US EPA typically takes the lead in its preparation, the blueprint is drafted by a workgroup composed of representatives from the agency’s various offices. It is then used by senior management across the agency to discuss how to proceed with the proposed activity. If US EPA decides that the best way forward is through a regulatory action, then—in a process described in greater detail in section 3.2.2 below—it will send information on the proposed action to the Office of Management and Budget for review.

US EPA’s internal guidance recommends that the blueprint focus on the following science-related issues: (1) the nature of the environmental and/or public health problem being addressed (including any disproportionate impact on sensitive subpopulations, threatened species, or critical ecosystems); (2) the nature of the anticipated regulated entities; (3) the uncertainties, assumptions, data gaps, and extent of agreement regarding the science; and (4) whether the process generating the science can meet the standards of quality assurance and peer review within the agency.

A review by the US EPA’s Office of Inspector General (OIG) highlights the importance of the scientific analyses (and of models, specifically) that buttress an Analytical Blueprint. (The US Congress established the OIG as an independent, fact-finding institution across the federal government to evaluate and to audit the conduct of governmental business.) In 2003, US EPA’s OIG conducted a study evaluating how science was used in 16 of the agency’s significant rulemakings. It identified 452 science documents that were critical to the outcome of the agency’s deliberations. It opined that if the results of any of these documents had been different, the agency might have altered its assessment of which entities to regulate, of compliance costs, and of the magnitude of risk facing the public and the environment. The OIG concluded, for example, that without the models and data demonstrating the downwind transport of ozone and its precursors, there would have been no scientific basis for the agency’s subsequent regulation imposing emissions limits for nitrogen oxides in 22 eastern states.

3.3.2. EPA’s Draft Guidance on Environmental Models

In May 2002, in a memorandum entitled “Strengthening Science at the Environmental Protection Agency,” the head of EPA enumerated several priority items to “improve the Agency’s policy-making process to better integrate the highest quality science.” Among these priorities was the “revitalization of the CREM (EPA’s Council for Regulatory Environmental Modeling)” in order to develop policy guidance “essential for regulatory environmental model development and application…” Pursuant to this directive, EPA recently issued draft guidance for developing, evaluating and using environmental models that highlight the importance of documenting a model’s history.[53] To evaluate a model, the Agency recommends focusing on four key questions—(1) How have scientific principles been addressed during model development? (2) How is the model supported by data? (3) How closely does it approximate the system? (4) How does it perform the specified task?—through various best practices (e.g., peer review; quantitative corroboration and uncertainty analysis, data quality assessment). The guidance reflects the view that ultimately, the fitness of a model’s use for a particular purpose has less to do with eliminating all uncertainties, than with the transparency with which policy-makers and other stakeholders reach decisions based on uncertain information.

3.4.    Stakeholder Oversight of Models

3.4.1. Review by the Office of Management and Budget

Under the US constitution, presidents may “require the opinion, in writing, of the principal officer” of agencies on matters relating to their duties. Since the Nixon administration, presidents have used this authority to require agencies to assess the impacts of their proposed regulations for submission to the Office of Management and Budget (OMB), an office under control of the White House.[54] Currently, Executive Order (EO) 12,866—introduced by the Clinton administration in 1992—outlines the regulatory procedures and policies that agencies must follow when developing regulations and delegates presidential authority to the OMB to coordinate agency rulemaking with the regulatory priorities of the president. Under G.W. Bush’s administration, OMB has supplemented EO 12,866 with various guidance documents and bulletins.

Under the EO, agencies must provide OMB with the text of proposed regulations that are deemed to be “significant,” along with a statement of the need for regulation, and “an assessment of the costs and benefits of the regulatory action,” a so-called Regulatory Impact Assessment (RIA). A proposed regulatory action is considered significant if, among other things, it likely will have a material impact on the US economy or will raise novel policy or legal issues. OMB, rather than the agency, ultimately determines whether a rule is significant.

The EO’s text prescribes that OMB and regulatory agencies make an effort to enhance transparency and public participation in the rulemaking process. Agencies should make public all assessments created for OMB review and should clearly and completely identify any changes in the regulatory action that were made at OMB’s behest. In turn, OMB should maintain a public log of any communications with external parties regarding a regulatory action under review. The EO does not, however, provide for public access to communications between OMB and the agencies.

It bears highlighting that, while EO 12,866 imposes procedural requirements on agency officials—such as sending proposed regulations to OMB for review—most statutes vest legal responsibility for a rule with the agency, rather than the president. While a US president may not be authorized by statute or by constitutional provision to substitute her judgment for that of an agency official regarding an agency action, she does have the authority to appoint—and to terminate the service of—upper-level agency officials.

Moreover, an EO does not serve as the basis for judicial review. Issues over how EO 12,866 is implemented—whether they pertain to the EO’s prescriptions for public participation or to substantive disagreements over OMB’s or an agency’s analyses—are, as one commentator puts it, “left to the political process.”[55]

Over the past five years, OMB has been issuing guidance directing agencies to document the quality of scientific information used and to more formally address the uncertainties in analytical models that underlie an RIA. OMB’s guidance and bulletins on peer review, on regulatory analyses, and on probabilistic risk assessment—all assert the need for agencies to document the process through which agencies reviewed the best available science and to formally analyze scientific uncertainties through techniques such as Monte Carlo simulations, expert elicitation, sensitivity analyses, and value-of-information methods.

3.4.2. Science Advisory Board Review

Oversight by a panel of “independent” experts often constitutes the most vigorous form of regular oversight of models.  In many environmental programs this oversight is either mandatory (for the CAA ambient air quality standards and the regulation of pesticides) or has become so routine that it is expected.[56]  Since a model is often part of the larger scientific record, however, the vigorousness of this scientific review will depend on the model’s significance to a regulation and the panel members’ interest in reviewing the model.  Some panel members may actually try to run a model and understand each of its assumptions; other panel members may review only secondary critiques of the model and take its reliability and its relationship to alternative models for granted.  Moreover, in most cases the agency is not bound by the scientific panel’s critiques; instead the agency may elect to offer a response to the expert report without altering its analysis.[57] 

3.4.3. Review by Stakeholders

Administrative law also requires that the rulemaking and all supporting technical information, including any and all models, be subject to notice and comment by affected parties.[58]  The parties generally are entitled to access of the model components and the input data, as along as they are not trade secret protected or protected as “critical infrastructure” information.  Stakeholders and the general public can submit comments, including detailed technical comments, during the public comment period.  The agency is required to review and respond to the comments, although it is generally not required to provide a response for each comment individually.  When models are used to support decisions that are not rulemakings but instead inform agency priorities, enforcement cases, and the like, the agency is not required to solicit public input.

The opportunities for affected parties to provide formal comments on models used by agencies have grown in recent years.  The “Data Quality Act” – passed as an appropriations rider in 2000 – allows any person to file a petition with the agency for the “correction” of information (including models) that the person believes is not reliable, objective, or is otherwise flawed.[59]  Several models have been the subject of challenge, including a large climate change model that was reviewed with approval by the National Academy of Sciences.[60]  In that case, the complainant argued that the model was so flawed that the agencies should “cease [its] dissemination” and remove it from all federal databases and web sites, a petition that the agencies ultimately denied.[61]  While in theory this additional review by affected parties could provide an agency with valuable feedback on its models, environmental groups have argued that the complaint process runs the risk of being dominated by affected parties who launch illegitimate challenges against models intended not to improve models, but only to delay their use.[62] 

3.4.4.        Challenges in Court

Finally, an affected party has the option of challenging a rulemaking in court based on a number of grounds, including arguing that the agency’s model is unreliable or is otherwise “arbitrary or capricious.”[63]  There have been a number of legal challenges to the EPA’s use of models that allege technical errors, problematic assumptions, process violations, or problems arising from the selection of one model over alternatives.  See Figure 4 below.  Most challenges to agency models, however, involve challenges to the reliability or accuracy of the model.[64]  Since the review of these challenges can become quite technical, courts are occasionally criticized for their misunderstanding of the nature of models and some judges have made unrealistic demands on the agency’s modeling efforts.  One of the most infamous examples is the court’s insistence in Ohio v. EPA that EPA needed to actually compare its computerized atmospheric model, CRSTER, against real data collected from the particular locale before it could legally employ the model to determine emissions limits.[65]  The court observed that with respect to modeling, the legislative history of the Clean Air Act instructed the courts to “conduct a ‘searching review’” of the agency’s model[66] and concluded that EPA had not adequately demonstrated that the CRSTER model took into account the “specific meteorological and geographic problems” of the plants.  It was therefore arbitrary and capricious for EPA to allow a 400% increase in emissions "without evaluation, validation, or empirical testing of the model at the site.”[67]  

3.5.     Potential Problems with the Use and Oversight of Models at EPA

One of the primary objectives of administrative oversight processes is to provide all affected stakeholders with an opportunity to contribute to the development and use of information used for regulation, including models.  Yet this process also allows stakeholders to mount challenges to models that are not neutral, but instead attack a model simply because it yields an unwelcome outcome or is based on policy assumptions with which they disagree.  The oversight of models in the U.S. can thus be quite adversarial and not always scientifically useful.  In Appalachian Power v. EPA (I), for example, the plaintiff electric utilities raised a number of complaints about EPA’s model, but their concerns were so poorly explained and unsupported that the court rejected their arguments on their face and held that they did not present a credible challenge.[68]  Model review under these situations also runs the risk of being dominated by those with resources and a great deal to lose if the model is accepted. Whether the agency can counteract these imbalances through its own internal review is a question that has not been resolved by administrative law scholars.

Even more worrisome is the possibility that this adversarial legal system serves perversely to reinforce the most damaging features of the sound scientific perspective.  If every stakeholder is provided with opportunities to comment on models and sues the agency in court if the models are considered flawed in any way, then the portrayal of models as “definitive science” provides ample room for mischief.  As discussed earlier, stakeholders can (and sometimes do) endeavour to grind agency rulemaking to a halt by arguing either that a protective model is insufficiently supported by science or that the science that was used is not perfect.  In the first case, the stakeholders argue that all regulation must be justified by “sound science”.  Thus even if Congress demanded that the agency err on the side of protecting public health, these stakeholders argue that a considerable body of definitive information is needed to support the agency’s regulation before it can be effective: A simple model that is based on contestable coefficients may not meet these demands for “sound science.”  In the second case, stakeholders will point out the ways that a model fails to represent reality, has not been adequately “verified”, involves tenuous assumptions, or is otherwise flawed, in arguing for its exclusion as a basis for regulation.  If the courts or policy officials presiding over these challenges also view models from the “sound science” frame, then model use will be placed in an untenable position for purposes of regulation.

In addition to the potential for lopsided and counterproductive review that results from special interest challenges mounted under the “sound science” perspective, there are other aspects of model oversight that also appear incomplete or imperfect in current regulatory processes in the U.S.  First, once a model survives the laborious process of administrative review, agencies face few if any incentives or opportunities to update the model as knowledge and other feedback accumulate.  An outdated but “final” model supporting a rule is generally immune from challenge, but as soon as the agency revises the model supporting that rule, then the multiple layers of regulatory oversight often must begin anew.  Administrative law more generally has been criticized for its inability to embrace adaptive management.[69]  Models provide a particularly good illustration of this problem since they benefit from constant adjustment and evaluation, yet the agency incentives are structured in a way that discourage this scientific vigilance. 

Second, U.S. regulators currently do not require any information about the provenance of a model, including serious conflicts of interest by those developing the model.[70]  Instead, a model created by a regulated party is presumptively as credible as a model produced by independent researchers who are not influenced by affected parties or regulators.  This inattention to provenance flies in the face of many empirical studies that suggest that conflicts of interest matter to the outcome of a research,[71] particularly for work that synthesizes and reviews existing studies – like models often do.[72] 

Third, models appear often to be used by agencies on an ad hoc basis: If they fill a regulatory need and come to the regulator’s attention, they may be included in the regulatory decision-making.  If this criticism is accurate, than EPA’s unsystematic approach to identifying and using models for regulation is problematic insofar as it does not require the agency to at least consider and explain: a) whether a model is more valuable than alternative inputs – say for example simply collecting data from the field; b) how a preferred model compares with alternative models; c) the regulatory goals for the model (i.e., precision, simplicity, etc.); and d) how the model will be updated and evaluated over time, including plans for the collection of data against which to compare model outputs.  Without a coherent plan for introducing models into regulation, most of these questions will be neither asked nor answered by the agency.  And those reviewing the models – expert scientists, stakeholders, and even the courts – may erroneously assume that the agency made these decisions in an informed and thoughtful way. 

Fourth, given the prevalence of trade secret protections in U.S. environmental law, or at least the generosity of the claim that allows regulated parties to assert the privilege without substantiating it,[73] some models and some of the input data for those models are not be available to other stakeholders.  Indeed, even agency expert panels may face impediments to obtaining information sufficient to evaluate these models.  Currently, however, there is little information available to assess whether trade secret claims actually do impair review of the reliability and application of models.

Finally, since regulatory procedures remain largely agnostic to models, it is not surprising that the federal infrastructure may pay insufficient attention to the much needed nurturing of model development.  In many settings models may provide an outstanding way to fill a regulatory gap or advance a regulatory program, yet agency staff may lack modeling capabilities or lack financing to use or even discover the ways that models could advance their goals.  There are no provisions in U.S. environmental law or regulation that require agency staff to do a comprehensive search of existing models or to assess how those models could be used to advance regulation.  CREM may help redress this problem, but it is not clear that CREM’s resources and authority allow it to play a significant role in all agency programs.

4.           The European Union

In the EU the explicit utilisation of environmental models is only just starting to occur and over the last decade environmental models have gone from playing a largely backroom role as a tool for decision-makers to playing a more explicit role as a focus for decision-making. In some contexts, models are being understood as tools for deliberative problem solving abut in other contexts models are primarily being utilised because they are understood as forms of sound scientific analysis.  This state of affairs is not surprising and reflects the multifaceted and often contradictory nature of the European integration process.

Before examining the role of models in the legal life cycle of EU regulation it is useful to explain why there are diverse and contradictory drivers that shaped EU risk regulation. That is done in the next section and the three sections after that consider

a)      The development of legal frameworks governing the role of models in risk regulation

b)      The implications of models for the rulemaking process

c)      The potential treatment of environmental models by the European Court of Justice (ECJ) and the Court of First Instance (CFI).

It should be kept in mind in that there is currently no legal discussion about models. The analysis in this section thus relies heavily on extrapolations from current discourses about risk regulation. 

4.1.    The development of legal frameworks governing the role of models in risk regulation

EU risk regulation is a product of a number of different forces and as such defies simple explanation. On the one hand, it can be seen as a neo-functional spill-over from the common market project and as simply a means of creating common rules to ensure a level playing field. On the other hand, however EU risk regulation was the product of its own politics and reflects the fact that the European integration process was about creating a polity.[74] Whatever the case, the development of this body of law has led to the creation a network of Community, national and sub-national administrative institutions acting in interconnected ways.[75] What this means is that EU integration cannot simply be understood as a process of harmonisation in which common rules are created.[76] Rather it is better understood as a process in which a new polity, a new sphere for debate and action, is being created. Needless to say that sphere bears little resemblance to any national system, albeit the legitimacy concerns are very similar. In particular, despite academic propositions to the contrary,[77] there is no obvious administrative/regulatory state in the EU although the EU for its operation relies heavily on decisions made by non-elected officials. In part this is due to the intergovernmental nature of the EU and in part due to it lacking a doctrine of separation of powers.[78]  EU risk regulation is thus a rich and varied landscape.

The first wave of EU risk legislation was in the 1970s, mainly on a command and control model although authorisation regimes placed authorisation powers in the hands of Member States with co-ordination and updating through intergovernmental scientific committees.[79] There was little in the way of Community administrative/scientific capacity, few frameworks regulating the regulatory process, and little in the way of litigation. This state of affairs began to change in the early 1990s. In particular, new more flexible regulatory techniques were introduced,[80] policies and laws integration environmental issues with other issues were developed,[81] and greater emphasis was placed on issues of legitimacy and transparency.[82] A consequence of these developments was that models were now playing a more public role both as a tool for deliberative problem-solving and as a form of sound scientific analysis.

Understanding models as a tool for deliberative problem solving was a product of a number of different drivers including failures in command and control regulation:[83] the promotion of the ideas of sustainable development and ecological modernisation leading to new approaches to environmental regulation;[84] a political desire to address large scale and open ended environmental problems; an awareness of the need to take action under scientific uncertainty;[85] and the embracement of general concepts of multilevel governance.[86] From this perspective, models enabled flexible environmental governance involving a range of stakeholders in ongoing discourses at numerous different levels of government. In many cases models were a part of ‘softer’ forms of regulation.[87]

From this perspective, models are used as foci for deliberative problem solving in situations where there is an awareness that there is a close interaction between different scientific and non-scientific variables. Thus modelling tends not just to be of biological processes but also economic and social processes. This type of use of environmental models has grown out of the development of more flexible and integrated forms of environmental governance and a key feature of it is involvement of numerous actors in the process of consultation and development.[88] Moreover, these type of models can be seen as associated with the calls from the 1990s onwards, in the wake of the Fifth Environment Action Programme and the policy of sustainable development, for the reform of mainstream economic and development models to take into account environmental issues.[89]

Perhaps the most high profile example of where environmental models are playing an increased administrative role is in regards to the Water Policy Framework Directive 2000/60/EC. That ambitious directive will ultimately replace a rigid command and control model with a flexible regime aimed at imporiving water quality.[90] Jointly legislated by the European Council and Parliament in 2000, the WFD dictates that all Union waters must achieve site-specific environmental objectives by 2015. The WFD’s prescriptive details are daunting, but are readily summarized. Under the WFD, member states must: (1) identify the bodies of water that comprise a river basin district and the pressures upon them; (2) establish environmental objectives for each water body; (3) monitor the environmental status of each water body; (4) establish a plan and implement measures to achieve and maintain the environmental objectives; and (5) review and update these plans as needed. The Directive explicitly allows ‘compenent authorities’ to use models[91] and pursuant to Article 16 there has been the development of the ‘combined monitoring-based and modelling-based priority setting’ (COMMPS) system to establish a European wide approach to developing a list of priority substances.[92] A key aspect of that system is getting Member States to provide data.[93] The use of models is in a context where there is also a reliance on risk assessment.[94] The development of the COMMPs system can be seen to be a product of the need for transparency, expert judgment and to address problems of scientific uncertainty.[95] It is also a regime that not only recognises that the EU system is diverse but that it is dynamic and models need to take into account the process of accession.[96]

The intricacies of using models in an EU-wide regulatory regime are exemplified in the WFD’s treatment of water quality models. Under the WFD, surface waters—i.e. rivers, lakes, and coastal waters—must achieve “good ecological status.” In defining this term, the EC recognized the need to balance (1) flexible, site-specific environmental objectives that account for the unique biological features of individual waters; against (2) the political requirement for transnational, harmonized objectives, without which state decision-makers might compromise environmental interests to pursue short-term economic gains. To achieve this balance, the WFD prescribes the use of a water quality index that ultimately depends on any number of modeling uncertainties—human error in measuring and monitoring environmental events; the inherent temporal and spatial variation of aquatic systems; analytical misconceptions about environmental interactions.

Acknowledging these uncertainties, the European Commission recommends that they be treated formally and transparently. [97] Consistent with the WFD’s requirement that states engage all interested stakeholders in implementing the directive,[98] the Commission recommends that states explicitly communicate the uncertainties associated with implementation:

While …diversity is important, it can also create problems for determining what is accepted as “evidence and proof”.…there will be many occasions where no scientific information is available or where considerable uncertainty exists either about the resource base of the consequences or of intended courses of action. There is no easy answer how to proceed under these conditions. However…debates over uncertainty can be aired and decisions taken with this is mind.[99]             

As such, the use of models thus become focus points for deliberating about scientific understandings of water quality and are means of creating a Transdisciplinary discourse.

Another example of the development of an environmental model is the TREMOVE model which developed out of the Auto Oil II programme.[100] The Auto Oil II programme was an example of the ‘new’ approach to environmental governance which involved all major stakeholders in a soft legal framework. Within that programme a model was developed to assess the different impacts of transport and environmental policy on emissions.[101] Since that programme the TREMOVE model has evolved further and is constantly in the process of being updated, particularly because there is an appreciation that the utility of the model depends on the assumptions the models are based on.[102] Similar type, deliberative problem solving models can be seen in other contexts, particularly those where there is a focus on integrated assessment.[103] In many cases these models are not been developed within administration but by researchers operating in the context of research programmes.

This however, is not the only way in which models have been characterised. In other contexts, they have also been understood as contributing to sound scientific analysis and as such the sole purpose of such models is to provide accuracy. his paradigm can be seen as of a product of: a belief that scientific objectivity guarantees accountability;[104] perceptions about what the WTO SPS Agreement requires;[105] general ‘better regulation reforms’;[106] and responses to controversies such as that over BSE. This paradigm has tended to be result in the creation of institutions whose sole task is to provide information. Moreover, an understanding of models in this way has been implicit in the promotion of risk assessment[107] and impact assessment.[108] At its most simple there has been greater reference to environmental models, and in particular what is an appropriate environmental model.[109] In many cases directives authorised the use of models in assessment,[110] but increasingly directives have also come to regulate how models are used[111] including requirements in relation to uncertainty.[112] Likewise, there are increasingly requirements that applicants[113] or Member States must explain the models they use in applications. In all cases, these references are in contexts where decision-makers are expected to be able to proved the accuracy of their decisions.

Most of the developments about are very new and there has been little in the way of legal discourse, litigation or policy discussion. However, it is useful to reflect on whether a deliberative problem solving understanding of models or a sound scientific analysis of models is likely to have a greater weight in the operation of EU risk regulation. We brief consider the rule-making process and then the role of case law. In both cases, are conclusions are highly tentative.

4.2.    The implications of models for the rulemaking process

Compared to the US there has been historically no defined ‘rulemaking process’ or generic framework that regulates it. [114]  While such frameworks may be starting to emerge they are doing so in a bifurcated way.

Thus in terms of the workings of Community institutions, particularly the role of committees, there has been an increasing realisation there is a need for greater accountability, particularly in how expertise operates in the system.[115] While there is still no overarching legal frameworks governing this area it can be seen that there is an growing number of documents and institutional arrangements which, influence how models are conceptualised and utilised by decision-makers. In most cases these would seem to promote a sound scientific analysis of models. Such documents include: the precautionary principle communication and its requirements for risk assessment; regulatory impact assessment;[116] reforms to scientific committees;[117] harmonisation of risk assessment procedures;[118] and reforms to the comitology process.[119] In nearly all cases, there is an expectation that greater objectivity will deliver great accountability and the use of models is part of the pursuit of objectivity. With that said, there are also examples of where a more reflective and reflexive approach is being taken to accountability particularly among those who focus on the precautionary principle[120] and those thinking about  the role of scientific advice.[121]

The other and smaller arena in which regulation is being developed is in the context of new forms of governance. These are typically outside traditional legal processes and in many cases, are form of deliberative problem solving. The context of the TREMOVE model is an example of this form of regulatory development process. An interesting question is the relationship between these forms of regulatory decision-making and the frameworks for rulemaking processes discussed above.

4.3.    The potential treatment of environmental models by the courts

Basically speaking, there has been little case law concerning the review of environmental models by the ECJ and CFI. The one case there has been, has understood the role of models as a tool for deliberative problem solving. In Commission v. France,[122] France argued that the model on which the European Commission based their argument that the Seine-Normandy basin was subject to eutrophication was flawed.[123] The Court’s response was to cite the precautionary principle and to stress that in light of emerging technical and scientific knowledge the ‘degree of probability’ was enough. The court did engage in a review of the Commission’s reasoning although found it satisfactory. In this case the Court was understanding the model as a tool for deliberative problem solving and this approach can be understood as consistent with some other cases in which the Court were dealing with issues of scientific uncertainty and science in challenges to Community decision-making.[124]

The COMMPs procedure under the Water Policy Framework directive has been subject to an Article 230 challenge in which it was argued it did not comply with either Article 16 or the procedures under Directive 91/414/EEC.[125] The challenge, however, was ruled inadmissible.[126]

Despite the lack of case law on the issue, it is useful to reflect on how European courts may understand models if legal issues arise in relation to them. The deliberative problem solving approach in Commission v. France should not be taken as indicative of how courts will always understand models. This is because there are at least six different legal contexts can be identified and it is useful to highlight how models may possibly be treated in each context.[127] The analysis below is focused around two questions – how intense is judicial scrutiny and how courts treat issues of scientific uncertainty? It should be made clear that the cases below do not relate to models but give a feeling for how courts might approach the review of models if the issue arose. What is clear is that the courts are using both understandings of deliberative problem-solving and sound scientific analysis as their starting points in a range of different contexts.

a) Challenges where Community institutions use models in carrying out their international obligations.[128] The most obvious example here is where models are being used in exercising rights of derogation under the WTO SPS Agreement[129] and the relevant legal arena is that of WTO dispute settlement. The nature of WTO dispute settlement is somewhat ambiguous,[130] as are standards of review,[131] but it is clear that Dispute Settlement Panels have increasingly seen their role as establishing the evidentiary basis for decisions.[132] This has resulted in intensive review and a lack of appreciation of the problems that scientific uncertainty creates for assessment.[133]

b) Challenges where Community institutions use models in the exercise of their power pursuant to a Community regulatory regime. These challenges are usually before the CFI or ECJ under Art 230 TEC and both courts have shown a willingness to engage in reasonable intensive judicial review, particularly under the ‘manifest error of assessment’ and proportionality grounds of review.[134] Historically, the former was understood as an error in reasoning rather than accuracy and the latter was not concerned with a form of cost/benefit analysis. In particular, courts recognised the problems of scientific uncertainty and the need to focus on assessing the qualitative reasoning of decisions and the norms underpinning a decision.[135] In recent years however, particularly due to reliance on the Precautionary Principle Communication,[136] there has been a demand by the Courts for decision-makers to base their decision on a quantitative risk assessment.[137] This is because the Communication is understood to require an assessment of risk before the principle can be applied.[138]

c)  Challenges to the use of models by Member States when operating pursuant to Community regulatory regimes. Models may be utilised by Member State authorities in applying a directive and this could give rise to legal challenges concerning whether such utilisation was valid in both national and European courts.[139] The former is very much a matter of national legal culture. In relation to the latter, the ECJ where Member States are pursuing Community norms, have recognised the legitimacy of precautionary action.[140]

d) Challenges to the use of models by Member States where there is a Community regulatory regime but a Member State wishes to rely on a model in derogating from the obligations of that regime. This may involve cases where the Member State are acting pursuant to specific provisions of a directive or regulation^[141] or provisions of the Treaty such as Articles 95(4) and 176.^[142] The ECJ in such circumstances has engaged in relatively stringent review,[143] mainly to ensure that Member States do not subvert the co-operative nature of risk regulation regimes.[144]

e) Challenges where there is no Community regulatory regime but Member States use models in action that prima facie infringes other Community obligations. The most obvious example here is where Member State action prima facie infringes Article 28 which requires Member States not to place quantitative restrictions or measures having equivalent effect on goods..^[145] The legal question which arises is whether a Member State is justified in taking such action under either Article 30 TEC or the Cassis mandatory requirements doctrine.^[146] In such cases a Member State must establish that their prohibitions of restrictions shall not ‘constitute a means of arbitrary discrimination or a disguised restriction on trade between Member States’.^[147] Broadly speaking this requires a Member State to establish that the measure is necessary, appropriate and proportional. Historically, the ECJ in establishing this was not concerned with quantitative proof or the adherence of the Member State to a methodology but rather focused on the general quality of the Member State’s reasoning and how it accorded with norms generally recognised throughout the community.[148] As such there was a sophisticated approach to scientific uncertainty. In more recent case law however, the court has demanded that Member States base their decisions on a risk assessment, particularly if they wish to apply the precautionary principle.[149] This again can be seen as a product of the Commission’s Communication on the Precautionary Principle.

f) Challenges to the use of models by Member States in matters with no relationship to EU law. This category is not directly relevant to our analysis here but is useful to highlight because how national courts address these challenges will affect how they address challenges in category c). English courts in such circumstances have tended to be highly deferential.[150]

The point of highlighting these different litigation contexts is to show how the question of the legal treatment of models in the EU is by no means a straightforward one. There are many different forums and legal contexts for litigation which will result in models being treated in many different ways.