Current Projects Read or Print all Projects

Quality Assurance of Models. Validation of the models we use for managing the environment has never not been a problem. There is today as much interest in the subject as there was three decades ago — more so, in fact. Federal government scrutiny has been bearing down in recent years on assuring the quality of the data on which the EPA bases its regulatory decision making. This dogged pursuit of quality assurance (QA) now applies to data used in, and generated by, models. It has led in no small part to the creation of EPA's Council on Regulatory Modeling (CREM) in 2000. In turn, one of CREM's first official actions was to issue a draft Guidance Document (2003/4) — on matters of model evaluation (validation) — for peer review through the Agency's Science Advisory Board (SAB) and as the platform for a National Academies review of the use of models in regulatory decision-making. Validation is thus clearly not merely a theoretical, or epistemological, problem — of the philosophical basis for the corroboration or refutation of scientific hypotheses. It is also of immense practical significance. Disputes and litigation over controversial decisions and regulations founded on predictions from models will eventually turn on just how valid are these models. But why should validation (model QA) be perceived now as such an acutely difficult challenge? It is in part because of the increasingly common requirement for grand extrapolation into conditions not previously encountered, wherein the conventional assessments based on the "matching of history" and peer review are proving not to be wholly adequate. It is also in the nature of modeling environmental systems to advance towards very high order models (VHOMs), which are especially difficult to evaluate, according to both conventions (of peer review and matching history). We are therefore engaged in developing a protocol for assuring the quality of "predictive" models, such as those used, for instance, to predict the environmental impacts of entirely novel chemical substances. Within this setting we believe there is scope for developing new forms of quantitative testing procedures. The key is to conceive of models not as truth-generating machines, but as tools designed to fulfil specified tasks (like hammers, screw-drivers, and so on). The key will also be to accommodate "extension of the peer-community for quality assurance" within the protocol, which is only to be expected under a regime of Post Normal Science, socially robust science, or Sustainability Science, if these come to prevail.