Supercomputer for Model Uncertainty and Sensitivity Evaluation (SuperMUSE)
EPA's Supercomputer for Model Uncertainty and Sensitivity Evaluation (SuperMUSE) is a key to enhancing quality assurance in environmental models and applications. Uncertainty analysis (UA) and sensitivity analysis (SA) remain critical, though often overlooked steps in the development and evaluation of computer models. While there is a high potential for exposure of humans and ecosystems to chemicals released into the environment, the degree to which this potential is realized is often uncertain.
- Uncertainty Analysis (UA) in model predictions is called for when there is a lack of knowledge (e.g., how well does the model itself reflect nature) and inaccuracies in model input data (e.g., quantity and quality of data) used to drive model simulations.
- Sensitivity Analysis (SA) can lead to a better understanding of how models respond to variation in their inputs, which in turn can be used to better focus laboratory and field-based data collection efforts on processes and parameters that contribute most to resolving uncertainty in model outputs.
As a result of the SuperMUSE hardware and software technology developed by EPA, its customers can now better investigate new and existing UA/SA methods. EPA can also more easily achieve UA/SA of complex, Windows-based environmental models, conducting analyses that have to date been impractical to consider.
The resolution of a model's UA/SA is critical because risk-based assessments based on model performance can result in societal and monetary impacts, locally and/or globally. UA/SA research for high-order, PC-based models cannot be accomplished without parallel computing (running model simulations simultaneously instead of sequentially). A fundamental characteristic of UA/SA is the need for high levels of computational capacity to perform many, many model simulations.
Models are becoming more complex as they begin to address multiple disciplines (e.g., human heath and ecological risk assessment). Models also increasingly include multi-pollutant, multi-scale, and multimedia components. For UA/SA, the runtime of model simulations can increase geometrically with the complexity of the model and the types of UA/SA being considered. Though CPUs have improved over time, the trend in model complexity has offset gains in CPU speed. The power of parallel computing is still needed to overcome the UA/SA model runtime problem.
Windows/Linux Based Approach
SuperMUSE is currently constructed from a collection of network computing switches and 186 personal computers (PCs), which support both Linux and Windows 98/2K/XP operating systems, with an equivalent CPU speed capacity of roughly 270 GHz. An expansion to 400 PCs is underway that is expected to bring the total equivalent computing capacity up to 700 to 1000 GHz. Dual-boot Windows/Linux capabilities also are currently being further developed and refined. [More Information about how SuperMUSE works.]
Historically, the Agency has relied heavily upon Unix- and Linux-based operating systems to achieve high performance computing capabilities through "mainframe" computers and Linux PC clusters. In juxtaposition, most Agency model developers and users are familiar with and reliant upon Windows-based operating systems. While the need for UA/SA to evaluate environmental models continues to grow, investigation of Windows-based models has been historically limited by a lack of Windows-based supercomputing capacity.
SuperMUSE helps fill this gap by providing parallel computing capabilities in both Windows and Linux environments. SuperMUSE is comprised of both a large cluster to facilitate EPA's ongoing research into various UA/SA methods and models, and a supporting software system that can be transferred to other clusters/grids. The SuperMUSE software can be applied in any PC network environment, and is generally extendable to performing UA/SA on any PC-based model.
Beneficial Impacts of PC-Based SuperMUSE
- Scalable to individual user (or Program, Regional, and State Office) needs
- Clustering from 2 to 2000+ PCs.
- Supports Windows- and Linux-based modeling systems.
- Can handle PC models with 10's to 1000's of variables.
- Solves "embarrassingly parallel" computing problems (i.e., running a model over and over again with slightly different inputs, where the number of model runs needed is much greater than the number of PCs available).
- Autonomy from supercomputing centers; removes barriers.
- Simple and inexpensive; it can be built/operated by PC novices.
- Ideal for debugging models (i.e., verification) and performing UA/SA.
- With an average model runtime of 2 minutes, SuperMUSE can currently run over 4 million simulations/month.
The 3MRA Version 1.0 Example
The new Multimedia, Multi-pathway, Multi-receptor Exposure and Risk Assessment (3MRA) technology provides the ability to conduct screening-level risk-based assessment of potential human and ecological health impacts resulting from long term (chronic) exposure to hazardous chemicals released from land-based waste management units (WMUs). The 3MRA Version 1.0 "integrated modeling system" consists of a set of 17 science model components with various post-processing capabilities. 3MRA resides within an underlying modeling system infrastructure called FRAMES. The FRAMES (Framework for Risk Analysis in Multimedia Environmental Systems) modeling infrastructure is envisioned as the foundation for eventually integrating other regulatory support decision tool needs anticipated in the future.
3MRA has undergone extensive quality assurance testing, over 40+ module-level peer-reviews, and initial uncertainty and sensitivity analyses achieved via SuperMUSE (including over 60 million model runs conducted to date). The material below gives more detailed information about a final systems-level peer-review of 3MRA and the findings of EPA's Science Advisory Board.
The 2003 SAB Review Materials for 3MRA Modeling System includes extensive material about the 3MRA Modeling System and the role SuperMUSE played
The SAB Report includes the reported findings of the SAB Panel's extensive peer-review of 3MRA and SuperMUSE, along with various supporting materials. An excerpt is provided here:
"The significant efforts that the Agency expended to develop hardware and supporting software tools for the SuperMUSE Windows-based parallel computing framework have greatly facilitated the verification process, not to mention the capability to conduct sensitivity and uncertainty analyses heretofore impractical with a model of this complexity. Also, the fact that the SuperMUSE [software] system is scalable to any number of networked computers allows stakeholders with a range of available resources to conduct national scale analyses with 3MRA. The panel recognizes the significant achievement that this [SuperMUSE] represents, and expresses its support for maintaining this resource."
SuperMUSE Related Highlights
- SuperMUSE Parallel Computing Hardware (PDF) (19 pp., 1.4 MB, about PDF) - (Chapter 5 of Volume IV: Evaluating Uncertainty and Sensitivity). Efforts to address computational needs for 3MRA model evaluation are outlined here.
- 3MRA Version 1.x - UA/SA Enhancements (PDF) (76 pp., 1 MB, about PDF) - (Chapter 6 of Volume IV: Evaluating Uncertainty and Sensitivity.) The discussion in this section focuses primarily on the software elements of the SuperMUSE design approach along with key enhancements needed for 3MRA model evaluation.
- Investigating Uncertainty and Sensitivity in Integrated, Multimedia Environmental Models: Tools for 3MRA (PDF) (6 pp., 215 KB, about PDF) - 2002 proceedings from the International Environmental Modeling and Software Society (IEMSs): Integrated Assessment and Decision.
- 3MRA Version 1.0 Software Specifications and Software Download. See especially the download for "Example 3MRA Exit Level Uncertainty Analysis Results for 7 Chemicals".
- PC- Based Supercomputing for Uncertainty and Sensitivity Analysis of Models (PDF) (17" x 11", 1 pp., 525 KB, about PDF) - 2004 poster
- The Relative Importance of the Vadose Zone in Multimedia Risk Assessment Modeling Applied at a National Scale: An Analysis of Benzene Using 3MRA (PDF) (17" x 11", 1 pp., 419 KB, about PDF) - poster presented at AGU, a worldwide scientific community, 2002 Spring Meeting. ( Poster Abstract )
- Investigating Uncertainty and Sensitivity in Integrated, Multimedia Environmental Models: Leveraging PC-Based Supercomputing Through Hardware and Software Tools (PDF) (17" x 11", 1 pp., 388 KB, , about PDF) - poster presented at EPA's 2002 Science Forum
Design Drawings Phase 4 SuperMUSE
- SuperMUSE Bank, Server, & Console Detail (PDF) (1 pp., 112 KB, about PDF)
- SuperMUSE Master Network Switch Wiring Details (PDF) (1 pp., 68 KB, about PDF)
- SuperMUSE Master KVM Switch Wiring Details (PDF) (1 pp., 102 KB, about PDF)
Current experimentation utilizing the SuperMUSE resource includes:
- Evaluating two promising global-based sensitivity analysis techniques (Regional Sensitivity Analysis and Tree Structured Density Estimation).
- Quantifying uncertainty in risk reduction resulting from a national Agency initiative to reduce persistent, bioaccumulative, and toxic (PBT) chemical disposal by 50% by 2005.
- Examining UA/SA of an engineered approach to control atmospheric CO2 levels through use of geological systems and associated natural underground storage capacities.
Babendreier, J. E. and K. J. Castleton. 2005. Investigating Uncertainty and Sensitivity in Integrated, Multimedia Environmental Models: Tools for FRAMES-3MRA. Environmental Modelling & Software. 20(8):1043-1055.
Justin E. Babendreier - biographical information, contact phone number and e-mail.