Skip to content
Project Logo Project Logo

Developed as part of the Scenario Compass Initiative.

SCI Logo SCI Logo

Scenario Evaluation Criteria

Licence: MIT Documentation Status CI DOI

A set of community criteria for evaluating integrated assessment emissions scenarios

Integrated Assessment Models (IAMs) are computational models that produce scenarios of economy, energy system, land use, and greenhouse-gas emissions developments over time, across regions, and under different policy assumptions. Those scenarios are often evaluated in the process of producing new scenarios or when collecting and assessing existing scenarios. Specifically, IAM scenarios are often checked against empirical observations, near-term projections, and sustainability targets to identify scenarios that are outdated, internally inconsistent, technically infeasible, or ecologically or socially undesirable.

This website documents the criteria used to evaluate scenarios in the Scenario Compass Initiative (SCI). The criteria are developed to curate a high-quality scenario ensemble suitable for a wide range of scenario users. Following a transparent, community-driven process, the scenario evaluation criteria are reviewed every year and published in a versioned, machine-readable format. In addition, this website details the chosen thresholds, reference data, and justification for each criterion.


Versions

The documentation is published separately for each version of the criteria.

Version Public review Source Zenodo Note
2026.08.03
Summary
Components
Tutorials
Comments collected 17 Jan–10 Feb 2025; revisions implemented Release v2026.08.03 10.5281/zenodo.21771915 Latest stable release; used in 1st SCI community paper (Riahi et al, 2026; still under review)
2026-update
Summary
Components
Tutorials
Comments collected 21 Aug–21 Sep 2026; revision ongoing Branch 2026-update — Proposed 2026 update; criteria undergoing public review; planned for use in 2nd SCI community paper
latest
Summary
Components
Tutorials
— Branch main — Development version; use only if you know what you're doing

How the criteria are developed

The criteria are developed by the SCI in collaboration with the IAMC Scientific Working Groups on Scenarios and on Data, following the subsequent steps:

  1. Proposal. SCI working groups derive criteria and thresholds from peer-reviewed literature and official data sources.
  2. Open review. Draft criteria are published on this site for public review. The call is advertised online, through community mailing lists, and through professional networks.
  3. Response and revision. Responses to reviewer comments are made public. In the first public review, reviewers received direct responses to their comments and the resulting changes to the criteria via email.
  4. Release. The revised criteria are released as a new version and applied in the annual update of the Scenario Compass ensemble.

In general, criteria are required to refer to variables that are critical for the global energy transition, reported consistently across scenarios in the database, and that can be linked to quantifiable thresholds derived from scientific literature or official data sources.


Two types of criteria

There are two types of evaluation criteria: historical vetting as well as feasibility and sustainability concerns.

  • Historical vetting. Scenarios are evaluated for correctly representing the current energy system. They are marked as inconsistent if they do not match reported historical real-world estimates for time periods in the past.
  • Feasibility and sustainability concerns. The feasibility evaluation concentrates on whether the near-term pace of technology deployment assumed in scenarios stays within empirically grounded bounds. An upper bound checks whether deployment is faster than existing capacity, project pipelines, lead times, and market dynamics can support. A lower bound, near-term plausibility, checks whether deployment falls implausibly low given capacity already operating or committed. The sustainability evaluation focuses on whether scenarios stay within resource and environmental limits, such as levels of bioenergy use, food availability, and cumulative geological CO₂ storage.
SCI scenario evaluation criteria and their implications for the community benchmarks SCI scenario evaluation criteria and their implications for the community benchmarks

SCI's scenario evaluation criteria and their implications for the community benchmarks reported in "Mitigation benchmarks from the 2025 community update of global emissions pathways" (Riahi et al., submitted).

The criteria are applied on the SCI scenario database to derive a high-quality scenario ensemble recommended to use, amongst others for community benchmarks (see Riahi et al., submitted for the first edition of the community benchmarks). Please note that no scenario is removed from the database: all scenarios are recorded and remain available to users. Evaluation criteria are only used to identify a recommended scenario ensemble for benchmarking and other uses of scenario information.

Failing or insufficiently reporting variables required for historical vetting leads to a scenario not being represented in the benchmarking scenario ensemble. When a feasibility or sustainability threshold is crossed, the scenario is flagged with a medium or high concern. A medium concern marks the point where problems may begin to emerge. Medium concerns do not affect whether a scenario is included in the scenario ensemble. A high concern means a value is either inconsistent even with the most optimistic reading of current evidence, or that a scenario raises serious sustainability concerns, leading to the scenario not being considered for the benchmark-relevant ensemble.


Citation

Please cite as:

Verpoort, P. C., Hasegawa, T., Weigmann, P., Mori, S., Osugi, Y., Totake, R., Huppmann, D., Scheifinger, K., & Luderer, G. (2026). Scenario Evaluation Criteria (Version 2026-update) [Computer software]. https://github.com/IAMconsortium/scenario-evaluation-criteria/tree/2026-update


License

All data and code are published under the MIT licence.


Acknowledgments

This collection of scenario evaluation criteria was curated as part of the Scenario Compass Initiative (SCI). Some of the reference data was previously derived in the PRISMA project.

SCI is supported financially by the Bezos Earth Fund. PRISMA has received funding from the European Union’s Horizon Europe research and innovation programme under grant agreement No. 101081604 (PRISMA).

SCI Logo SCI Logo PRISMA Logo PRISMA Logo