diff --git a/CHANGES.md b/CHANGES.md index e097dbdf..0452c1ee 100644 --- a/CHANGES.md +++ b/CHANGES.md @@ -6,24 +6,13 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/), and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html). ## [Unreleased] -* Added `TimexLCASettings`, which holds everything one calculation needs - the demand, the method, the background selection, and every timeline, LCI and LCIA option - so it doubles as the record of what was run. `TimexLCA.from_settings(settings)` builds an object from one, and `TimexLCA.run()` executes the whole pipeline (`build_timeline()`, `lci()`, `static_lcia()`, and `dynamic_lcia()` unless disabled) -* Added repeated `TimexLCA.run()` calls on one object, with per-run overrides (`tlca.run(time_horizon=20)`) that leave the settings unmodified. The background caches, and the timeline itself where its parameters are unchanged, are reused between calls; only a changed demand or method recalculates the base LCA. Changing the background databases is refused, since they fix the columns of the time-explicit matrices -* Added `TimexLCA.compare([settings, ...])` for scenario comparison, returning a `ComparisonResult` whose `summary` DataFrame holds one row per calculation: its scores, the settings behind them, `scenario_*` columns, timeline size, and runtime. It builds one `TimexLCA` per distinct background and runs every calculation sharing that background on it, so a scenario × demand grid only pays for a new object when the background actually changes. `keep_objects=True` keeps the objects for digging into single results; `on_error="record"` puts failures in the table instead of aborting a long sweep -* Removed two noisy log messages: the per-producer warning that a producer was not found in every time-explicit vintage (it fires once per producer, and a background that doesn't cover every producer is normal - the resulting temporal market shares are on the timeline either way), and the `lci()` note listing the mapped databases the timeline does not source from -* Added `TimexLCA(scenario={..., "years": [...]}, create_missing=True)`, which builds background databases the scenario names but the project does not hold, using premise (optional extra: `pip install "bw_timex[premise]"`). Only missing years are built, ecoinvent is imported first if absent, and credentials are read from `PREMISE_KEY` / `ECOINVENT_USERNAME` / `ECOINVENT_PASSWORD` unless passed explicitly. Also available standalone as `bw_timex.ensure_scenario_databases` -* Added `lci(group_background_by_time=...)`, selecting how the background unit LCIs are solved: `None` (default) picks whichever strategy needs fewer solves, `True` forces per-time-step solving, `False` forces per-process solving. Pin it to `False` when re-running `lci()` several times in one session - grouped right-hand sides are sums specific to a run and are never cached, while per-process unit LCIs are, which makes every run after the first free. Requesting `True` where it cannot apply logs a warning and is ignored -* Added per-time-step background solving to `lci(expand_technosphere=False, keep_activity_dimension=False)`. With no activity dimension every temporal-market row landing at the same point in time is summed into the same column anyway, so their background demands can be summed *before* the solve rather than after (`sum_r B A^-1 d_r` = `B A^-1 sum_r d_r`). That trades one solve per distinct background process for one solve per `(time, block)` pair - a market interpolates between vintages in different databases, so a grouped demand is split across the blocks it touches. Which is cheaper is a property of the model, so `lci()` counts both and takes the smaller: on a 123-process foreground with a yearly grouping it is 174 solves against 37, while on the premise EV case (a three-node foreground) per-process stays ahead at 16 against 38, and a warm unit-LCI cache always wins because grouped right-hand sides cannot be cached. On a wide foreground over a premise background (80 distinct background targets, 43k-node vintages) that is 160 solves against 15 (yearly) or 28 (monthly), worth ~1.3-2.5x on cold-run `lci()` wall clock; on models whose background is small the solves are cheap enough that fewer of them changes nothing. Scores, emission timing and the dynamic inventory are unchanged; only the row order of `dynamic_inventory` differs, since the grouped entries are emitted after the timeline loop -* Reduced the runtime of `lci()` on timelines that reuse background processes, by memoizing the *translated* background unit LCIs rather than only their cached payloads. The supply/aggregate caches store node-id-keyed payloads so they can be shared between solvers, but every read had to scatter one back into the reading solver's index space - a `searchsorted` plus a dense allocation - and `lci(expand_technosphere=False)` asks for the same background activity once per timeline row that consumes it. On a 300-activity block read 40 times over, that walk drops from 3.4 s to 1.3 s with the same 300 solves. Aggregates are memoized on first sight; supply columns, which are an order of magnitude larger and read exactly once by the matrix build, only from the second request, so the memory-bound path pays nothing for the memo that `disaggregate_background_lci()` benefits from -* Reduced the runtime of background unit LCIs by memoizing each block's slice of the biosphere matrix, which `unit_aggregate` previously rebuilt on every background activity (0.216 s per 300 activities, against 0.001 s for a single slice). Results are unchanged -* Reduced runtime and memory of `lci()` by solving background unit LCIs against the database they come from instead of the whole time-explicit matrix. A time-explicit technosphere is block lower triangular - the new foreground, temporalized and temporal-market columns sit on top of unmodified copies of the background databases - so each background unit LCI only needs its own block. On a premise EV case (262k time-explicit columns) `lci()` drops from ~25 s to ~22 s, with identical scores; individual background solves go from 0.174 s to 0.048 s -* Reduced the memory of background LCI caching by an order of magnitude: the cache now holds the supply column of a background process (0.35 MB, ~19k nonzeros on ecoinvent) instead of its full inventory matrix `B @ diag(x)` (5.5 MB, ~457k nonzeros). The disaggregated background inventory of a temporal market is rebuilt from those columns when `disaggregate_background_lci()` asks for it, which costs ~6 ms per market and is what previously made large time-explicit systems run out of memory. Results are unchanged -* Fixed background unit LCIs being cached under a key that could not tell a background process from its time-explicit copy (`traverse_background=True`), where a copy could be served the original's supply column -* Added `representative_time` database metadata as the default timing source: `TimexLCA` now maps background databases to points in time by reading their Brightway metadata (as written by premise >= 2.4.9.2), making `database_dates` optional ([#217](https://github.com/brightway-lca/bw_timex/issues/217)) -* Added `set_database_metadata` to record what a database represents (`representative_time`, and scenario fields such as `iam_model` or `pathway`) for databases that don't bring the metadata themselves -* Added `TimexLCA(scenario={...})` to select one background scenario when a project holds several; `TimexLCA` raises and lists the scenarios it found if the choice is ambiguous -* Added pruning of unused vintages in `lci()`: only the databases the timeline actually sources from (plus the dynamic ones, the demand's own, and the traversed processes') are loaded into the time-explicit matrices. Vintages that get no temporal market share - e.g. a 2050 database for a system ending in 2042, or another study's databases in the same project - contributed no matrix entries but still added their processes as columns. Scores are unchanged; on a premise EV case study with three unused databases, the expanded technosphere shrank from 262,141 to 174,758 columns and `lci()` from 31.3 s to 23.1 s -* Added `UnmappedDatabaseError`, raised by `build_timeline()` when the graph traversal reaches a database that is mapped to no point in time - typically a second foreground database that neither holds the functional unit (which is marked `"dynamic"` automatically) nor was marked itself. This previously surfaced as a bare `KeyError` on a node id; the error now names the database, an affected process, and how to map it ([#217](https://github.com/brightway-lca/bw_timex/issues/217)) -* Fixed `TimexLCA(scenario={...})` silently falling back to a plain (non-time-explicit) LCA when the filter matched no database at all, e.g. a typo in a key or value; it now raises a `ValueError` naming the filter and what each of its keys is actually declared as across the project's databases. Also reworded the `database_dates`-specific error messages in `validation.py`, `timeline_builder.py`, and `edge_extractor.py` to also credit `representative_time` metadata as a source of timing ([#217](https://github.com/brightway-lca/bw_timex/issues/217)) +* Added `TimexLCA.run()`, which executes the whole pipeline (`build_timeline()`, `lci()`, `static_lcia()` and `dynamic_lcia()`) in one call, taking every argument of the four stages, either explicitly or through a `TimexLCASettings` object ([#223](https://github.com/brightway-lca/bw_timex/pull/223)) +* Added `TimexLCASettings`, which holds everything one calculation needs - the demand, the method, the background selection, and every timeline, LCI and LCIA option - so it doubles as the record of what was run. Pass one straight to the constructor, `TimexLCA(settings)`, and `run()` needs no arguments ([#223](https://github.com/brightway-lca/bw_timex/pull/223)) +* Added a `__repr__` for `TimexLCA` ([#223](https://github.com/brightway-lca/bw_timex/pull/223)) +* Added `TimexLCA.compare([settings, ...])` for scenario comparison, returning a `ComparisonResult` whose `summary` DataFrame holds one row per calculation: its scores, the settings behind them, `scenario_*` columns, timeline size, and runtime ([#223](https://github.com/brightway-lca/bw_timex/pull/223)) +* Added `TimexLCA(scenario={..., "years": [...]}, create_missing=True)`, which builds background databases the scenario names but the project does not hold, using premise (optional extra: `pip install "bw_timex[premise]"`). Only missing years are built, ecoinvent is imported first if absent, and credentials are read from `PREMISE_KEY` / `ECOINVENT_USERNAME` / `ECOINVENT_PASSWORD` unless passed explicitly. Also available standalone as `bw_timex.ensure_scenario_databases` ([#222](https://github.com/brightway-lca/bw_timex/pull/222)) +* Added `representative_time` database metadata as the default timing source: `TimexLCA` now maps background databases to points in time by reading their Brightway metadata (as written by premise >= 2.4.9.2), making `database_dates` optional ([#217](https://github.com/brightway-lca/bw_timex/issues/217)). Multiple scenarios can be differentiated through a `scenario` metadata field. Database metadata can also be set through new utility function `set_database_metadata` ([#222](https://github.com/brightway-lca/bw_timex/pull/222)) +* Added batched solve for background processes by first collecting demands and solving inventory with diagonalized demand vector ([#220](https://github.com/brightway-lca/bw_timex/pull/220)) ## [1.2.1] - 2026-08-14 * Fixed `ShapeMismatch` in `lci()` for processes with more than one biosphere exchange, by sizing the biosphere `flip_array` to the number of matrix entries (only raised with `bw_processing` >= 1.5; no numeric results change) ([#213](https://github.com/brightway-lca/bw_timex/pull/213)) diff --git a/bw_timex/timex_lca.py b/bw_timex/timex_lca.py index 75825f19..ceb0d476 100644 --- a/bw_timex/timex_lca.py +++ b/bw_timex/timex_lca.py @@ -1,5 +1,5 @@ from collections.abc import Mapping -from dataclasses import dataclass, field, replace +from dataclasses import MISSING, InitVar, dataclass, field, fields, replace from datetime import datetime from functools import partial from time import perf_counter @@ -30,7 +30,6 @@ from peewee import fn from scipy import sparse -from ._lci_cache import BACKGROUND_UNIT_LCI_CACHE, LCI_SOLVE_CACHE, NODES_CACHE FACTORIZE_SOLVES_THRESHOLD = 8 from .database_metadata import resolve_database_dates_from_metadata, split_scenario @@ -76,6 +75,23 @@ class TimexLCASettings: them: they are set when the `TimexLCA` is built and cannot be changed per run. Everything else - the demand, the method, and all the timeline, LCI and LCIA knobs - may vary from run to run on the same object. + + Every knob can also be written grouped by the stage it belongs to, which + keeps a long settings block readable: + + ```python + TimexLCASettings( + demand=demand, + method=method, + timeline={"starting_datetime": datetime(2024, 1, 1), "temporal_grouping": "month"}, + lci={"build_dynamic_biosphere": False}, + lcia={"metric": "GWP", "time_horizon": 20}, + ) + ``` + + The groups are only a way of writing the call: they are unpacked into the + same flat fields, so the two spellings produce equal objects, and + `dataclasses.replace` and `run(**overrides)` stay flat either way. """ #: Fields that pick the background, and so cannot vary between runs of one @@ -83,6 +99,39 @@ class TimexLCASettings: #: `TimexLCA.compare` does for you). FIXED_FIELDS = ("database_dates", "scenario", "use_global_lci_cache") + #: The stage each knob belongs to, for the grouped spelling above. Also + #: what the reference documentation is organised by. + STAGE_GROUPS = { + "timeline": ( + "starting_datetime", + "temporal_grouping", + "interpolation_type", + "edge_filter_function", + "cutoff", + "max_calc", + "graph_traversal", + "traverse_background", + "timeline_args", + "timeline_kwargs", + ), + "lci": ( + "build_dynamic_biosphere", + "expand_technosphere", + "keep_activity_dimension", + ), + "lcia": ( + "static_lcia_enabled", + "dynamic_lcia_enabled", + "metric", + "time_horizon", + "fixed_time_horizon", + "time_horizon_start", + "characterization_functions", + "characterization_function_co2", + "use_disaggregated_lci", + ), + } + # Core parameters demand: dict method: tuple @@ -111,7 +160,14 @@ class TimexLCASettings: # LCIA parameters static_lcia_enabled: bool = True - dynamic_lcia_enabled: bool = True + #: Whether [`run`][bw_timex.timex_lca.TimexLCA.run] characterizes the + #: inventory dynamically. `None` (the default) means "if it can": dynamic + #: characterization needs a characterization function per biosphere flow, + #: and those are only found automatically for ecoinvent / `biosphere3` + #: flows, so a model on its own biosphere database is characterized + #: statically only, unless `characterization_functions` says how. `True` + #: asks for it explicitly, and raises if the flows cannot be matched. + dynamic_lcia_enabled: Optional[bool] = None metric: str = "radiative_forcing" time_horizon: int = 100 fixed_time_horizon: bool = False @@ -120,6 +176,63 @@ class TimexLCASettings: characterization_function_co2: Optional[dict] = None use_disaggregated_lci: bool = False + # The grouped spelling. These are init-only: they are unpacked into the + # flat fields above and never stored, so the object has one shape no matter + # how it was written. `dataclasses.replace` rebuilds from the flat fields + # and leaves these at None, which is why an empty group must be a no-op. + timeline: InitVar[Optional[dict]] = None + lci: InitVar[Optional[dict]] = None + lcia: InitVar[Optional[dict]] = None + + def __post_init__(self, timeline, lci, lcia) -> None: + for group, given in (("timeline", timeline), ("lci", lci), ("lcia", lcia)): + if not given: + continue + for name, value in given.items(): + self._check_group_key(group, name) + # A knob given both flat and in a group is a conflict - unless + # the flat one still holds its default, which is + # indistinguishable from not having been passed at all. + current = getattr(self, name) + if current != _field_default(name) and current != value: + raise TypeError( + f"`{name}` was given both directly (={current!r}) and in " + f"{group}={{{name!r}: {value!r}}}. Pass it once." + ) + setattr(self, name, value) + + @classmethod + def _check_group_key(cls, group: str, name: str) -> None: + """Reject a key that does not belong in the stage group it was put in.""" + if name in cls.STAGE_GROUPS[group]: + return + + for other, names in cls.STAGE_GROUPS.items(): + if name in names: + raise TypeError( + f"`{name}` is a {other} setting, but was passed in {group}={{...}}. " + f"Move it to {other}={{{name!r}: ...}}." + ) + + if name in {f.name for f in fields(cls)}: + raise TypeError( + f"`{name}` is not a stage setting - pass it directly, as " + f"TimexLCASettings({name}=...), not in {group}={{...}}." + ) + + raise TypeError( + f"{group}={{{name!r}: ...}} is not a setting. Valid {group} settings " + f"are: {sorted(cls.STAGE_GROUPS[group])}." + ) + + +def _field_default(name: str): + """The value a `TimexLCASettings` field has when it isn't passed.""" + for f in fields(TimexLCASettings): + if f.name == name: + return f.default_factory() if f.default_factory is not MISSING else f.default + raise KeyError(name) + @dataclass class ComparisonResult: @@ -212,20 +325,26 @@ class TimexLCA: }, ) + # Run the whole calculation. `run` takes every argument of the four stages + # it calls, e.g. metric="GWP" - also available: "pGWP", "pGTP", + # "prospective_radiative_forcing". + tlca.run() + print(tlca.static_score) + print(tlca.dynamic_score) + + # The stages can also be called one at a time, to inspect an intermediate + # result or to re-run only part of the calculation: tlca.build_timeline() # has many optional arguments tlca.lci() tlca.static_lcia() - print(tlca.static_score) - # also available: "GWP", "pGWP", "pGTP", "prospective_radiative_forcing" tlca.dynamic_lcia(metric="radiative_forcing") - print(tlca.dynamic_score) ``` """ def __init__( self, - demand: dict, - method: tuple, + demand: "dict | TimexLCASettings", + method: tuple = None, database_dates: dict = None, scenario: dict = None, create_missing: bool = False, @@ -240,12 +359,17 @@ def __init__( Parameters ---------- - demand : dict[object: float] + demand : dict[object: float] or TimexLCASettings The demand for which the LCA will be calculated. The keys can be Brightway `Node` instances, `(database, code)` tuples, or integer ids. + A [`TimexLCASettings`][bw_timex.timex_lca.TimexLCASettings] can be + passed instead of the demand, in which case it supplies the demand, + the method and the background selection, and becomes the default + settings of [`run`][bw_timex.timex_lca.TimexLCA.run]. method : tuple Tuple defining the LCIA method, such as `('foo', 'bar')` or default methods, such as - `("EF v3.1", "climate change", "global warming potential (GWP100)")` + `("EF v3.1", "climate change", "global warming potential (GWP100)")`. + Required unless a `TimexLCASettings` is given as the first argument. database_dates : dict, optional Fallback for mapping the databases yourself instead of letting `bw_timex` read their metadata - useful for databases written by @@ -304,6 +428,35 @@ def __init__( logger.info("Initializing TimexLCA object...") + settings = None + if isinstance(demand, TimexLCASettings): + settings = demand + also_given = sorted( + name + for name, value in ( + ("method", method), + ("database_dates", database_dates), + ("scenario", scenario), + ) + if value is not None + ) + if also_given: + raise TypeError( + f"A TimexLCASettings already carries {also_given}, so passing " + "it separately is ambiguous. Put everything into the settings " + "(dataclasses.replace makes a modified copy)." + ) + demand = settings.demand + method = settings.method + database_dates = settings.database_dates + scenario = settings.scenario + use_global_lci_cache = settings.use_global_lci_cache + elif method is None: + raise TypeError( + "`method` is required, unless a TimexLCASettings is passed as the " + "first argument." + ) + self.demand = demand self.method = method self.scenario = scenario @@ -334,7 +487,7 @@ def __init__( # Settings this object was built from, if any, and the raw values of the # fields that pick the background. Kept as passed (not as resolved), so # `run` can tell whether a settings object asks for the same background. - self.settings = None + self.settings = settings self._fixed_fields = { "database_dates": database_dates, "scenario": scenario, @@ -436,31 +589,35 @@ def __init__( logger.info("TimexLCA initialized.") + def __repr__(self) -> str: + """What a notebook shows for a cell ending in `tlca.run()`. + + `run` returns the object for chaining, so this is the first thing a + user sees of their results - report the scores it has, rather than an + address in memory. + """ + parts = [f"method={self.method}", f"{len(self.database_dates)} databases"] + for name in ("base_score", "static_score", "dynamic_score"): + try: + parts.append(f"{name}={float(getattr(self, name)):.4g}") + except Exception: # not calculated (or not calculable) - just omit it + pass + return f"" + @classmethod def from_settings(cls, settings: TimexLCASettings) -> "TimexLCA": """Build a `TimexLCA` from a [`TimexLCASettings`][bw_timex.timex_lca.TimexLCASettings]. - The settings' background fields (`TimexLCASettings.FIXED_FIELDS`) are - used to construct the object; the rest become the default arguments of - [`run`][bw_timex.timex_lca.TimexLCA.run]. + Same as passing the settings straight to the constructor, which is the + shorter way to write it: - Examples - -------- ```python settings = TimexLCASettings(demand=demand, method=method, database_dates=dates) - tlca = TimexLCA.from_settings(settings).run() + tlca = TimexLCA(settings).run() print(tlca.static_score) ``` """ - tlca = cls( - demand=settings.demand, - method=settings.method, - database_dates=settings.database_dates, - scenario=settings.scenario, - use_global_lci_cache=settings.use_global_lci_cache, - ) - tlca.settings = settings - return tlca + return cls(settings) def _settings_for_run( self, settings: TimexLCASettings | None, overrides: dict @@ -550,8 +707,10 @@ def run( ) -> "TimexLCA": """Run the whole calculation: timeline, LCI, and LCIA. - Runs `build_timeline()`, `lci()`, `static_lcia()` and, unless disabled, - `dynamic_lcia()`. + Runs `build_timeline()`, `lci()`, `static_lcia()` and `dynamic_lcia()`. + Dynamic characterization is skipped, with a warning, when no + characterization function can be found for the method's biosphere flows + - see `dynamic_lcia_enabled`. Can be called repeatedly on one object to vary the demand, the method, or any knob. The background caches and, where the timeline parameters @@ -629,7 +788,19 @@ def run( logger.info("Step 3/4: Skipping static LCIA (disabled).") # Calculate dynamic LCIA - if settings.dynamic_lcia_enabled: + if settings.dynamic_lcia_enabled is False: + logger.info("Step 4/4: Skipping dynamic LCIA (disabled).") + elif settings.dynamic_lcia_enabled is None and not self._can_characterize_dynamically( + settings.characterization_functions + ): + logger.warning( + "Step 4/4: Skipping dynamic LCIA - no characterization function was " + f"found for the biosphere flows of {self.method}. These are matched " + "automatically for ecoinvent / biosphere3 flows only; for your own " + "flows, pass `characterization_functions={flow_id: function}`. Pass " + "`dynamic_lcia_enabled=True` to raise instead of skipping." + ) + else: logger.info("Step 4/4: Calculating dynamic LCIA...") self.dynamic_lcia( metric=settings.metric, @@ -640,12 +811,31 @@ def run( characterization_function_co2=settings.characterization_function_co2, use_disaggregated_lci=settings.use_disaggregated_lci, ) - else: - logger.info("Step 4/4: Skipping dynamic LCIA (disabled).") logger.info("TimexLCA.run() completed successfully.") return self + def _can_characterize_dynamically(self, characterization_functions) -> bool: + """Whether dynamic characterization has a function for every flow. + + `dynamic_characterization` derives them from the LCIA method, but only + for flows it can resolve in the configured biosphere database - a model + on its own biosphere database gets nothing. Probing it here keeps + `run()`'s default ("characterize dynamically if you can") from turning + into a crash on a project that never asked for it. + """ + if characterization_functions: + return True + + from dynamic_characterization import ( + create_characterization_functions_from_method, + ) + + try: + return bool(create_characterization_functions_from_method(self.method)) + except Exception: # whatever it can't resolve, it can't characterize + return False + @staticmethod def _background_key(settings: TimexLCASettings) -> tuple: """Hashable identity of the background a settings object asks for. @@ -672,9 +862,9 @@ def score(name): "static_score": ( score("static_score") if settings.static_lcia_enabled else float("nan") ), - "dynamic_score": ( - score("dynamic_score") if settings.dynamic_lcia_enabled else float("nan") - ), + # nan when dynamic LCIA was disabled, and when it was skipped + # because the flows could not be characterized. + "dynamic_score": score("dynamic_score"), } for key, value in (settings.scenario or {}).items(): row[f"scenario_{key}"] = value @@ -694,7 +884,7 @@ def score(name): "expand_technosphere": settings.expand_technosphere, "build_dynamic_biosphere": settings.build_dynamic_biosphere, "keep_activity_dimension": settings.keep_activity_dimension, - "metric": settings.metric if settings.dynamic_lcia_enabled else None, + "metric": getattr(self, "current_metric", None), "time_horizon": settings.time_horizon, "fixed_time_horizon": settings.fixed_time_horizon, "timeline_rows": len(self.timeline) if hasattr(self, "timeline") else 0, @@ -997,16 +1187,7 @@ def build_timeline( ) if timeline_cache_key == self._last_timeline_build_key: self.timeline = self._cached_timeline - return self.timeline[ - [ - "date_producer", - "producer_name", - "date_consumer", - "consumer_name", - "amount", - "temporal_market_shares", - ] - ] + return self.timeline_summary if edge_filter_function is None and not traverse_background: logger.info( @@ -1085,16 +1266,7 @@ def build_timeline( self._cached_timeline = self.timeline self._dynamic_lcia_inventory_cache.clear() - return self.timeline[ - [ - "date_producer", - "producer_name", - "date_consumer", - "consumer_name", - "amount", - "temporal_market_shares", - ] - ] + return self.timeline_summary def lci( self, @@ -1919,6 +2091,31 @@ def dynamic_lcia( # Core properties # ################### + @property + def timeline_summary(self) -> pd.DataFrame: + """The readable view of `timeline`: what happens when, and sourced from where. + + `timeline` itself carries the bookkeeping columns the calculation needs + (hashes, time-mapped ids). This is what `build_timeline` returns, and + what to look at after [`run`][bw_timex.timex_lca.TimexLCA.run], which + returns the object rather than a timeline. + """ + if not hasattr(self, "timeline"): + raise AttributeError( + "Timeline not yet built. Call TimexLCA.build_timeline() or " + "TimexLCA.run() first." + ) + return self.timeline[ + [ + "date_producer", + "producer_name", + "date_consumer", + "consumer_name", + "amount", + "temporal_market_shares", + ] + ] + @property def base_score(self) -> float: """ diff --git a/docs/api/timex_lca.md b/docs/api/timex_lca.md index c9811728..7ff75f48 100644 --- a/docs/api/timex_lca.md +++ b/docs/api/timex_lca.md @@ -8,4 +8,8 @@ tags: The main user-facing class of `bw_timex`. A `TimexLCA` takes a Brightway demand, an LCIA method and a set of time-specific databases, and produces a time-explicit inventory. +`TimexLCA.run()` runs the whole calculation - timeline, inventory, characterization - in one call, and is the recommended entry point; the stage methods it calls remain available for finer control. + +`TimexLCASettings` holds everything one calculation needs, and can be passed straight to `TimexLCA()`; `TimexLCA.compare()` runs a list of them and returns a `ComparisonResult`. See [Repeated Runs & Scenario Comparison](../content/getting_started/configured_runs.md). + ::: bw_timex.timex_lca diff --git a/docs/content/create_premise_dbs.md b/docs/content/create_premise_dbs.md new file mode 100644 index 00000000..9cb2afb5 --- /dev/null +++ b/docs/content/create_premise_dbs.md @@ -0,0 +1,29 @@ +--- +icon: lucide/database +tags: + - premise + - background database +--- + +# Creating `premise`-Databases On-the-fly + +If your project does not hold any `premise` databases yet, `bw_timex` can build them +with premise on-the-fly. Add the years to the scenario and pass +`create_missing=True`, alongside the premise key and ecoinvent credentials: + +```python +tlca = TimexLCA( + demand={("foreground", "ev"): 1}, + method=("EF v3.1", "climate change", "global warming potential (GWP100)"), + scenario={ + "iam_model": "remind", + "pathway": "SSP2-PkBudg500", + "system_model": "cutoff", + "ecoinvent_version": "3.12", + "years": [2020, 2030, 2040], + }, + create_missing=True, + premise_key="dummy_premise_decryption_key", # or $PREMISE_KEY + ecoinvent_credentials=("dummy_user", "dummy_password"), # or $ECOINVENT_USERNAME / _PASSWORD +) +``` diff --git a/docs/content/getting_started/adding_temporal_information.md b/docs/content/getting_started/adding_temporal_information.md index f7b4c0c2..fa9da1aa 100644 --- a/docs/content/getting_started/adding_temporal_information.md +++ b/docs/content/getting_started/adding_temporal_information.md @@ -247,8 +247,8 @@ end ) ``` -So, as you can see, the processes at specific time steps reside within a separate normal -Brightway database. `bw_timex` picks these up automatically, as long as each database +Background processes at specific time steps reside within normal +Brightway databases. `bw_timex` picks these up automatically, as long as each database says which point in time it represents: ```python @@ -259,11 +259,14 @@ set_database_metadata("background", representative_time=datetime(2020, 1, 1)) set_database_metadata("background_2030", representative_time=datetime(2030, 1, 1)) ``` -You only do this once per database - it is stored in your Brightway project. Databases -exported by [premise](https://premise.readthedocs.io/en/latest/introduction.html) -**>= 2.4.9.2** bring this metadata with them, so there is nothing to do for those; -for databases from an earlier premise, set it yourself as above. The foreground doesn't -represent a specific point in time and is distributed over time instead; `bw_timex` +If you want to assess different scenarios, you can also add the scenario as metadata: + +```python +set_database_metadata("background_2030_low", representative_time=datetime(2030, 1, 1), scenario="low") +set_database_metadata("background_2030_high", representative_time=datetime(2030, 1, 1), scenario="high") +``` + +The foreground doesn't represent a specific point in time and is distributed over time instead; `bw_timex` treats the databases holding your functional unit that way automatically. !!! tip "Foreground split across several databases" @@ -278,10 +281,22 @@ treats the databases holding your functional unit that way automatically. Otherwise `build_timeline()` raises an `UnmappedDatabaseError`, naming the database it could not place in time. +You only do this once per database - it is stored in your Brightway project. + !!! tip "Data sources" You can use whatever data source you want for the time-specific process data. [premise](https://premise.readthedocs.io/en/latest/introduction.html) is a nice package from the Brightway cosmos, but you can also use any custom scenario. +### Working with `premise`-generated databases + +Databases generated by [premise](https://premise.readthedocs.io/en/latest/introduction.html) +**>= 2.4.9.2** bring the `representative_date` metadata with them, so there is nothing to do for those; +for databases from an earlier premise, set it yourself as above. + +In addition to the `representative_date`, `premise` databases also carry metadata on what IAM scenario they reflect, e.g., "REMIND-EU NDC", similar to the `scenario` metadata shown above. `bw_timex` can use this data as well, e.g., when comparing different scenarios; see [Step 2](build_process_timeline.md). + + + ### Several databases for the same point in time More than one database may carry the same date. This is useful when you modify @@ -298,16 +313,7 @@ set_database_metadata("my_background_2030", representative_time=datetime(2030, 1 For each process, `bw_timex` interpolates only between the databases that actually contain it, matched on `name`, `reference product` and `location`. A copy that only exists in `my_background_2020` and `my_background_2030` is therefore sourced from -those two, while an untouched process is sourced from the `ecoinvent_*` ones. - -!!! warning - - Give your copies a distinct `name`, `reference product` or `location`. If the - same triplet occurs in two databases that share a date, `bw_timex` cannot tell - which one you mean and raises an error. A process that exists at only some of - the points in time is sourced from the ones it does exist at: it is interpolated - between those, or — if it exists at only one of them — used unchanged for every - point in time. Either way, `bw_timex` logs a warning naming the databases it used. +those two. ## Foreground exchanges that change over time diff --git a/docs/content/getting_started/build_process_timeline.md b/docs/content/getting_started/build_process_timeline.md index 00d800ab..228814fb 100644 --- a/docs/content/getting_started/build_process_timeline.md +++ b/docs/content/getting_started/build_process_timeline.md @@ -9,8 +9,7 @@ tags: # Step 2 - Building the process timeline With all the temporal information prepared, we can now instantiate our TimexLCA object. -This is just like a normal Brightway LCA object - the timing of the background databases -comes from their metadata: +This is just like a normal Brightway LCA object: ```python from bw_timex import TimexLCA @@ -21,11 +20,8 @@ tlca = TimexLCA( ) ``` -The metadata is written by premise >= 2.4.9.2, or by you with -`set_database_metadata` (see [Step 1](adding_temporal_information.md)). - -If your project holds several IAM scenarios, say which one you want - `bw_timex` lists -what it found rather than guessing: +If you have multiple background databases representing different scenarios in your Brightway-project, +you additionally need to specify which scenario you wish to assess. ```python tlca = TimexLCA( @@ -36,13 +32,14 @@ tlca = TimexLCA( ``` Any metadata key filters (`iam_model`, `pathway`, `system_model`, ...), and databases -that don't carry it - your foreground, your own vintages - are kept. Comparing scenarios -is the same script in a loop over filters. +that don't carry it - your foreground, your own vintages - are kept. To compare several +scenarios, hand them all to +[`TimexLCA.compare()`](configured_runs.md#comparing-scenarios), which runs them into one +table. !!! tip "Mapping the databases by hand" - `database_dates` maps database names to dates yourself and replaces the metadata - entirely: + You can also explicitly pass `database_dates` to `TimexLCA`, not requiring any database metadata: ```python tlca = TimexLCA(demand, method, database_dates={ diff --git a/docs/content/getting_started/configured_runs.md b/docs/content/getting_started/configured_runs.md new file mode 100644 index 00000000..6f62bea5 --- /dev/null +++ b/docs/content/getting_started/configured_runs.md @@ -0,0 +1,113 @@ +--- +icon: lucide/git-compare +tags: + - configured runs + - comparison + - scenario +--- + +# Configured Runs & Scenario Comparisons + +The main functions to call to run a `TimexLCA` are `build_timeline()`, `lci()`, `static_lcia()` and `dynamic_lcia()`, see Walkthrough Steps 2-4. To save you from typing this repeatedly, and capture all configurations in one place, we provide the `TimexLCA.run()` function. + +## One calculation as one object + +To specify the configuration of the run, we use the `TimexLCASettings` class: + +```python +settings = TimexLCASettings( + demand=demand, + method=method, + scenario={ + "iam_model": "remind", + "pathway": "SSP2-NDC", + }, + timeline={ + "starting_datetime": datetime(2020, 6, 1), + "graph_traversal": "bfs", # breadth first search + # other settings from .build_timeline() + }, + lci={ + "build_dynamic_biosphere": False, + # other settings from .lci() + }, + lcia={ + "metric": "radiative_forcing", + "time_horizon": 100, + # other settings from .static_lcia() or .dynamic_lcia() + }, + label="built 2020", # optional +) +``` + +A `TimexLCASettings` object holds everything a calculation needs, including demand, method, scenario, and every chosen option otherwise passed through `.build_timeline`, `.lci()`, `.static_lcia()` or `.dynamic_lcia()`. + +The `timeline` / `lci` / `lcia` groups in the settings configuration keep a long settings block readable, but they are entirely +optional: `TimexLCASettings(..., starting_datetime=..., graph_traversal=..., metric=...)` builds the same +object. + +## Running a configured time-explicit LCA + +To run the whole time-explicit LCA pipeline based on `TimexLCASettings`, you can directly type: + +```python +tlca = TimexLCA(settings).run() + +print(tlca.static_score, tlca.dynamic_score) +``` + +`run()` executes `build_timeline()`, `lci()`, `static_lcia()` and `dynamic_lcia()`. +Every argument of those four methods is a field on the settings, so one object is +also the record of what was run: keep it, log it, or put a list of them into +[`compare()`](#comparing-scenarios). + +## Re-running with different settings + +Call `run()` again on the same object. Individual overrides apply to that call +only, leaving both the settings object and the object's own settings untouched: + +```python +tlca.run(time_horizon=20) +tlca.run(starting_datetime=datetime(2030, 1, 1)) +tlca.run(demand={("foreground", "B"): 1}) +``` + +!!! tip "Passing a whole settings object" + + `run()` also takes a replacement: `tlca.run(other_settings)`. Build it with + `dataclasses.replace(settings, starting_datetime=...)` so the other fields carry over + unchanged. + +## Comparing scenarios + +`TimexLCA.compare()` takes a list of settings and returns a `ComparisonResult`: + +```python +from dataclasses import replace + +comparison = TimexLCA.compare( + [ + replace(settings, scenario={"pathway": "SSP2-Base"}, label="Base"), + replace(settings, scenario={"pathway": "SSP2-PkBudg500"}, label="PkBudg500"), + ] +) + +comparison.summary # one row per calculation +``` + +`summary` is a DataFrame carrying the scores next to every setting that produced +them, plus `scenario_*` columns, the timeline size and the runtime - so the table +is its own record of the comparison, and plots directly: + +```python +comparison.summary.plot.bar(x="label", y="static_score") +``` + +Each distinct background gets its own `TimexLCA`, and every calculation sharing +that background runs on it - so a scenario × demand grid only pays for a new +object when the background actually changes. + +| Argument | Default | Description | +|---|---|---| +| `keep_objects` | `False` | Keep each `TimexLCA` in `ComparisonResult.objects`, to dig into one result's timeline or inventory afterwards. Off by default because a large comparison holds a lot of memory this way | +| `on_error` | `"raise"` | `"record"` puts the failure in the row's `error` column and carries on, instead of aborting a long unattended sweep | \ No newline at end of file diff --git a/docs/content/getting_started/index.md b/docs/content/getting_started/index.md index b7d6e42d..f736ac2c 100644 --- a/docs/content/getting_started/index.md +++ b/docs/content/getting_started/index.md @@ -14,14 +14,10 @@ This section will help you quickly getting started with your time-explicit LCA p In the following sections, we'll walk through the steps 1-4, considering a very simple dummy system. If you directly want to look at a more complex example, take a look at our [example collection](../examples/index.md). If you're interested in the full details on how `bw_timex` works, you can also skip to our [Theory Section](../theory.md). For some general guidance on what kind of LCA is suitable for what kind of case, check out our [decision tree](../decisiontree.md). -!!! tip "In a hurry?" +!!! tip "Run an entire `TimexLCA` with a single function call" - Our [Quick Start](quickstart.md) page condenses all of this into one page of code and reference tables. + Instead of following steps 2-4 in sequence, you can configure everything in a single object and just call a simple `.run()` function, see [Configured Runs & Scenario Comparisons](configured_runs.md). !!! tip "You want more interaction?" - [Launch this tutorial on Binder!](https://mybinder.org/v2/gh/brightway-lca/bw_timex/HEAD?labpath=notebooks%2Ftutorials%2F1_getting_started.ipynb) In this interactive environment, you can directly run the bw_timex code yourself whilst following along. - -!!! tip "Using an AI coding agent?" - - Install our [Brightway agent skills](https://github.com/TimoDiepers/brightway-skills) — they teach Claude Code (or any other [agentskills.io](https://agentskills.io)-compatible agent) this workflow, including the parts that are easy to get wrong. + [Launch this tutorial on Binder!](https://mybinder.org/v2/gh/brightway-lca/bw_timex/HEAD?labpath=notebooks%2Ftutorials%2F1_getting_started.ipynb) In this interactive environment, you can directly run the bw_timex code yourself whilst following along. \ No newline at end of file diff --git a/docs/content/getting_started/lcia.md b/docs/content/getting_started/lcia.md index f3104aea..859f34ae 100644 --- a/docs/content/getting_started/lcia.md +++ b/docs/content/getting_started/lcia.md @@ -96,4 +96,6 @@ tlca.plot_dynamic_characterized_inventory() ![Plot showing the radiative forcing over time](../data/dynamic_characterized_inventory_gwp.svg){ style="display:block;margin:0 auto" }
-For most of the functions we used here, there are numerous optional arguments and settings you can tweak. We explore some of them in our other [Examples](../examples/index.md), but when in doubt check out our [docstrings](../../api/index.md), which provide information also for the more advanced settings - so please browse through them as needed ☀️ +For most of the functions we used here, there are numerous optional arguments and settings you can tweak. We explore some of them in our other [Examples](../examples/index.md), but when in doubt check out our [docstrings](../../api/index.md), which provide information also for the more advanced settings - so please browse through them as needed ☀️ + +Instead of running the whole `build_timeline()`, `lci()`, `static_lcia()` and `dynamic_lcia()` pipeline yourself, you can also take the short route through a simple `.run`-call, as described under [Configured Runs & Scenario Comparisons](configured_runs.md). \ No newline at end of file diff --git a/docs/content/getting_started/quickstart.md b/docs/content/getting_started/quickstart.md index 78003eca..857d33a0 100644 --- a/docs/content/getting_started/quickstart.md +++ b/docs/content/getting_started/quickstart.md @@ -84,6 +84,33 @@ print(tlca.dynamic_score) tlca.plot_dynamic_characterized_inventory() ``` +As an alternative to running `.build_timeline()`, `.lci()`and `.static_lcia()`/`.dynamic_lcia()`, you can also do: + +```python +from bw_timex import TimexLCA, TimexLCASettings + +settings = TimexLCASettings( + demand={("foreground", "A"): 1}, + method=("our", "method"), + time_horizon=100, + ) +tlca = TimexLCA(settings).run() +print(tlca.dynamic_score) + +tlca.run(time_horizon=20) # re-run, but with different time horizon +print(tlca.dynamic_score) + +``` + +For comparing different settings conveniently, you can simply run: + +```python +comparison = tlca.compare([settings, other_settings]) +print(comparison.summary) +``` + + + --- ## Temporal Information Cheat Sheet @@ -154,7 +181,7 @@ and it overrides the metadata entirely. If the project does not hold the scenario's databases yet, `bw_timex` can build them with premise instead of raising. Add the years to the scenario and pass -`create_missing=True`: +`create_missing=True`, alongside the premise key and ecoinvent credentials: ```python tlca = TimexLCA( @@ -168,27 +195,11 @@ tlca = TimexLCA( "years": [2020, 2030, 2040], }, create_missing=True, + premise_key="dummy_premise_decryption_key", # or $PREMISE_KEY + ecoinvent_credentials=("dummy_user", "dummy_password"), # or $ECOINVENT_USERNAME / _PASSWORD ) ``` -Only missing years are built, so running this again builds nothing. ecoinvent is -imported first if the project has none. Each vintage is a full copy of ecoinvent: -expect tens of minutes and roughly 2-4 GB per year. Building needs the optional -`premise` extra, which belongs in [its own environment](../installation.md#building-background-databases-automatically). - -Two optional scenario keys tune the build. `sectors` narrows what premise updates -(all sectors by default), and `source_database` names the ecoinvent to build from, -if it is not the one `import_ecoinvent_release` writes. A vintage built with -narrowed `sectors` only satisfies a later request for the same sectors, so it is -recorded in the database metadata. - -The sector names are premise's own, and a name it does not know is only rejected -once premise has extracted ecoinvent, so check them here first: - -`biomass`, `electricity`, `cement`, `steel`, `fuels`, `heat`, `renewable`, -`metals`, `mining`, `battery`, `cdr`, `emissions`, `final energy`, `cars`, -`two_wheelers`, `trucks`, `buses`, `trains`, `ships`, `external`. - ### `build_timeline()` | Argument | Default | Description | @@ -221,6 +232,22 @@ Result is stored in `tlca.timeline` (a DataFrame with `date_producer`, `producer | `time_horizon_start` | `None` | Start of the fixed time horizon, defaults to now | | `characterization_functions` | `None` | `{biosphere_flow_id: function}`. Not needed for ecoinvent / `biosphere3`, where flows are mapped automatically | +### `run()` and `compare()` + +See [Repeated Runs & Scenario Comparison](configured_runs.md). + +| Call | Description | +|---|---| +| `TimexLCA(settings)` | Build the object from a `TimexLCASettings` | +| `tlca.run()` | The four steps above, in order, with that object's settings | +| `tlca.run(**overrides)` | The same, with settings overridden for this call only. Refuses a changed `scenario` / `database_dates` | +| `TimexLCA.compare([settings, ...])` | Run several, into `ComparisonResult.summary` | + +`TimexLCASettings` takes every argument listed above - `demand`, `method`, +`database_dates`, `scenario`, plus all of `build_timeline()`, `lci()` and +`dynamic_lcia()` - and a `label` naming its row in a comparison. Enable or skip +the LCIA steps with `static_lcia_enabled` / `dynamic_lcia_enabled`. + --- ## Results diff --git a/docs/superpowers/plans/2026-08-03-multiple-databases-per-date.md b/docs/superpowers/plans/2026-08-03-multiple-databases-per-date.md deleted file mode 100644 index 59275321..00000000 --- a/docs/superpowers/plans/2026-08-03-multiple-databases-per-date.md +++ /dev/null @@ -1,957 +0,0 @@ -# Multiple background databases per point in time — Implementation Plan - -> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. - -**Goal:** Let several static background databases share the same date in `database_dates`, so modified copies of background processes can live in their own databases instead of being written into ecoinvent/premise. - -**Architecture:** Today `TimelineBuilder` and the background-traversal edge extractor invert `database_dates` into `{date: database}`, which collapses same-date databases. Both are replaced by a *per producer* resolution: a producer's candidate databases are the static databases containing a `(name, reference product, location)` match for it, and interpolation runs over the dates of those candidates only. Public API is unchanged. - -**Tech Stack:** Python, bw2data / bw2calc / bw_temporalis, pandas, pytest, loguru. - -## Global Constraints - -- Spec: `docs/superpowers/specs/2026-08-03-multiple-databases-per-date-design.md`. -- No public API change. `database_dates` stays `{database_name: datetime | "dynamic"}`. -- Cross-database matching key stays `(name, reference product, location)`. -- Same triplet in two databases at the same date is an error, never a silent pick. -- A producer present in fewer vintages than exist is allowed, with one `logger.warning`. -- TDD: write the failing test, watch it fail, then implement. -- Commit messages: Conventional Commits, no Claude/AI attribution, no `Co-Authored-By` trailer. -- Run tests with the project venv: `.venv/bin/pytest`. - ---- - -### Task 1: Per-producer temporal market shares - -**Files:** -- Create: `tests/fixtures/same_date_databases_fixture.py` -- Create: `tests/test_same_date_databases.py` -- Modify: `tests/conftest.py` -- Modify: `bw_timex/timeline_builder.py:508-596` (`add_column_temporal_market_shares_to_timeline`) and `:667-693` (delete `add_interpolation_weights_at_intersection_to_background`) - -**Interfaces:** -- Consumes: nothing from earlier tasks. -- Produces: - - `TimelineBuilder.candidate_databases_for_producers(producers: set) -> dict[int, dict[datetime, str]]` - - `TimelineBuilder.market_producer_matches: dict[int, dict[str, int]]` — `{producer_id: {database_name: node_id}}`, set as a side effect of the above. Task 3 consumes it. - - pytest fixture `same_date_db` (no return value; writes the databases into a fresh project). - -- [ ] **Step 1: Write the fixture** - -Create `tests/fixtures/same_date_databases_fixture.py`: - -```python -import bw2data as bd -import pytest -from bw2data.tests import bw2test - - -@pytest.fixture -@bw2test -def same_date_db(): - """Four static background databases on two dates. - - `background_2020` / `background_2030` hold an untouched `electricity` - process. `modified_2020` / `modified_2030` hold a copy of `steel` with its - end-of-life removed, named `steel, without EOL`; they carry the *same* - dates as the two `background_*` databases. The foreground consumes one of - each, so both must become temporal markets that interpolate within their - own family of databases. - - CO2 amounts differ per vintage so the interpolation is visible in the score: - electricity 10 (2020) / 5 (2030), steel 20 (2020) / 10 (2030). - """ - biosphere = bd.Database("biosphere") - biosphere.write({("biosphere", "CO2"): {"type": "emission", "name": "carbon dioxide"}}) - node_co2 = biosphere.get("CO2") - - amounts = { - "background_2020": {"electricity": 10, "steel": 20}, - "background_2030": {"electricity": 5, "steel": 10}, - } - - for year in ("2020", "2030"): - background = bd.Database(f"background_{year}") - background.register() - modified = bd.Database(f"modified_{year}") - modified.register() - - electricity = background.new_node("electricity", name="electricity", unit="kWh") - electricity["reference product"] = "electricity" - electricity["location"] = "GLO" - electricity.save() - electricity.new_edge(input=electricity, amount=1, type="production").save() - electricity.new_edge( - input=node_co2, - amount=amounts[f"background_{year}"]["electricity"], - type="biosphere", - ).save() - - steel = background.new_node("steel", name="steel", unit="kg") - steel["reference product"] = "steel" - steel["location"] = "GLO" - steel.save() - steel.new_edge(input=steel, amount=1, type="production").save() - steel.new_edge( - input=node_co2, - amount=amounts[f"background_{year}"]["steel"], - type="biosphere", - ).save() - - # The study's own copy of `steel`, without EOL, in its own database. - steel_copy = steel.copy(code="steel_without_eol", database=f"modified_{year}") - steel_copy["name"] = "steel, without EOL" - steel_copy["reference product"] = "steel, without EOL" - steel_copy.save() - - foreground = bd.Database("foreground") - foreground.register() - fu = foreground.new_node("fu", name="fu", unit="unit") - fu["reference product"] = "fu" - fu["location"] = "GLO" - fu.save() - fu.new_edge(input=fu, amount=1, type="production").save() - fu.new_edge( - input=bd.Database("background_2020").get("electricity"), amount=1, type="technosphere" - ).save() - fu.new_edge( - input=bd.Database("modified_2020").get("steel_without_eol"), - amount=1, - type="technosphere", - ).save() - - bd.Method(("GWP", "example")).write([(("biosphere", "CO2"), 1)]) - - for dbname in bd.databases: - bd.Database(dbname).process() -``` - -Register it in `tests/conftest.py` by adding this import next to the other fixture imports: - -```python -from .fixtures.same_date_databases_fixture import same_date_db -``` - -- [ ] **Step 2: Write the failing tests** - -Create `tests/test_same_date_databases.py`: - -```python -"""Several static background databases may share the same date.""" - -from datetime import datetime - -import bw2data as bd -import pytest -from loguru import logger - -from bw_timex import TimexLCA - -METHOD = ("GWP", "example") -DATABASE_DATES = { - "background_2020": datetime.strptime("2020", "%Y"), - "background_2030": datetime.strptime("2030", "%Y"), - "modified_2020": datetime.strptime("2020", "%Y"), - "modified_2030": datetime.strptime("2030", "%Y"), - "foreground": "dynamic", -} - - -def _shares_by_producer(timeline): - return { - row.producer_name: row.temporal_market_shares - for row in timeline.itertuples() - if row.temporal_market_shares - } - - -def test_shares_route_within_each_database_family(same_date_db): - tlca = TimexLCA({("foreground", "fu"): 1}, METHOD, DATABASE_DATES) - tlca.build_timeline(starting_datetime="2025-01-01") - shares = _shares_by_producer(tlca.timeline) - - assert set(shares["electricity"]) == {"background_2020", "background_2030"} - assert set(shares["steel, without EOL"]) == {"modified_2020", "modified_2030"} - assert shares["electricity"]["background_2020"] == pytest.approx(0.5, abs=0.01) - assert shares["steel, without EOL"]["modified_2020"] == pytest.approx(0.5, abs=0.01) - - -def test_score_interpolates_within_each_family(same_date_db): - tlca = TimexLCA({("foreground", "fu"): 1}, METHOD, DATABASE_DATES) - tlca.build_timeline(starting_datetime="2025-01-01") - tlca.lci() - tlca.static_lcia() - # electricity: 0.5*10 + 0.5*5 = 7.5; steel: 0.5*20 + 0.5*10 = 15 - assert tlca.static_score == pytest.approx(22.5, abs=0.2) - - -def test_same_triplet_at_same_date_raises(same_date_db): - """A copy that keeps name/reference product/location is ambiguous.""" - collision = bd.Database("modified_2020").new_node( - "electricity_collision", name="electricity", unit="kWh" - ) - collision["reference product"] = "electricity" - collision["location"] = "GLO" - collision.save() - collision.new_edge(input=collision, amount=1, type="production").save() - bd.Database("modified_2020").process() - - tlca = TimexLCA({("foreground", "fu"): 1}, METHOD, DATABASE_DATES) - with pytest.raises(ValueError, match="more than one database"): - tlca.build_timeline(starting_datetime="2025-01-01") - - -def test_producer_in_a_single_vintage_warns_and_is_time_invariant(same_date_db): - """A copy made into only one vintage stays constant over time, with a warning.""" - bd.Database("modified_2030").get("steel_without_eol").delete() - bd.Database("modified_2030").process() - - messages = [] - sink_id = logger.add(messages.append, level="WARNING") - try: - tlca = TimexLCA({("foreground", "fu"): 1}, METHOD, DATABASE_DATES) - tlca.build_timeline(starting_datetime="2025-01-01") - finally: - logger.remove(sink_id) - - shares = _shares_by_producer(tlca.timeline) - assert shares["steel, without EOL"] == {"modified_2020": 1} - assert any("steel, without EOL" in message for message in messages) -``` - -- [ ] **Step 3: Run the tests and watch them fail** - -Run: `.venv/bin/pytest tests/test_same_date_databases.py -v` -Expected: failures. `test_shares_route_within_each_database_family` fails because -`reversed_database_dates` maps each date to a single database, so both producers get -the same two database names (whichever of `background_*` / `modified_*` won the -inversion); `test_same_triplet_at_same_date_raises` fails because nothing raises. - -- [ ] **Step 4: Add the candidate resolver** - -In `bw_timex/timeline_builder.py`, insert this method directly above -`add_column_temporal_market_shares_to_timeline`: - -```python - def candidate_databases_for_producers(self, producers: set) -> dict: - """Map each producer to the static databases that hold a match for it. - - Returns ``{producer_id: {date: database_name}}``. A candidate is a static - background database containing a node with the same ``(name, reference - product, location)`` as the producer. Several databases may share a date; - if two of them hold a match for the same producer, the model is ambiguous - and this raises. - - As a side effect, ``self.market_producer_matches`` is filled with - ``{producer_id: {database_name: node_id}}``, which is exactly what - ``TimexLCA.add_interdatabase_activity_mapping_from_timeline`` needs. - """ - triplets = {} - for producer in producers: - node = self.nodes[producer] - key = (node["name"], node.get("reference product"), node["location"]) - triplets.setdefault(key, []).append(producer) - - candidates = {producer: {} for producer in producers} - matches = {producer: {} for producer in producers} - for node in self.nodes.values(): - date = self.database_dates_static.get(node["database"]) - if date is None: - continue - key = (node["name"], node.get("reference product"), node["location"]) - for producer in triplets.get(key, ()): - already = candidates[producer].get(date) - if already is not None and already != node["database"]: - raise ValueError( - f"Producer '{node['name']}' was found in more than one database " - f"at {date:%Y-%m-%d}: '{already}' and '{node['database']}'. " - "bw_timex cannot tell which one its temporal market should use. " - "Give the copy a distinct name, reference product or location." - ) - candidates[producer][date] = node["database"] - matches[producer][node["database"]] = node.id - - number_of_dates = len(set(self.database_dates_static.values())) - for producer, producer_candidates in candidates.items(): - if len(producer_candidates) < number_of_dates: - logger.warning( - "Producer '{}' was only found in {} of {} time-explicit database " - "date(s): {}. Its temporal market can only draw on those.", - self.nodes[producer]["name"], - len(producer_candidates), - number_of_dates, - sorted(producer_candidates.values()), - ) - - self.market_producer_matches = matches - return candidates -``` - -- [ ] **Step 5: Rewrite the share computation** - -Replace the body of `add_column_temporal_market_shares_to_timeline` from the -`dates_list = [` assignment through the `return tl_df` (currently -`timeline_builder.py:538-596`) with: - -```python - if "date_producer" not in list(tl_df.columns): - raise ValueError("The timeline does not contain dates.") - - if interpolation_type not in ("linear", "nearest"): - raise ValueError( - f"Sorry, but {interpolation_type} interpolation is not available yet." - ) - - if self.traverse_background: - market_producers = self._leaf_background_producers(tl_df) - else: - market_producers = self.node_collections["first_level_background_static"] - - producers_in_timeline = { - producer - for producer in tl_df["producer"].unique() - if producer in market_producers - } - candidate_databases = self.candidate_databases_for_producers( - producers_in_timeline - ) - - weight_cache = {} - shares = [] - for producer, producer_date in zip(tl_df["producer"], tl_df["date_producer"]): - if producer not in producers_in_timeline: - shares.append(None) - continue - candidates = candidate_databases[producer] - sorted_dates = tuple(sorted(candidates)) - cache_key = (sorted_dates, producer_date) - if cache_key not in weight_cache: - if interpolation_type == "nearest": - weights = self.find_closest_date(producer_date, sorted_dates) - else: - weights = self.get_weights_for_interpolation_between_nearest_years( - producer_date, sorted_dates, interpolation_type - ) - weight_cache[cache_key] = { - candidates[date]: share for date, share in weights.items() - } - shares.append(weight_cache[cache_key]) - - tl_df["temporal_market_shares"] = shares - - return tl_df -``` - -Keep the `if not self.database_dates_static:` early return at the top of the method -exactly as it is. - -- [ ] **Step 6: Delete the dead method** - -Delete `add_interpolation_weights_at_intersection_to_background` -(`bw_timex/timeline_builder.py:667-693`). It is the only other reader of the removed -`self.reversed_database_dates` and has no call site. - -Verify: `grep -rn "reversed_database_dates\|add_interpolation_weights_at_intersection" bw_timex/ tests/` -Expected: no matches. - -- [ ] **Step 7: Run the new tests** - -Run: `.venv/bin/pytest tests/test_same_date_databases.py -v` -Expected: 4 passed. - -- [ ] **Step 8: Run the full suite for regressions** - -Run: `.venv/bin/pytest tests/ -q` -Expected: all pass. `tests/test_timeline_builder.py::TestGetWeightsForInterpolation::test_unsupported_interpolation_type` still passes because -`get_weights_for_interpolation_between_nearest_years` keeps its own type check. - -- [ ] **Step 9: Commit** - -```bash -git add bw_timex/timeline_builder.py tests/test_same_date_databases.py \ - tests/fixtures/same_date_databases_fixture.py tests/conftest.py -git commit -m "feat: resolve temporal market databases per producer - -Several static databases may now share a date in database_dates. A -producer's candidate databases are those holding a (name, reference -product, location) match for it, so copies kept in a separate database -interpolate within their own family." -``` - ---- - -### Task 2: Same-date routing when traversing the background - -**Files:** -- Modify: `bw_timex/edge_extractor.py:115-137` (`_variant_shares_for_date`) and its call site at `:379` -- Modify: `tests/fixtures/same_date_databases_fixture.py` -- Modify: `tests/test_same_date_databases.py` - -**Interfaces:** -- Consumes: `TimelineBuilder.candidate_databases_for_producers` from Task 1 (behaviour only — the extractor resolves candidates from `interdatabase_activity_mapping`, which `TimexLCA.add_full_interdatabase_activity_mapping` fills up front whenever `traverse_background=True`). -- Produces: `VariantSplitMixin._candidate_databases_for_node(node_id: int) -> dict[datetime, str]` and the new signature `_variant_shares_for_date(producer_date, node_id: int) -> dict[str, float]`. - -- [ ] **Step 1: Add a second fixture with a background chain** - -The traversal test needs a chain *inside* the modified family, but adding it to -`same_date_db` would change the scores asserted in Task 1. Refactor the fixture file -into a shared writer plus two fixtures. Replace the whole of -`tests/fixtures/same_date_databases_fixture.py` with: - -```python -import bw2data as bd -import numpy as np -import pytest -from bw2data.tests import bw2test -from bw_temporalis import TemporalDistribution - - -def _write_same_date_databases(with_background_chain: bool = False): - """Write four static background databases on two dates. - - `background_2020` / `background_2030` hold an untouched `electricity` - process. `modified_2020` / `modified_2030` hold a copy of `steel` with its - end-of-life removed, named `steel, without EOL`; they carry the *same* - dates as the two `background_*` databases. The foreground consumes one of - each, so both must become temporal markets that interpolate within their - own family of databases. - - CO2 amounts differ per vintage so the interpolation is visible in the score: - electricity 10 (2020) / 5 (2030), steel 20 (2020) / 10 (2030). - - With `with_background_chain=True`, each modified database also holds a - `smelting` process that the copy consumes through a temporal distribution. - It exists only in the modified family and is reached only when the - background is traversed. - """ - biosphere = bd.Database("biosphere") - biosphere.write({("biosphere", "CO2"): {"type": "emission", "name": "carbon dioxide"}}) - node_co2 = biosphere.get("CO2") - - amounts = { - "background_2020": {"electricity": 10, "steel": 20}, - "background_2030": {"electricity": 5, "steel": 10}, - } - - for year in ("2020", "2030"): - background = bd.Database(f"background_{year}") - background.register() - modified = bd.Database(f"modified_{year}") - modified.register() - - electricity = background.new_node("electricity", name="electricity", unit="kWh") - electricity["reference product"] = "electricity" - electricity["location"] = "GLO" - electricity.save() - electricity.new_edge(input=electricity, amount=1, type="production").save() - electricity.new_edge( - input=node_co2, - amount=amounts[f"background_{year}"]["electricity"], - type="biosphere", - ).save() - - steel = background.new_node("steel", name="steel", unit="kg") - steel["reference product"] = "steel" - steel["location"] = "GLO" - steel.save() - steel.new_edge(input=steel, amount=1, type="production").save() - steel.new_edge( - input=node_co2, - amount=amounts[f"background_{year}"]["steel"], - type="biosphere", - ).save() - - # The study's own copy of `steel`, without EOL, in its own database. - steel_copy = steel.copy(code="steel_without_eol", database=f"modified_{year}") - steel_copy["name"] = "steel, without EOL" - steel_copy["reference product"] = "steel, without EOL" - steel_copy.save() - - if with_background_chain: - # Reached only by descending into the background. The 10-year offset - # pushes it towards the 2030 vintage of the modified family. - smelting = modified.new_node("smelting", name="smelting", unit="kg") - smelting["reference product"] = "smelting" - smelting["location"] = "GLO" - smelting.save() - smelting.new_edge(input=smelting, amount=1, type="production").save() - smelting.new_edge( - input=node_co2, - amount=amounts[f"background_{year}"]["steel"], - type="biosphere", - ).save() - - copy_to_smelting = steel_copy.new_edge( - input=smelting, amount=1, type="technosphere" - ) - copy_to_smelting["temporal_distribution"] = TemporalDistribution( - date=np.array([10], dtype="timedelta64[Y]"), - amount=np.array([1.0]), - ) - copy_to_smelting.save() - - foreground = bd.Database("foreground") - foreground.register() - fu = foreground.new_node("fu", name="fu", unit="unit") - fu["reference product"] = "fu" - fu["location"] = "GLO" - fu.save() - fu.new_edge(input=fu, amount=1, type="production").save() - fu.new_edge( - input=bd.Database("background_2020").get("electricity"), amount=1, type="technosphere" - ).save() - fu.new_edge( - input=bd.Database("modified_2020").get("steel_without_eol"), - amount=1, - type="technosphere", - ).save() - - bd.Method(("GWP", "example")).write([(("biosphere", "CO2"), 1)]) - - for dbname in bd.databases: - bd.Database(dbname).process() - - -@pytest.fixture -@bw2test -def same_date_db(): - """Four static background databases on two dates, no background chain.""" - _write_same_date_databases() - - -@pytest.fixture -@bw2test -def same_date_deep_db(): - """Same as `same_date_db`, plus a `smelting` chain in the modified family.""" - _write_same_date_databases(with_background_chain=True) -``` - -Add the second fixture to `tests/conftest.py`, next to the existing import: - -```python -from .fixtures.same_date_databases_fixture import same_date_db, same_date_deep_db -``` - -(replacing the single-name import added in Task 1). - -- [ ] **Step 2: Write the failing test** - -Append to `tests/test_same_date_databases.py`: - -```python -def test_background_traversal_routes_within_the_modified_family(same_date_deep_db): - """Descending into the background must not confuse same-date databases.""" - tlca = TimexLCA({("foreground", "fu"): 1}, METHOD, DATABASE_DATES) - tlca.build_timeline( - starting_datetime="2025-01-01", - graph_traversal="bfs", - traverse_background=True, - ) - producers = set(tlca.timeline["producer_name"]) - assert "smelting" in producers - - # `smelting` exists only in the modified family, so no background_* database - # may be picked up for it. - smelting_rows = tlca.timeline[tlca.timeline["producer_name"] == "smelting"] - for shares in smelting_rows["temporal_market_shares"]: - if shares: - assert set(shares) <= {"modified_2020", "modified_2030"} - - tlca.lci() - tlca.static_lcia() - assert tlca.static_score > 0 -``` - -- [ ] **Step 3: Run it and watch it fail** - -Run: `.venv/bin/pytest tests/test_same_date_databases.py::test_background_traversal_routes_within_the_modified_family -v` -Expected: FAIL — `_variant_shares_for_date` inverts `database_dates_static` globally, so -the 2030 slot resolves to whichever of `background_2030` / `modified_2030` came last, and -resolving `smelting` there raises `KeyError` (or routes to the wrong family). - -- [ ] **Step 4: Resolve candidates per node in the extractor** - -In `bw_timex/edge_extractor.py`, replace `_variant_shares_for_date` with: - -```python - def _candidate_databases_for_node(self, node_id: int) -> dict: - """``{date: database_name}`` for the static databases holding a match. - - Candidates come from the interdatabase mapping (built up front by - ``TimexLCA.add_full_interdatabase_activity_mapping`` whenever the - background is traversed), plus the node's own database. - """ - dates_static = getattr(self, "database_dates_static", None) or {} - try: - siblings = dict(self.interdatabase_activity_mapping[node_id]) - except KeyError: - siblings = {} - db_names = set(siblings) - node = self.bw_node_proxies.get(node_id) - if node is not None: - db_names.add(node["database"]) - - candidates = {} - for db_name in sorted(db_names): - date = dates_static.get(db_name) - if date is None: - continue - if date in candidates: - raise ValueError( - f"Node {node_id} was found in more than one database at " - f"{date:%Y-%m-%d}: '{candidates[date]}' and '{db_name}'. " - "bw_timex cannot tell which one to use. Give the copy a " - "distinct name, reference product or location." - ) - candidates[date] = db_name - return candidates - - def _variant_shares_for_date(self, producer_date, node_id: int) -> dict: - """Return ``{db_name: weight}`` interpolation shares for a cohort date. - - Maps the producer's absolute cohort date onto the static background - databases that actually hold the producer, using the same interpolation - as the timeline builder so leaf and descended routing agree. - """ - from datetime import datetime as _dt - - candidates = self._candidate_databases_for_node(node_id) - sorted_dates = tuple(sorted(candidates)) - if not sorted_dates: - return {} - - if isinstance(producer_date, np.datetime64): - producer_date = producer_date.astype("datetime64[s]").astype(_dt) - - if getattr(self, "interpolation_type", "linear") == "nearest": - weights = nearest_date_weight(producer_date, sorted_dates) - else: - weights = linear_interpolation_weights(producer_date, sorted_dates) - return {candidates[d]: w for d, w in (weights or {}).items()} -``` - -- [ ] **Step 5: Update the call site** - -At `bw_timex/edge_extractor.py:379`, inside `_emit_variant_split_single_consumer`, -change: - -```python - for db_name, weight in self._variant_shares_for_date(date).items(): -``` - -to: - -```python - for db_name, weight in self._variant_shares_for_date(date, node_id).items(): -``` - -`node_id` is a keyword-only parameter of that method's signature; confirm with -`grep -n "def _emit_variant_split" -A 4 bw_timex/edge_extractor.py` before editing, and -if a caller passes it under another name, use that name. - -- [ ] **Step 6: Run the traversal tests** - -Run: `.venv/bin/pytest tests/test_same_date_databases.py tests/test_background_traversal.py -v` -Expected: all pass. - -- [ ] **Step 7: Run the full suite** - -Run: `.venv/bin/pytest tests/ -q` -Expected: all pass. - -- [ ] **Step 8: Commit** - -```bash -git add bw_timex/edge_extractor.py tests/fixtures/same_date_databases_fixture.py \ - tests/test_same_date_databases.py -git commit -m "feat: route traversed background nodes per node, not per date - -The variant split resolved a cohort date through a global date -> database -inversion, which collapsed databases sharing a date. Candidates now come -from the node's own interdatabase mapping." -``` - ---- - -### Task 3: Reuse the triplet scan for the interdatabase mapping - -**Files:** -- Modify: `bw_timex/timex_lca.py:1568-1621` (`add_interdatabase_activity_mapping_from_timeline`) -- Modify: `tests/test_same_date_databases.py` - -**Interfaces:** -- Consumes: `TimelineBuilder.market_producer_matches` (`{producer_id: {database_name: node_id}}`) from Task 1. -- Produces: nothing new. - -- [ ] **Step 1: Write the failing test** - -Append to `tests/test_same_date_databases.py`: - -```python -def test_interdatabase_mapping_is_filled_by_the_timeline_builder(same_date_db): - """The builder's triplet scan feeds the mapping; no second scan is needed.""" - tlca = TimexLCA({("foreground", "fu"): 1}, METHOD, DATABASE_DATES) - tlca.build_timeline(starting_datetime="2025-01-01") - - steel_copy_2020 = bd.Database("modified_2020").get("steel_without_eol") - steel_copy_2030 = bd.Database("modified_2030").get("steel_without_eol") - assert ( - tlca.interdatabase_activity_mapping.find_match(steel_copy_2020.id, "modified_2030") - == steel_copy_2030.id - ) - - electricity_2020 = bd.Database("background_2020").get("electricity") - electricity_2030 = bd.Database("background_2030").get("electricity") - assert ( - tlca.interdatabase_activity_mapping.find_match(electricity_2020.id, "background_2030") - == electricity_2030.id - ) - # The copy has no counterpart in the untouched family, and none is invented. - with pytest.raises(KeyError): - tlca.interdatabase_activity_mapping.find_match(steel_copy_2020.id, "background_2030") -``` - -- [ ] **Step 2: Run it** - -Run: `.venv/bin/pytest tests/test_same_date_databases.py::test_interdatabase_mapping_is_filled_by_the_timeline_builder -v` -Expected: PASS already — `add_interdatabase_activity_mapping_from_timeline` builds the -same mapping with its own scan. This test pins the behaviour that the next step must not -break. If it fails, stop and fix the mapping before continuing. - -- [ ] **Step 3: Reuse the builder's matches** - -In `bw_timex/timex_lca.py`, in `add_interdatabase_activity_mapping_from_timeline`, -insert directly after the `if not hasattr(self, "timeline"): raise AttributeError(...)` -block: - -```python - # The timeline builder already resolved every temporal-market producer to - # its counterparts while computing the market shares. Reuse that instead - # of scanning every background node a second time. - matches = getattr(self.timeline_builder, "market_producer_matches", None) - if matches: - self.interdatabase_activity_mapping.update(matches) - self.interdatabase_activity_mapping.make_reciprocal() - return -``` - -- [ ] **Step 4: Run the tests** - -Run: `.venv/bin/pytest tests/test_same_date_databases.py -v` -Expected: all pass, including the mapping test from Step 1. - -- [ ] **Step 5: Run the full suite** - -Run: `.venv/bin/pytest tests/ -q` -Expected: all pass. - -- [ ] **Step 6: Commit** - -```bash -git add bw_timex/timex_lca.py tests/test_same_date_databases.py -git commit -m "perf: reuse the timeline builder's producer matches - -The market-share computation already resolves every temporal-market -producer to its counterparts in the other static databases, so the -post-timeline scan over all background nodes is redundant." -``` - ---- - -### Task 4: Document databases sharing a date - -**Files:** -- Modify: `docs/content/getting_started/adding_temporal_information.md:266-276` - -**Interfaces:** -- Consumes: the behaviour from Tasks 1 and 2. -- Produces: nothing consumed by later tasks. - -- [ ] **Step 1: Add the documentation** - -In `docs/content/getting_started/adding_temporal_information.md`, after the `!!! note` -block that ends with the premise link (currently line 275), insert: - -```markdown -### Several databases for the same point in time - -More than one database may carry the same date. This is useful when you modify -background processes: keep the modified copies in your own database per point in -time, instead of writing them into ecoinvent or premise. - -```python -database_dates = { - "ecoinvent_2020": datetime.strptime("2020", "%Y"), - "ecoinvent_2030": datetime.strptime("2030", "%Y"), - "my_background_2020": datetime.strptime("2020", "%Y"), # your modified copies - "my_background_2030": datetime.strptime("2030", "%Y"), - "foreground": "dynamic", -} -``` - -For each process, `bw_timex` interpolates only between the databases that actually -contain it, matched on `name`, `reference product` and `location`. A copy that only -exists in `my_background_2020` and `my_background_2030` is therefore sourced from -those two, while an untouched process is sourced from the `ecoinvent_*` ones. - -!!! warning - - Give your copies a distinct `name`, `reference product` or `location`. If the - same triplet occurs in two databases that share a date, `bw_timex` cannot tell - which one you mean and raises an error. A process that exists in only some of - the points in time is used unchanged for all of them, and logs a warning. -``` - -- [ ] **Step 2: Check the docs build** - -Run: `.venv/bin/python -c "import pathlib; print(pathlib.Path('docs/content/getting_started/adding_temporal_information.md').read_text().count('Several databases for the same point in time'))"` -Expected: `1` - -- [ ] **Step 3: Commit** - -```bash -git add docs/content/getting_started/adding_temporal_information.md -git commit -m "docs: document several databases sharing a point in time" -``` - ---- - -### Task 5: Move the notebook copies out of the premise databases - -**Files:** -- Modify: `notebooks/example_electric_vehicle_premise_simple.ipynb` (the "Standard brightway modelling" cell, cell id `cell-4`, and the `database_dates` cell, `cell-12`, and the `add_temporal_distribution_to_exchange` cell, `cell-10`) -- Modify: `notebooks/example_electric_vehicle_premise.ipynb` (same copy loop; find it with `grep -n "glider_production_without_eol"`) -- Modify: `notebooks/teaching/teaching_example_ev_premise.ipynb` (same) - -**Interfaces:** -- Consumes: the behaviour from Tasks 1-4. -- Produces: nothing. - -**Prerequisite:** this task needs the local Brightway project `ei312_REMIND_EU` with the -three premise databases. If it is not available, stop and report — do not fake the run. - -- [ ] **Step 1: Rewrite the copy loop in the simple notebook** - -In `notebooks/example_electric_vehicle_premise_simple.ipynb`, replace the copy loop -(the block starting at the comment `# The ecoinvent processes for the ev parts already -contain their end-of-life treatment.` and ending at the `exc.delete()` line) with: - -```python -# The ecoinvent processes for the ev parts already contain their end-of-life treatment. -# We want to model the end of life separately, so we create copies without it. Those -# copies live in our own databases - one per point in time - so the premise databases -# stay untouched. bw_timex allows several databases to share a date. -modified_dbs = {} -for db in [db_2020, db_2030, db_2040]: - year = db.name[-4:] - modified_name = f"ev_background_{year}" - if modified_name in bd.databases: - del bd.databases[modified_name] - modified_db = bd.Database(modified_name) - modified_db.register() - modified_dbs[db.name] = modified_db - - for name, code_, eol_name in [ - ( - "glider production, passenger car", - "glider_production_without_eol", - "market for used glider, passenger car", - ), - ( - "powertrain production, for electric passenger car", - "powertrain_production_without_eol", - "market for used powertrain from electric passenger car, manual dismantling", - ), - # For the battery, some waste treatment is buried in the cell production. - # For simplicity, we just leave it in there. - ( - "battery production, Li-ion, LiMn2O4, rechargeable", - "battery_production_without_eol", - None, - ), - ]: - without_eol = db.get(name=name).copy(code=code_, database=modified_name) - without_eol["name"] = f"{name}, without EOL" - without_eol.save() - if eol_name: - for exc in without_eol.exchanges(): - if exc.input["name"] == eol_name: - exc.delete() - - modified_db.process() -``` - -- [ ] **Step 2: Point the foreground at the new databases** - -In the same cell, change the three lookups of the copies from `db_2020` to the modified -database: - -```python -# Background processes our foreground links to -ev_background_2020 = modified_dbs[db_2020.name] -glider_production = ev_background_2020.get(code="glider_production_without_eol") -powertrain_production = ev_background_2020.get(code="powertrain_production_without_eol") -battery_production = ev_background_2020.get(code="battery_production_without_eol") -``` - -The remaining lookups (`glider_eol`, `powertrain_eol`, `battery_eol`, -`electricity_production`) stay on `db_2020`. - -- [ ] **Step 3: Update the temporal distribution calls** - -In the `add_temporal_distribution_to_exchange` cell, change `input_database=db_2020.name` -to `input_database=ev_background_2020.name` for the three `without EOL` inputs (glider, -powertrain, battery). The electricity and end-of-life calls keep `db_2020.name`. - -- [ ] **Step 4: Add the new databases to `database_dates`** - -In the `database_dates` cell: - -```python -database_dates = { - db_2020.name: datetime.strptime("2020", "%Y"), - db_2030.name: datetime.strptime("2030", "%Y"), - db_2040.name: datetime.strptime("2040", "%Y"), - "ev_background_2020": datetime.strptime("2020", "%Y"), - "ev_background_2030": datetime.strptime("2030", "%Y"), - "ev_background_2040": datetime.strptime("2040", "%Y"), - "foreground": "dynamic", -} -``` - -- [ ] **Step 5: Run the notebook end to end** - -Run: `.venv/bin/jupyter nbconvert --to notebook --execute --inplace notebooks/example_electric_vehicle_premise_simple.ipynb` -Expected: no errors; the timeline's `temporal_market_shares` for the `without EOL` -producers name `ev_background_*` databases, and `tlca.static_score` stays close to the -previously recorded 10744 kg CO2-eq (the model is unchanged, only where the copies live). - -- [ ] **Step 6: Apply the same change to the other two notebooks** - -`notebooks/example_electric_vehicle_premise.ipynb` and -`notebooks/teaching/teaching_example_ev_premise.ipynb` contain the same copy loop. -Locate it with `grep -n "glider_production_without_eol" notebooks/example_electric_vehicle_premise.ipynb notebooks/teaching/teaching_example_ev_premise.ipynb`, -and apply Steps 1-4 there, keeping each notebook's own variable names for the premise -databases. - -Run: `.venv/bin/jupyter nbconvert --to notebook --execute --inplace notebooks/example_electric_vehicle_premise.ipynb` -Run: `.venv/bin/jupyter nbconvert --to notebook --execute --inplace notebooks/teaching/teaching_example_ev_premise.ipynb` -Expected: both execute without errors. - -- [ ] **Step 7: Commit** - -```bash -git add notebooks/example_electric_vehicle_premise_simple.ipynb \ - notebooks/example_electric_vehicle_premise.ipynb \ - notebooks/teaching/teaching_example_ev_premise.ipynb -git commit -m "docs: keep the modified ev background processes in their own databases" -``` - ---- - -## Verification - -- [ ] `.venv/bin/pytest tests/ -q` — all pass -- [ ] `grep -rn "reversed_database_dates" bw_timex/` — no matches -- [ ] The three premise notebooks execute end to end and no longer write into the - `ei312_REMIND-EU_SSP2_NDC_*` databases diff --git a/docs/superpowers/plans/2026-08-21-representative-time-metadata.md b/docs/superpowers/plans/2026-08-21-representative-time-metadata.md deleted file mode 100644 index e23a9643..00000000 --- a/docs/superpowers/plans/2026-08-21-representative-time-metadata.md +++ /dev/null @@ -1,1510 +0,0 @@ -# Representative time as database metadata — Implementation Plan - -> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. - -**Goal:** `TimexLCA` learns the point in time each background database represents from that database's own Brightway metadata (`representative_time`, as written by premise), so `database_dates` becomes an optional explicit override instead of a required argument. - -**Architecture:** A new module `bw_timex/database_metadata.py` owns everything about database metadata: writing it (`set_database_metadata`) and resolving a `{database: datetime | "dynamic"}` mapping out of the project (`resolve_database_dates_from_metadata`), including scenario filtering and the ambiguity error. `TimexLCA.__init__` calls it in one place and is otherwise untouched: everything downstream still consumes `self.database_dates`. - -**Tech Stack:** Python 3.10+, `bw2data` (database metadata lives in `bd.databases[name]`, a JSON-serialized dict), `pydantic` (input validation, see `bw_timex/validation.py`), `loguru`, `pytest` with `bw2data.tests.bw2test` fixtures. - -**Spec:** `docs/superpowers/specs/2026-08-21-representative-time-metadata-design.md` - -## Global Constraints - -- Run everything with the project venv: `.venv/bin/python`, `.venv/bin/pytest`. -- `database_dates` semantics do not change. When it is passed, it is the whole mapping and metadata is never read. -- `bd.databases` is serialized to JSON. A `datetime` written into it breaks `bd.databases.flush()`. Every date stored in metadata is an ISO 8601 string. -- Metadata keys, exactly as premise writes them: `representative_time`, `iam_model`, `pathway`, `system_model`, `ecoinvent_version`, `premise_version`, `external_scenarios`, `scenarios`. -- Scenario identity keys (the ambiguity signature) are exactly: `("iam_model", "pathway", "system_model", "ecoinvent_version", "external_scenarios")`. `premise_version` is deliberately not one of them. -- `notebooks/examples/paper_case_study.ipynb` must not be modified by any task. -- Commit messages carry no AI attribution and no `Co-Authored-By` trailer. - ---- - -### Task 1: `database_metadata` module — writing metadata - -**Files:** -- Create: `bw_timex/database_metadata.py` -- Modify: `bw_timex/validation.py` (append a `DatabaseMetadataInputs` model) -- Modify: `bw_timex/__init__.py` (export `set_database_metadata`) -- Create: `tests/test_database_metadata.py` - -**Interfaces:** -- Consumes: nothing from earlier tasks. -- Produces: - - `bw_timex.database_metadata.set_database_metadata(database: str | bd.Database, **metadata) -> dict` - - constants `REPRESENTATIVE_TIME: str = "representative_time"`, `SCENARIOS: str = "scenarios"`, `DYNAMIC: str = "dynamic"`, `SCENARIO_SIGNATURE_KEYS: tuple[str, ...]`, `BRIGHTWAY_METADATA_KEYS: frozenset[str]` - - `bw_timex.database_metadata._normalize_representative_time(value, database: str) -> datetime | str` - -- [ ] **Step 1: Write the failing tests** - -Create `tests/test_database_metadata.py`: - -```python -"""Tests for reading and writing what a Brightway database represents.""" - -from datetime import datetime - -import bw2data as bd -import pytest - -from bw_timex import set_database_metadata - -# ─── Tests for set_database_metadata ─── - - -@pytest.mark.usefixtures("temporal_grouping_db_monthly") -class TestSetDatabaseMetadata: - - def test_datetime_is_stored_as_iso_string(self): - set_database_metadata("db_2022", representative_time=datetime(2022, 1, 1)) - assert bd.databases["db_2022"]["representative_time"] == "2022-01-01T00:00:00" - - def test_iso_string_is_stored_as_given(self): - set_database_metadata("db_2022", representative_time="2022-01-01") - assert bd.databases["db_2022"]["representative_time"] == "2022-01-01" - - def test_dynamic_is_allowed(self): - set_database_metadata("foreground", representative_time="dynamic") - assert bd.databases["foreground"]["representative_time"] == "dynamic" - - def test_scenario_fields_are_stored(self): - set_database_metadata( - "db_2022", - representative_time=datetime(2022, 1, 1), - iam_model="remind", - pathway="SSP2-PkBudg500", - ) - assert bd.databases["db_2022"]["iam_model"] == "remind" - assert bd.databases["db_2022"]["pathway"] == "SSP2-PkBudg500" - - def test_database_object_is_accepted(self): - set_database_metadata( - bd.Database("db_2022"), representative_time=datetime(2022, 1, 1) - ) - assert bd.databases["db_2022"]["representative_time"] == "2022-01-01T00:00:00" - - def test_existing_metadata_is_kept(self): - before = bd.databases["db_2022"]["backend"] - set_database_metadata("db_2022", representative_time=datetime(2022, 1, 1)) - assert bd.databases["db_2022"]["backend"] == before - - def test_survives_flush_and_reload(self): - set_database_metadata("db_2022", representative_time=datetime(2022, 1, 1)) - bd.databases.__init__() # re-read from disk - assert bd.databases["db_2022"]["representative_time"] == "2022-01-01T00:00:00" - - def test_unregistered_database_raises(self): - with pytest.raises(ValueError, match="not registered"): - set_database_metadata("no_such_db", representative_time=datetime(2022, 1, 1)) - - def test_unparseable_representative_time_raises(self): - with pytest.raises(ValueError, match="representative_time"): - set_database_metadata("db_2022", representative_time="whenever") - - def test_non_serializable_value_raises(self): - with pytest.raises(ValueError, match="JSON"): - set_database_metadata("db_2022", pathway=object()) - - def test_no_metadata_raises(self): - with pytest.raises(ValueError, match="at least one"): - set_database_metadata("db_2022") -``` - -- [ ] **Step 2: Run the tests to verify they fail** - -Run: `.venv/bin/pytest tests/test_database_metadata.py -v` -Expected: FAIL — `ImportError: cannot import name 'set_database_metadata' from 'bw_timex'` - -- [ ] **Step 3: Write the module** - -Create `bw_timex/database_metadata.py`: - -```python -"""Read and write what a Brightway database represents. - -`bw_timex` needs to know which point in time each background database stands -for. That information is stored in the database's own Brightway metadata -(`bw2data.databases[name]`), where premise also writes it when it exports a -prospective database: - -```python -{ - "premise_version": "2.4.9.1", - "iam_model": "remind", - "pathway": "SSP2-PkBudg500", - "representative_time": "2050-01-01T00:00:00", - "ecoinvent_version": "3.10.1", - "system_model": "cutoff", -} -``` - -Brightway stores this mapping as JSON, so dates are kept as ISO 8601 strings. -""" - -from __future__ import annotations - -import json -from datetime import datetime -from typing import Any - -import bw2data as bd - -REPRESENTATIVE_TIME = "representative_time" -SCENARIOS = "scenarios" -DYNAMIC = "dynamic" - -#: Metadata keys that identify the scenario a database represents. Two -#: databases differing in any of these represent different scenarios. -#: `premise_version` is deliberately absent: re-running premise on the same -#: pathway must not look like a second scenario. -SCENARIO_SIGNATURE_KEYS = ( - "iam_model", - "pathway", - "system_model", - "ecoinvent_version", - "external_scenarios", -) - -#: Keys Brightway maintains itself, filtered out when reporting to the user -#: which metadata a project's databases carry. -BRIGHTWAY_METADATA_KEYS = frozenset( - { - "backend", - "depends", - "dirty", - "format", - "geocollections", - "modified", - "number", - "processed", - "searchable", - } -) - - -def _database_name(database: Any) -> str: - """The name of a database given either as a name or as a `bd.Database`.""" - name = getattr(database, "name", database) - if not isinstance(name, str): - raise ValueError( - f"database must be a database name or a bw2data Database, got " - f"{type(database).__name__}." - ) - return name - - -def _normalize_representative_time(value: Any, database: str) -> datetime | str: - """Turn a stored `representative_time` into a datetime or `"dynamic"`.""" - if isinstance(value, datetime): - return value - if isinstance(value, str): - if value == DYNAMIC: - return DYNAMIC - try: - return datetime.fromisoformat(value) - except ValueError: - raise ValueError( - f"Database '{database}' has an invalid `{REPRESENTATIVE_TIME}` " - f"metadata value: {value!r}. Expected an ISO 8601 datetime string " - f"(e.g. '2030-01-01'), a datetime, or '{DYNAMIC}'." - ) from None - raise ValueError( - f"Database '{database}' has an invalid `{REPRESENTATIVE_TIME}` metadata " - f"value of type {type(value).__name__}: {value!r}. Expected an ISO 8601 " - f"datetime string, a datetime, or '{DYNAMIC}'." - ) - - -def set_database_metadata(database: str | bd.Database, **metadata) -> dict: - """ - Store what a database represents in its Brightway metadata. - - Use this for databases that don't bring the metadata themselves, e.g. - databases you built yourself or that were exported by a premise version - older than the one writing scenario metadata. `TimexLCA` reads - `representative_time` from all databases of the project to map them to - points in time, so this replaces passing `database_dates`. - - Parameters - ---------- - database : str or bw2data.Database - Name of the database, or the database itself. Must be registered. - **metadata : - Metadata to store. `representative_time` accepts a `datetime`, an ISO - 8601 string, or `"dynamic"` and is always stored as a string, because - Brightway serializes database metadata to JSON. Any other key is stored - as given and must be JSON-serializable. Keys that premise writes, and - that `TimexLCA(scenario=...)` can select on, are `iam_model`, - `pathway`, `system_model`, `ecoinvent_version` and `premise_version`. - - Returns - ------- - dict - The database's metadata after the update. - - Examples - -------- - ```python - set_database_metadata("db_2030", representative_time=datetime(2030, 1, 1)) - set_database_metadata( - "my_2050_variant", - representative_time="2050-01-01", - iam_model="remind", - pathway="SSP2-PkBudg500", - ) - ``` - """ - from .validation import DatabaseMetadataInputs - - name = _database_name(database) - DatabaseMetadataInputs(database=name, metadata=metadata) - - if name not in bd.databases: - raise ValueError( - f"Database '{name}' is not registered in this Brightway project. " - f"Available databases: {sorted(bd.databases)}." - ) - - serialized = {} - for key, value in metadata.items(): - if key == REPRESENTATIVE_TIME: - normalized = _normalize_representative_time(value, name) - serialized[key] = ( - normalized if normalized == DYNAMIC else normalized.isoformat() - ) - continue - try: - json.dumps(value) - except TypeError: - raise ValueError( - f"Metadata value for '{key}' is not JSON-serializable: {value!r}. " - f"Brightway stores database metadata as JSON." - ) from None - serialized[key] = value - - bd.databases[name].update(serialized) - bd.databases.flush() - return bd.databases[name] -``` - -- [ ] **Step 4: Add the validation model** - -Append to `bw_timex/validation.py`: - -```python -class DatabaseMetadataInputs(BaseModel): - """Validates inputs to set_database_metadata""" - - model_config = {"arbitrary_types_allowed": True} - - database: str - metadata: dict - - @field_validator("metadata") - @classmethod - def validate_metadata(cls, v: dict) -> dict: - if not v: - raise ValueError( - "Provide at least one metadata field, e.g. " - "`representative_time=datetime(2030, 1, 1)`." - ) - for key in v: - if not isinstance(key, str): - raise ValueError( - f"Metadata keys must be strings, got {type(key).__name__}: {key}." - ) - return v -``` - -- [ ] **Step 5: Export it** - -In `bw_timex/__init__.py`, add the import next to the other helper imports and the name to `__all__` (in the `# utils` block, alphabetically after `plot_characterized_inventory_as_waterfall`): - -```python -from .database_metadata import set_database_metadata -``` - -```python - "set_database_metadata", -``` - -- [ ] **Step 6: Run the tests to verify they pass** - -Run: `.venv/bin/pytest tests/test_database_metadata.py -v` -Expected: PASS (12 tests) - -- [ ] **Step 7: Commit** - -```bash -git add bw_timex/database_metadata.py bw_timex/validation.py bw_timex/__init__.py tests/test_database_metadata.py -git commit -m "feat: add set_database_metadata to store what a database represents" -``` - ---- - -### Task 2: Resolve database dates from metadata - -**Files:** -- Modify: `bw_timex/database_metadata.py` -- Modify: `tests/test_database_metadata.py` - -**Interfaces:** -- Consumes: `REPRESENTATIVE_TIME`, `SCENARIOS`, `DYNAMIC`, `_normalize_representative_time`, `set_database_metadata` from Task 1. -- Produces: `resolve_database_dates_from_metadata(scenario: dict | None = None) -> dict[str, datetime | str]` — every registered database carrying `representative_time`, mapped to a `datetime` or `"dynamic"`. Multi-scenario databases are excluded. Scenario filtering and the ambiguity error come in Task 3; this task's version accepts the argument and ignores it. - -- [ ] **Step 1: Write the failing tests** - -Append to `tests/test_database_metadata.py`: - -```python -from bw_timex.database_metadata import resolve_database_dates_from_metadata - -# ─── Tests for resolving database dates from metadata ─── - - -@pytest.mark.usefixtures("temporal_grouping_db_monthly") -class TestResolveFromMetadata: - - def test_empty_project_metadata_resolves_to_nothing(self): - assert resolve_database_dates_from_metadata() == {} - - def test_iso_strings_resolve_to_datetimes(self): - set_database_metadata("db_2022", representative_time="2022-01-01") - set_database_metadata("db_2024", representative_time="2024-01-01") - assert resolve_database_dates_from_metadata() == { - "db_2022": datetime(2022, 1, 1), - "db_2024": datetime(2024, 1, 1), - } - - def test_dynamic_metadata_resolves_to_dynamic(self): - set_database_metadata("db_2022", representative_time="2022-01-01") - set_database_metadata("foreground", representative_time="dynamic") - resolved = resolve_database_dates_from_metadata() - assert resolved["foreground"] == "dynamic" - assert resolved["db_2022"] == datetime(2022, 1, 1) - - def test_databases_without_metadata_are_ignored(self): - set_database_metadata("db_2022", representative_time="2022-01-01") - assert set(resolve_database_dates_from_metadata()) == {"db_2022"} - - def test_multi_scenario_database_is_skipped(self): - set_database_metadata("db_2022", representative_time="2022-01-01") - set_database_metadata( - "db_2024", - representative_time="2024-01-01", - scenarios=[ - {"pathway": "SSP2-Base", "representative_time": "2024-01-01"}, - {"pathway": "SSP2-PkBudg500", "representative_time": "2024-01-01"}, - ], - ) - assert set(resolve_database_dates_from_metadata()) == {"db_2022"} - - def test_invalid_metadata_value_raises_naming_the_database(self): - bd.databases["db_2022"]["representative_time"] = "whenever" - bd.databases.flush() - with pytest.raises(ValueError, match="db_2022"): - resolve_database_dates_from_metadata() -``` - -- [ ] **Step 2: Run the tests to verify they fail** - -Run: `.venv/bin/pytest tests/test_database_metadata.py::TestResolveFromMetadata -v` -Expected: FAIL — `ImportError: cannot import name 'resolve_database_dates_from_metadata'` - -- [ ] **Step 3: Implement discovery** - -Append to `bw_timex/database_metadata.py` (and add `from loguru import logger` to the imports): - -```python -def _candidate_databases() -> dict[str, dict]: - """Registered databases that declare a `representative_time`. - - Multi-scenario databases (superstructure and scenario-array exports, which - carry a `scenarios` list) are skipped: `bw_timex` needs one technosphere - per point in time and cannot pick a scenario out of such a database. They - can still be used by naming them in `database_dates`. - """ - candidates = {} - for name in bd.databases: - metadata = bd.databases[name] - if REPRESENTATIVE_TIME not in metadata: - continue - if metadata.get(SCENARIOS): - logger.info( - f"Skipping database '{name}': it holds " - f"{len(metadata[SCENARIOS])} scenarios, so the point in time it " - f"represents is ambiguous. Map it explicitly with `database_dates` " - f"if you want to use it anyway." - ) - continue - candidates[name] = metadata - return candidates - - -def resolve_database_dates_from_metadata( - scenario: dict | None = None, -) -> dict[str, datetime | str]: - """ - Map the databases of the current project to the points in time they represent. - - Reads the `representative_time` metadata of every registered database (see - [`set_database_metadata`][bw_timex.database_metadata.set_database_metadata]). - - Parameters - ---------- - scenario : dict, optional - Metadata a database must match to be included, e.g. - `{"iam_model": "remind", "pathway": "SSP2-PkBudg500"}`. Databases that - don't declare a filtered key at all are kept. - - Returns - ------- - dict - Mapping of database name to `datetime` or `"dynamic"`, ready to be used - as `TimexLCA.database_dates`. - """ - candidates = _candidate_databases() - return { - name: _normalize_representative_time(metadata[REPRESENTATIVE_TIME], name) - for name, metadata in candidates.items() - } -``` - -- [ ] **Step 4: Run the tests to verify they pass** - -Run: `.venv/bin/pytest tests/test_database_metadata.py -v` -Expected: PASS (18 tests) - -- [ ] **Step 5: Commit** - -```bash -git add bw_timex/database_metadata.py tests/test_database_metadata.py -git commit -m "feat: resolve database dates from representative_time metadata" -``` - ---- - -### Task 3: Scenario filtering and the ambiguity error - -**Files:** -- Modify: `bw_timex/database_metadata.py` -- Modify: `tests/test_database_metadata.py` - -**Interfaces:** -- Consumes: `resolve_database_dates_from_metadata`, `_candidate_databases`, `SCENARIO_SIGNATURE_KEYS`, `BRIGHTWAY_METADATA_KEYS` from Tasks 1–2. -- Produces: `resolve_database_dates_from_metadata(scenario)` now filters, and raises `ValueError` on an unknown filter key or on several scenario sets. No new public names. - -- [ ] **Step 1: Write the failing tests** - -Append to `tests/test_database_metadata.py`: - -```python -# ─── Tests for scenario selection ─── - - -@pytest.mark.usefixtures("temporal_grouping_db_monthly") -class TestScenarioSelection: - - @pytest.fixture(autouse=True) - def two_scenarios(self): - """db_2022 and db_2024 hold the same year in two different pathways.""" - set_database_metadata( - "db_2022", - representative_time="2022-01-01", - iam_model="remind", - pathway="SSP2-PkBudg500", - premise_version="2.4.9.1", - ) - set_database_metadata( - "db_2024", - representative_time="2024-01-01", - iam_model="remind", - pathway="SSP2-Base", - premise_version="2.4.9.1", - ) - - def test_two_scenario_sets_without_selection_raises(self): - with pytest.raises(ValueError, match="Several background scenarios"): - resolve_database_dates_from_metadata() - - def test_error_names_the_differing_key_and_values(self): - with pytest.raises(ValueError) as excinfo: - resolve_database_dates_from_metadata() - message = str(excinfo.value) - assert "pathway" in message - assert "SSP2-PkBudg500" in message - assert "SSP2-Base" in message - # iam_model is identical in both sets, so it isn't part of the report - assert "iam_model" not in message - - def test_scenario_selects_one_set(self): - resolved = resolve_database_dates_from_metadata( - scenario={"pathway": "SSP2-Base"} - ) - assert resolved == {"db_2024": datetime(2024, 1, 1)} - - def test_databases_without_scenario_metadata_survive_the_filter(self): - set_database_metadata("foreground", representative_time="dynamic") - resolved = resolve_database_dates_from_metadata( - scenario={"pathway": "SSP2-Base"} - ) - assert resolved == { - "db_2024": datetime(2024, 1, 1), - "foreground": "dynamic", - } - - def test_several_filter_keys_are_combined(self): - resolved = resolve_database_dates_from_metadata( - scenario={"iam_model": "remind", "pathway": "SSP2-Base"} - ) - assert set(resolved) == {"db_2024"} - - def test_filter_matching_nothing_resolves_to_nothing(self): - assert resolve_database_dates_from_metadata( - scenario={"pathway": "SSP2-PkBudg1150"} - ) == {} - - def test_unknown_filter_key_raises_listing_available_keys(self): - with pytest.raises(ValueError) as excinfo: - resolve_database_dates_from_metadata(scenario={"pathwya": "SSP2-Base"}) - message = str(excinfo.value) - assert "pathwya" in message - assert "pathway" in message - - def test_same_scenario_from_two_premise_versions_is_not_ambiguous(self): - set_database_metadata("db_2024", pathway="SSP2-PkBudg500") - set_database_metadata("db_2024", premise_version="2.4.9.2") - assert set(resolve_database_dates_from_metadata()) == {"db_2022", "db_2024"} -``` - -- [ ] **Step 2: Run the tests to verify they fail** - -Run: `.venv/bin/pytest tests/test_database_metadata.py::TestScenarioSelection -v` -Expected: FAIL — no error is raised, `resolve_database_dates_from_metadata` currently ignores `scenario` - -- [ ] **Step 3: Implement filtering and the ambiguity check** - -In `bw_timex/database_metadata.py`, add `from collections import defaultdict` to the imports and insert before `resolve_database_dates_from_metadata`: - -```python -def _as_set(value: Any) -> set: - """Compare list-valued metadata (e.g. `external_scenarios`) order-insensitively.""" - if isinstance(value, (list, tuple, set)): - return {str(item) for item in value} - return {str(value)} - - -def _values_match(declared: Any, wanted: Any) -> bool: - if isinstance(declared, (list, tuple, set)) or isinstance(wanted, (list, tuple, set)): - return _as_set(declared) == _as_set(wanted) - return str(declared) == str(wanted) - - -def _check_filter_keys(scenario: dict, candidates: dict[str, dict]) -> None: - """Reject filter keys no database declares, instead of silently matching nothing.""" - declared = set() - for metadata in candidates.values(): - declared.update(set(metadata) - BRIGHTWAY_METADATA_KEYS) - unknown = sorted(set(scenario) - declared) - if not unknown: - return - available = ", ".join(sorted(declared)) or "none" - raise ValueError( - f"No database in this project declares the metadata key(s) " - f"{unknown}. Keys declared by the databases of this project: {available}. " - f"Add the metadata with `bw_timex.set_database_metadata`, or check the " - f"spelling of your `scenario` filter." - ) - - -def _scenario_signature(metadata: dict) -> tuple: - return tuple( - (key, tuple(sorted(_as_set(metadata[key]))) if key in metadata else None) - for key in SCENARIO_SIGNATURE_KEYS - ) - - -def _format_scenario_sets(groups: dict[tuple, list[str]]) -> str: - """One line per scenario set, naming only the keys that actually differ.""" - differing = [ - key - for index, key in enumerate(SCENARIO_SIGNATURE_KEYS) - if len({signature[index][1] for signature in groups}) > 1 - ] - lines = [] - for signature, names in groups.items(): - values = dict(signature) - description = ", ".join( - f"{key}={', '.join(values[key]) if values[key] else 'not set'}" - for key in differing - ) - lines.append(f" {description}: {', '.join(sorted(names))}") - return "\n".join(lines) - - -def _check_unambiguous(candidates: dict[str, dict]) -> None: - groups = defaultdict(list) - for name, metadata in candidates.items(): - if any(key in metadata for key in SCENARIO_SIGNATURE_KEYS): - groups[_scenario_signature(metadata)].append(name) - if len(groups) <= 1: - return - raise ValueError( - f"Several background scenarios found in this project:\n" - f"{_format_scenario_sets(groups)}\n" - f"Select one, e.g. scenario={{'pathway': '...'}}, or map the databases " - f"explicitly with `database_dates`." - ) -``` - -Then replace the body of `resolve_database_dates_from_metadata` with: - -```python - candidates = _candidate_databases() - if scenario: - _check_filter_keys(scenario, candidates) - candidates = { - name: metadata - for name, metadata in candidates.items() - if all( - key not in metadata or _values_match(metadata[key], wanted) - for key, wanted in scenario.items() - ) - } - _check_unambiguous(candidates) - return { - name: _normalize_representative_time(metadata[REPRESENTATIVE_TIME], name) - for name, metadata in candidates.items() - } -``` - -- [ ] **Step 4: Run the tests to verify they pass** - -Run: `.venv/bin/pytest tests/test_database_metadata.py -v` -Expected: PASS (26 tests) - -- [ ] **Step 5: Commit** - -```bash -git add bw_timex/database_metadata.py tests/test_database_metadata.py -git commit -m "feat: select background scenarios by database metadata" -``` - ---- - -### Task 4: Wire it into `TimexLCA` - -**Files:** -- Modify: `bw_timex/timex_lca.py` (imports, class docstring `Examples` block, `__init__` signature + docstring, the `database_dates` fallback block at `timex_lca.py:146-160`) -- Modify: `bw_timex/validation.py` (`TimexLCAInputs`) -- Modify: `tests/test_database_metadata.py` - -**Interfaces:** -- Consumes: `resolve_database_dates_from_metadata(scenario)` from Task 3. -- Produces: `TimexLCA(demand, method, database_dates=None, scenario=None, use_global_lci_cache=True)`; `TimexLCA.scenario` holds the filter that was used; `TimexLCA.database_dates` is the resolved mapping, exactly as before for callers who pass `database_dates`. - -- [ ] **Step 1: Write the failing tests** - -Append to `tests/test_database_metadata.py`: - -```python -from bw_timex import TimexLCA - -# ─── Tests for TimexLCA using database metadata ─── - - -@pytest.mark.usefixtures("temporal_grouping_db_monthly") -class TestTimexLCAFromMetadata: - - @pytest.fixture - def fu(self): - return bd.get_node(database="foreground", code="A") - - def test_no_arguments_uses_metadata(self, fu): - set_database_metadata("db_2022", representative_time="2022-01-01") - set_database_metadata("db_2024", representative_time="2024-01-01") - tlca = TimexLCA(demand={fu.key: 1}, method=("GWP", "example")) - assert tlca.database_dates == { - "db_2022": datetime(2022, 1, 1), - "db_2024": datetime(2024, 1, 1), - "foreground": "dynamic", - } - - def test_demand_database_metadata_is_respected(self, fu): - set_database_metadata("db_2022", representative_time="2022-01-01") - set_database_metadata("foreground", representative_time="dynamic") - tlca = TimexLCA(demand={fu.key: 1}, method=("GWP", "example")) - assert tlca.database_dates["foreground"] == "dynamic" - - def test_scenario_is_forwarded(self, fu): - set_database_metadata( - "db_2022", representative_time="2022-01-01", pathway="SSP2-Base" - ) - set_database_metadata( - "db_2024", representative_time="2024-01-01", pathway="SSP2-PkBudg500" - ) - tlca = TimexLCA( - demand={fu.key: 1}, - method=("GWP", "example"), - scenario={"pathway": "SSP2-Base"}, - ) - assert tlca.database_dates == { - "db_2022": datetime(2022, 1, 1), - "foreground": "dynamic", - } - - def test_database_dates_is_exclusive(self, fu): - set_database_metadata("db_2022", representative_time="2022-01-01") - set_database_metadata("db_2024", representative_time="2024-01-01") - tlca = TimexLCA( - demand={fu.key: 1}, - method=("GWP", "example"), - database_dates={ - "db_2024": datetime(2024, 1, 1), - "foreground": "dynamic", - }, - ) - assert tlca.database_dates == { - "db_2024": datetime(2024, 1, 1), - "foreground": "dynamic", - } - - def test_database_dates_with_scenario_raises(self, fu): - with pytest.raises(ValueError, match="only applies when"): - TimexLCA( - demand={fu.key: 1}, - method=("GWP", "example"), - database_dates={"foreground": "dynamic"}, - scenario={"pathway": "SSP2-Base"}, - ) - - def test_no_metadata_anywhere_falls_back_to_dynamic_demand(self, fu): - tlca = TimexLCA(demand={fu.key: 1}, method=("GWP", "example")) - assert tlca.database_dates == {"foreground": "dynamic"} - - def test_metadata_and_database_dates_give_the_same_score(self, fu): - explicit = TimexLCA( - demand={fu.key: 1}, - method=("GWP", "example"), - database_dates={ - "db_2022": datetime(2022, 1, 1), - "db_2024": datetime(2024, 1, 1), - "foreground": "dynamic", - }, - ) - explicit.build_timeline(starting_datetime=datetime(2024, 1, 2)) - explicit.lci() - explicit.static_lcia() - - set_database_metadata("db_2022", representative_time="2022-01-01") - set_database_metadata("db_2024", representative_time="2024-01-01") - from_metadata = TimexLCA(demand={fu.key: 1}, method=("GWP", "example")) - from_metadata.build_timeline(starting_datetime=datetime(2024, 1, 2)) - from_metadata.lci() - from_metadata.static_lcia() - - assert from_metadata.static_score == pytest.approx(explicit.static_score) -``` - -- [ ] **Step 2: Run the tests to verify they fail** - -Run: `.venv/bin/pytest tests/test_database_metadata.py::TestTimexLCAFromMetadata -v` -Expected: FAIL — `TypeError: TimexLCA.__init__() got an unexpected keyword argument 'scenario'`, and `test_no_arguments_uses_metadata` fails because only the demand database is mapped - -- [ ] **Step 3: Change the signature and resolution** - -In `bw_timex/timex_lca.py`, add to the imports: - -```python -from .database_metadata import resolve_database_dates_from_metadata -``` - -Change the signature: - -```python - def __init__( - self, - demand: dict, - method: tuple, - database_dates: dict = None, - scenario: dict = None, - use_global_lci_cache: bool = True, - ) -> None: -``` - -Replace the `database_dates` docstring entry and add one for `scenario`: - -``` - database_dates : dict, optional - Dictionary mapping database names to the point in time they - represent, as a `datetime`, or to `"dynamic"` for databases whose - processes are distributed over time (typically the foreground). - Several databases may share the same date, e.g. to keep your own - modified copies of background processes in their own database - instead of writing them into the shared background database for - that vintage. If not given, the mapping is read from the - databases' own `representative_time` metadata (which premise - writes when exporting, and which you can set yourself with - `bw_timex.set_database_metadata`). Passing this argument replaces - the metadata entirely: only the databases listed here are used. - scenario : dict, optional - Metadata a background database must match to be used, e.g. - `{"iam_model": "remind", "pathway": "SSP2-PkBudg500"}`. Only - needed when the project holds several scenarios - `TimexLCA` - raises and lists them otherwise. Databases that don't declare the - filtered key (your foreground, a hand-built vintage) are always - kept. Cannot be combined with `database_dates`. -``` - -Replace the fallback block (`self.database_dates = database_dates` through the `if not self.database_dates:` block) with: - -```python - self.scenario = scenario - self.database_dates = self._resolve_database_dates( - demand=demand, database_dates=database_dates, scenario=scenario - ) -``` - -Add the method right after `__init__`: - -```python - @staticmethod - def _resolve_database_dates( - demand: dict, database_dates: dict | None, scenario: dict | None - ) -> dict: - """Map databases to the points in time they represent. - - Either from the explicit `database_dates` argument, which is then the - whole mapping, or from the databases' own `representative_time` - metadata. Databases holding the demand default to `"dynamic"`. - """ - if database_dates: - if scenario: - raise ValueError( - "`scenario` selects background databases by their metadata and " - "only applies when `database_dates` is not given. Pass one or " - "the other." - ) - return dict(database_dates) - - resolved = resolve_database_dates_from_metadata(scenario) - - if not resolved: - logger.info( - "No database_dates provided, and no database in this project carries " - "`representative_time` metadata. Treating the databases containing the " - "functional unit as dynamic. No remapping of inventories to time " - "explicit databases will be done." - ) - - for key in demand: - database = bd.get_node(id=get_id(key))["database"] - resolved.setdefault(database, "dynamic") - - return resolved -``` - -- [ ] **Step 4: Accept `scenario` in the input validation** - -In `bw_timex/validation.py`, add the field and its validator to `TimexLCAInputs`: - -```python - scenario: Optional[dict] = None -``` - -```python - @field_validator("scenario") - @classmethod - def validate_scenario(cls, v: Optional[dict]) -> Optional[dict]: - if v is None: - return v - if not v: - raise ValueError("scenario must be a non-empty dictionary if provided.") - for key, value in v.items(): - if not isinstance(key, str): - raise ValueError( - f"scenario keys must be strings (database metadata keys), got " - f"{type(key).__name__}." - ) - if not isinstance(value, (str, int, float, bool, list, tuple)): - raise ValueError( - f"scenario values must be scalars or lists of scalars, got " - f"{type(value).__name__} for key '{key}'." - ) - return v -``` - -And pass it in `timex_lca.py`, where `TimexLCAInputs` is instantiated: - -```python - TimexLCAInputs( - demand=self.demand, - method=self.method, - database_dates=self.database_dates, - scenario=self.scenario, - ) -``` - -- [ ] **Step 5: Update the class docstring example** - -In the `Examples` block of the `TimexLCA` class docstring, put the metadata path first and keep the explicit mapping as the alternative: - -```python - from bw_timex import TimexLCA, set_database_metadata - - demand = {("my_foreground_database", "my_process"): 1} - method = ("some_method_family", "some_category", "some_method") - - # Databases exported by premise already know the point in time they - # represent. For your own databases, say so once: - set_database_metadata("my_background_database_one", representative_time=datetime(2020, 1, 1)) - set_database_metadata("my_background_database_two", representative_time=datetime(2030, 1, 1)) - - tlca = TimexLCA(demand, method) - - # ... or map the databases explicitly, which then replaces the metadata: - tlca = TimexLCA( - demand, - method, - database_dates={ - "my_background_database_one": datetime(2020, 1, 1), - "my_background_database_two": datetime(2030, 1, 1), - # Several databases may share the same date, e.g. to keep your own - # modified copies of background processes in their own database: - "my_modified_background_2020": datetime(2020, 1, 1), - "my_foreground_database": "dynamic", - }, - ) - - tlca.build_timeline() # has many optional arguments - tlca.lci() - tlca.static_lcia() - print(tlca.static_score) - # also available: "GWP", "pGWP", "pGTP", "prospective_radiative_forcing" - tlca.dynamic_lcia(metric="radiative_forcing") - print(tlca.dynamic_score) -``` - -- [ ] **Step 6: Run the new tests** - -Run: `.venv/bin/pytest tests/test_database_metadata.py -v` -Expected: PASS (33 tests) - -- [ ] **Step 7: Run the whole suite to prove nothing regressed** - -Run: `.venv/bin/pytest -x -q` -Expected: PASS, same count as on `main` plus the new tests. Every existing test passes `database_dates`, so the exclusive branch must keep them green. - -- [ ] **Step 8: Commit** - -```bash -git add bw_timex/timex_lca.py bw_timex/validation.py tests/test_database_metadata.py -git commit -m "feat: read database timing from metadata by default in TimexLCA" -``` - ---- - -### Task 5: Documentation - -**Files:** -- Create: `docs/content/background_database_metadata.md` -- Create: `docs/api/database_metadata.md` -- Modify: `zensical.toml` (User Guide nav, API nav) -- Modify: `docs/content/getting_started/quickstart.md:58-70`, `:96`, `:128-132` -- Modify: `docs/content/getting_started/adding_temporal_information.md:250-282` -- Modify: `docs/content/getting_started/build_process_timeline.md:11-21` -- Modify: `CHANGES.md` - -**Interfaces:** -- Consumes: `set_database_metadata`, `TimexLCA(scenario=...)` from Tasks 1–4. -- Produces: no code. - -- [ ] **Step 1: Write the new reference page** - -Create `docs/content/background_database_metadata.md`: - -````markdown ---- -icon: lucide/calendar-clock -tags: - - background databases ---- - -# What a database represents - -`bw_timex` needs to know which point in time each background database stands for. -That information lives in the database's own Brightway metadata, so it only has to -be recorded once - not in every script. - -```python -import bw2data as bd - -bd.databases["ei_cutoff_3.10.1_remind_SSP2-PkBudg500_2050"] -``` - -```python -{ - # written by brightway - "format": "Ecoinvent XML", "backend": "sqlite", "number": 43648, ..., - # written by premise - "premise_version": "2.4.9.1", - "iam_model": "remind", - "pathway": "SSP2-PkBudg500", - "representative_time": "2050-01-01T00:00:00", - "ecoinvent_version": "3.10.1", - "system_model": "cutoff", -} -``` - -Only `representative_time` is required. `TimexLCA` reads it from every database of -your project, so a study on premise databases needs no timing argument at all: - -```python -tlca = TimexLCA(demand={("foreground", "A"): 1}, method=("our", "method")) -``` - -!!! info "premise version" - - premise writes this metadata from the version following 2.4.9.2 onwards. For - databases written by an earlier version, set it yourself as shown below - it is - a one-liner per database. - -## Setting it yourself - -For databases you built yourself, use -[`set_database_metadata`][bw_timex.database_metadata.set_database_metadata]: - -```python -from datetime import datetime -from bw_timex import set_database_metadata - -set_database_metadata("background_2020", representative_time=datetime(2020, 1, 1)) -set_database_metadata("background_2030", representative_time=datetime(2030, 1, 1)) -``` - -The value is stored as an ISO 8601 string, because Brightway keeps database -metadata as JSON. You only do this once per database: it is stored in the project, -not in your script. - -Your foreground doesn't represent a point in time - its processes get distributed -over time. `TimexLCA` treats the databases holding your functional unit as -`"dynamic"` automatically, but you can also say so explicitly: - -```python -set_database_metadata("foreground", representative_time="dynamic") -``` - -## Several databases for the same point in time - -More than one database may carry the same date. This is useful when you modify -background processes: keep the modified copies in your own database per point in -time, instead of writing them into ecoinvent or premise. - -```python -set_database_metadata("my_background_2020", representative_time=datetime(2020, 1, 1)) -set_database_metadata("my_background_2030", representative_time=datetime(2030, 1, 1)) -``` - -For each process, `bw_timex` interpolates only between the databases that actually -contain it, matched on `name`, `reference product` and `location`. - -## Choosing a scenario - -A project often holds more than one IAM scenario. `bw_timex` refuses to guess and -tells you what it found: - -``` -Several background scenarios found in this project: - pathway=SSP2-PkBudg500: ei_..._2030, ei_..._2040, ei_..._2050 - pathway=SSP2-Base: ei_..._2030, ei_..._2040, ei_..._2050 -Select one, e.g. scenario={'pathway': '...'}, or map the databases explicitly with -`database_dates`. -``` - -Pick one with the `scenario` argument, which filters the databases on their -metadata: - -```python -tlca = TimexLCA( - demand={("foreground", "A"): 1}, - method=("our", "method"), - scenario={"pathway": "SSP2-PkBudg500"}, -) -``` - -Any metadata key works - `iam_model`, `pathway`, `system_model`, -`ecoinvent_version`, `premise_version`, or anything you set yourself. Databases -that don't carry the key at all (your foreground, your own vintages) are never -filtered out. - -Comparing scenarios is then a loop over filters: - -```python -scores = {} -for pathway in ("SSP2-Base", "SSP2-PkBudg500"): - tlca = TimexLCA(demand, method, scenario={"pathway": pathway}) - tlca.build_timeline() - tlca.lci() - tlca.static_lcia() - scores[pathway] = tlca.static_score -``` - -!!! warning "Superstructure databases" - - Databases holding several scenarios at once (premise superstructure or - scenario-array exports) are skipped: they have no single technosphere per point - in time. Use one database per scenario and year. - -## Mapping the databases explicitly - -`database_dates` still does what it always did, and takes over completely: when you -pass it, metadata is not read at all and only the databases you list are used. - -```python -tlca = TimexLCA( - demand={("foreground", "A"): 1}, - method=("our", "method"), - database_dates={ - "background": datetime(2020, 1, 1), - "background_2030": datetime(2030, 1, 1), - "foreground": "dynamic", - }, -) -``` - -Use it when you want to restrict a calculation to a subset of the databases in your -project, or when a database's metadata is wrong and you don't want to change it. -```` - -- [ ] **Step 2: Add both pages to the nav** - -In `zensical.toml`, add the User Guide entry after the Walkthrough block (after the line `]},` that closes `Walkthrough`, before `{ "What LCA should I do?" ...`): - -```toml - { "What a database represents" = "content/background_database_metadata.md" }, -``` - -And in the API nav block, next to the other API pages: - -```toml - { "Database metadata" = "api/database_metadata.md" }, -``` - -Create `docs/api/database_metadata.md`, following `docs/api/utils.md`: - -```markdown ---- -icon: lucide/calendar-clock -tags: - - api ---- - -# Database metadata - -Reading and writing what a Brightway database represents: the point in time -(`representative_time`) and, for prospective databases, the scenario it was built -for. - -::: bw_timex.database_metadata -``` - -- [ ] **Step 3: Update the quickstart** - -In `docs/content/getting_started/quickstart.md`, replace step 3 and the `TimexLCA` call: - -```python -# 3. Say what your time-specific background databases represent -# (premise databases already know - skip this for them) -set_database_metadata("background", representative_time=datetime(2020, 1, 1)) -set_database_metadata("background_2030", representative_time=datetime(2030, 1, 1)) - -# 4. Create the TimexLCA object -tlca = TimexLCA( - demand={("foreground", "A"): 1}, - method=("our", "method"), -) -``` - -Add `set_database_metadata` to the `from bw_timex import ...` line at the top of that -code block. In the cheat sheet, change the background row to: - -``` -| *How the background changes* over time | one database per point in time, each with `representative_time` metadata | background databases | -``` - -And replace the trailing paragraph about `database_dates` (lines 128-132) with: - -```markdown -Absolute dates (`dtype="datetime64[s]"`) are also allowed in a `TemporalDistribution`, -e.g. for the timing of the functional unit itself. Relative dates -(`dtype="timedelta64[Y]"`) are relative to the consuming process. Several databases may -represent the same point in time, e.g. if you keep modified copies of background -processes in their own database instead of writing them into the shared vintage. See -[Time-specific background databases](../background_database_metadata.md) for scenario selection -and for mapping databases explicitly with `database_dates`. -``` - -- [ ] **Step 4: Update walkthrough step 1** - -In `docs/content/getting_started/adding_temporal_information.md`, replace the paragraph and code block at lines 250-262 with: - -````markdown -So, as you can see, the processes at specific time steps reside within a separate normal -Brightway database. `bw_timex` picks these up automatically, as long as each database -says which point in time it represents: - -```python -from datetime import datetime -from bw_timex import set_database_metadata - -set_database_metadata("background", representative_time=datetime(2020, 1, 1)) -set_database_metadata("background_2030", representative_time=datetime(2030, 1, 1)) -``` - -You only do this once per database - it is stored in your Brightway project. Databases -exported by [premise](https://premise.readthedocs.io/en/latest/introduction.html) bring -this metadata with them, so there is nothing to do for those. The foreground doesn't -represent a specific point in time and is distributed over time instead; `bw_timex` -treats the databases holding your functional unit that way automatically. -```` - -Replace the code block in the "Several databases for the same point in time" section -(lines 274-282) with: - -```python -set_database_metadata("ecoinvent_2020", representative_time=datetime(2020, 1, 1)) -set_database_metadata("ecoinvent_2030", representative_time=datetime(2030, 1, 1)) -set_database_metadata("my_background_2020", representative_time=datetime(2020, 1, 1)) -set_database_metadata("my_background_2030", representative_time=datetime(2030, 1, 1)) -``` - -- [ ] **Step 5: Update walkthrough step 2** - -In `docs/content/getting_started/build_process_timeline.md`, replace lines 11-21 with: - -````markdown -With all the temporal information prepared, we can now instantiate our TimexLCA object. -This is just like a normal Brightway LCA object - the timing of the background databases -comes from their metadata: - -```python -from bw_timex import TimexLCA - -tlca = TimexLCA( - demand={("foreground", "A"): 1}, - method=("our", "method"), -) -``` - -If your project holds several scenarios, select one with -`scenario={"pathway": "SSP2-PkBudg500"}`; to map the databases by hand instead, pass -`database_dates`. Both are covered in -[Time-specific background databases](../background_database_metadata.md). -```` - -- [ ] **Step 6: Add the changelog entry** - -Under `## [Unreleased]` in `CHANGES.md`: - -```markdown -* Added `representative_time` database metadata as the default timing source: `TimexLCA` now maps background databases to points in time by reading their Brightway metadata (as written by premise), making `database_dates` optional ([#217](https://github.com/brightway-lca/bw_timex/issues/217)) -* Added `set_database_metadata` to record Time-specific background databases (`representative_time`, and scenario fields such as `iam_model` or `pathway`) for databases that don't bring the metadata themselves -* Added `TimexLCA(scenario={...})` to select one background scenario when a project holds several; `TimexLCA` raises and lists the scenarios it found if the choice is ambiguous -``` - -- [ ] **Step 7: Verify the docs build** - -Run: `.venv/bin/python -m zensical build 2>&1 | tail -20` -Expected: build succeeds, no warning about `background_database_metadata.md` or `api/database_metadata.md` being missing from the nav. If `zensical` is not installed in the venv, run `.venv/bin/python -c "import tomllib, pathlib; tomllib.loads(pathlib.Path('zensical.toml').read_text())"` to at least prove the nav edit is valid TOML, and say in the commit that the build was not run. - -- [ ] **Step 8: Commit** - -```bash -git add docs zensical.toml CHANGES.md -git commit -m "docs: document representative_time database metadata" -``` - ---- - -### Task 6: Tutorial notebooks - -**Files:** -- Modify: `notebooks/tutorials/1_getting_started.ipynb` -- Modify: `notebooks/tutorials/2_electric_vehicle_from_scratch.ipynb` -- Modify: `notebooks/tutorials/3_dynamic_characterization.ipynb` -- Modify: `notebooks/tutorials/4_import_model_from_excel.ipynb` - -**Interfaces:** -- Consumes: `set_database_metadata`, `TimexLCA()` without `database_dates` from Tasks 1–4. -- Produces: no code. - -These notebooks build their own small databases, so they can be re-executed. - -- [ ] **Step 1: Find every occurrence** - -Run: `grep -n "database_dates" notebooks/tutorials/*.ipynb` -Note which cells build the mapping and which pass it to `TimexLCA`. - -- [ ] **Step 2: Edit the cells** - -In each notebook, use `NotebookEdit` to: -1. Replace the cell that builds `database_dates` with `set_database_metadata` calls, one per background database, keeping the surrounding markdown explanation in sync (it must no longer say "we define a dictionary that maps databases to dates"). -2. Drop the `database_dates=database_dates` argument from the `TimexLCA(...)` call. -3. Add `set_database_metadata` to the `from bw_timex import ...` cell. - -Pattern: - -```python -# before -database_dates = { - "db_2020": datetime.strptime("2020", "%Y"), - "db_2030": datetime.strptime("2030", "%Y"), - "foreground": "dynamic", -} -tlca = TimexLCA(demand={fu.key: 1}, method=method, database_dates=database_dates) - -# after -set_database_metadata("db_2020", representative_time=datetime(2020, 1, 1)) -set_database_metadata("db_2030", representative_time=datetime(2030, 1, 1)) -tlca = TimexLCA(demand={fu.key: 1}, method=method) -``` - -- [ ] **Step 3: Re-execute each notebook** - -Run, one notebook at a time: - -```bash -.venv/bin/jupyter nbconvert --to notebook --execute --inplace notebooks/tutorials/1_getting_started.ipynb -``` - -Expected: completes without error. If a notebook needs data that isn't in the repo, do -not execute it — leave the stored outputs, and note that in the commit message. - -- [ ] **Step 4: Check the diff for accidental churn** - -Run: `git diff --stat notebooks/tutorials` -Expected: only the edited cells plus their re-executed outputs. If execution rewrote -every cell id or bumped unrelated metadata, restore and re-run with -`--ClearMetadataPreprocessor.enabled=True` off, keeping the diff readable. - -- [ ] **Step 5: Commit** - -```bash -git add notebooks/tutorials -git commit -m "docs: use database metadata instead of database_dates in the tutorials" -``` - ---- - -### Task 7: Remaining notebooks - -**Files:** -- Modify: `notebooks/advanced/background_temporal_distributions.ipynb` -- Modify: `notebooks/advanced/uncertainty_with_datapackages.ipynb` -- Modify: `notebooks/advanced/background_temporal_distributions_premise.ipynb` -- Modify: `notebooks/teaching/ev_walkthrough_premise.ipynb` -- Modify: `notebooks/teaching/exercise_ev_vs_petrol_solutions.ipynb` -- Modify: `notebooks/examples/electric_vehicle_premise.ipynb` -- Modify: `notebooks/examples/electric_vehicle_premise_detailed.ipynb` -- Modify: `notebooks/development/benchmarking.ipynb` -- **Do not touch:** `notebooks/examples/paper_case_study.ipynb` - -**Interfaces:** -- Consumes: `set_database_metadata`, `TimexLCA(scenario=...)` from Tasks 1–4. -- Produces: no code. - -The first two build their own databases and can be re-executed. The rest need premise -or ecoinvent databases that aren't in the repo: edit the cell sources only and leave -the stored outputs alone. - -- [ ] **Step 1: Edit the two self-contained notebooks** - -`background_temporal_distributions.ipynb` and `uncertainty_with_datapackages.ipynb`: -same replacement as Task 6 Step 2, then re-execute: - -```bash -.venv/bin/jupyter nbconvert --to notebook --execute --inplace notebooks/advanced/background_temporal_distributions.ipynb -.venv/bin/jupyter nbconvert --to notebook --execute --inplace notebooks/advanced/uncertainty_with_datapackages.ipynb -``` - -- [ ] **Step 2: Edit the premise/ecoinvent notebooks** - -For each of the six remaining notebooks, replace the `database_dates` cell. These use -premise databases, which carry the metadata already, so the mapping usually -disappears entirely: - -```python -# before -database_dates = { - "ei312_REMIND-EU_SSP2_NDC_2020": datetime.strptime("2020", "%Y"), - "ei312_REMIND-EU_SSP2_NDC_2030": datetime.strptime("2030", "%Y"), - "foreground": "dynamic", -} -tlca = TimexLCA(demand={fu.key: 1}, method=method, database_dates=database_dates) - -# after -# The premise databases carry the point in time they represent in their -# metadata, so bw_timex finds them by itself. -tlca = TimexLCA(demand={fu.key: 1}, method=method) -``` - -Two things to get right per notebook: -- If the notebook creates its own modified copies of background processes in extra - databases (the electric-vehicle notebooks do, e.g. `..., without EOL` copies), those - copies need `set_database_metadata(..., representative_time=...)` with the same date - as the vintage they were copied from, or they drop out of the mapping. -- If the notebook's project could hold more than one pathway, show the `scenario` - argument in the markdown right below, e.g. - `scenario={"pathway": "SSP2-PkBudg500"}`. - -Update the surrounding markdown text wherever it explains `database_dates`. - -- [ ] **Step 3: Verify no notebook lost its outputs** - -Run: `git diff --stat notebooks` -Expected: for the six premise notebooks, only source cells change - no `outputs` churn. - -- [ ] **Step 4: Confirm the paper case study is untouched** - -Run: `git status --porcelain notebooks/examples/paper_case_study.ipynb` -Expected: no output. - -- [ ] **Step 5: Commit** - -```bash -git add notebooks -git commit -m "docs: use database metadata instead of database_dates in the notebooks" -``` - ---- - -### Task 8: Final verification - -**Files:** none - -- [ ] **Step 1: Full test suite** - -Run: `.venv/bin/pytest -q` -Expected: all pass. - -- [ ] **Step 2: Nothing still teaches the old default** - -Run: `grep -rn "database_dates" --include="*.md" --include="*.ipynb" docs notebooks | grep -v paper_case_study | grep -v superpowers` -Expected: only the places that deliberately document `database_dates` as the explicit -override — `background_database_metadata.md`, the quickstart's closing paragraph, and -step 2's pointer. Anything else is a leftover. - -- [ ] **Step 3: Public API test still describes the namespace** - -Run: `.venv/bin/pytest tests/test_public_api.py -v` -Expected: PASS. If it asserts an exact `__all__`, add `set_database_metadata` to it. - -- [ ] **Step 4: Commit any fixes and push the branch** - -```bash -git add -A -git commit -m "fix: address leftovers from the metadata migration" -git push -u origin feat/representative-time-metadata -``` diff --git a/docs/superpowers/plans/2026-08-23-premise-scenario-autocreation.md b/docs/superpowers/plans/2026-08-23-premise-scenario-autocreation.md deleted file mode 100644 index bc47dbea..00000000 --- a/docs/superpowers/plans/2026-08-23-premise-scenario-autocreation.md +++ /dev/null @@ -1,1523 +0,0 @@ -# Premise scenario auto-creation Implementation Plan - -> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. - -**Goal:** When `TimexLCA(scenario={...}, create_missing=True)` names background vintages the project does not have, build them with premise (importing ecoinvent first if needed) instead of raising. - -**Architecture:** A new module `bw_timex/scenario_builder.py` owns the whole "find or build" decision. It splits the `scenario` dict into *filter keys* (matched against database metadata, as today) and *build keys* (`years`, `sectors`, `source_database`), finds which years the project already satisfies, and builds only the rest. All premise/bw2io contact is confined to two logic-free seam functions that import their library inside the function body, so premise stays an optional extra and the rest of the module is testable without it. `TimexLCA.__init__` calls `ensure_scenario_databases` before `_resolve_database_dates`, then resolves dates from metadata exactly as it does today — one code path decides what a database represents. - -**Tech Stack:** Python ≥3.11, bw2data ≥4.6, pydantic ≥2, pytest, loguru, premise ≥2.4.9.2 (optional extra), bw2io (via premise's `bw25` extra). - -**Spec:** `docs/superpowers/specs/2026-08-23-premise-scenario-autocreation-design.md` - -## Global Constraints - -- Optional dependency, exact string: `premise = ["premise[bw25]>=2.4.9.2"]`. The `[bw25]` extra is mandatory — bare `premise` leaves `bw2data`/`bw2io` unpinned and can resolve to `bw2data==3.6.6`, contradicting bw_timex's `bw2data>=4.6`. -- `premise` and `bw2io` are imported **inside function bodies only**. No module-level import of either, anywhere in `bw_timex/`. -- Default behaviour is unchanged: `create_missing=False`. Without it, no premise import, no build, and the existing error text is preserved (one line added). -- Filter keys are exactly `("iam_model", "pathway", "system_model", "ecoinvent_version")`. Build keys are exactly `("years", "sectors", "source_database")`. -- Built database names: `ei_{system_model}_{ecoinvent_version}_{iam_model}_{pathway}_{year}`. -- Environment variable names, verbatim: `PREMISE_KEY`, `ECOINVENT_USERNAME`, `ECOINVENT_PASSWORD`. Explicit arguments always win over the environment. -- premise ≥ 2.4.9.2 writes `representative_time`, `iam_model`, `pathway`, `system_model`, `ecoinvent_version`, `premise_version` itself. Do **not** re-write those. Only `sectors` is written by us, and only when narrowed. -- No test may require premise, bw2io, a premise key, ecoinvent credentials, or network access. -- Existing code style: `loguru.logger` for user-facing progress, pydantic models in `bw_timex/validation.py` for argument validation, tests use `bw2data.tests.bw2test` fixtures from `tests/fixtures/`. - -## File Structure - -| File | Responsibility | -|---|---| -| `bw_timex/database_metadata.py` (modify) | Gains the scenario-key vocabulary (`SCENARIO_BUILD_KEYS`, `split_scenario`) and one reusable predicate (`database_matches_scenario`). Stays side-effect-free. | -| `bw_timex/scenario_builder.py` (create) | `ensure_scenario_databases` + credential resolution + the two premise/bw2io seams. The only module that knows premise exists. | -| `bw_timex/validation.py` (modify) | `ScenarioBuildInputs`; `TimexLCAInputs` gains the three new arguments and their combination rules. | -| `bw_timex/timex_lca.py` (modify) | Argument plumbing in `__init__` only, plus one added line in the "matched no database" error. | -| `bw_timex/__init__.py` (modify) | Re-export `ensure_scenario_databases`. | -| `tests/test_scenario_builder.py` (create) | Everything on this side of the seams, with the seams faked. | -| `tests/test_database_metadata.py` (modify) | Tests for the new metadata helpers. | -| `pyproject.toml`, `docs/`, `CHANGES.md` (modify) | Extra, documentation, changelog. Folded into the last task. | - ---- - -### Task 1: Scenario key vocabulary and the match predicate - -Build keys must be stripped before any metadata filtering, or `_check_filter_keys` raises "No database in this project declares the metadata key(s) ['years']" for every `create_missing` call. And the "is this year already satisfied?" question in Task 2 must use exactly the resolver's keep-rule, or it builds a duplicate vintage next to a database the resolver would have used. - -**Files:** -- Modify: `bw_timex/database_metadata.py` -- Test: `tests/test_database_metadata.py` - -**Interfaces:** -- Consumes: nothing (first task). -- Produces: - - `SCENARIO_FILTER_KEYS: tuple[str, ...]` = `("iam_model", "pathway", "system_model", "ecoinvent_version")` - - `SCENARIO_BUILD_KEYS: tuple[str, ...]` = `("years", "sectors", "source_database")` - - `split_scenario(scenario: dict | None) -> tuple[dict, dict]` returning `(filters, build)` - - `database_matches_scenario(metadata: dict, scenario: dict | None) -> bool` - - `resolve_database_dates_from_metadata(scenario)` now tolerates build keys in `scenario` - -- [ ] **Step 1: Write the failing tests** - -Append to `tests/test_database_metadata.py`: - -```python -# ─── Tests for the scenario key vocabulary ─── - - -class TestSplitScenario: - - def test_build_keys_are_separated(self): - from bw_timex.database_metadata import split_scenario - - filters, build = split_scenario( - { - "iam_model": "remind", - "pathway": "SSP2-PkBudg500", - "years": [2020, 2030], - "sectors": ["electricity"], - "source_database": "my_ecoinvent", - } - ) - assert filters == {"iam_model": "remind", "pathway": "SSP2-PkBudg500"} - assert build == { - "years": [2020, 2030], - "sectors": ["electricity"], - "source_database": "my_ecoinvent", - } - - def test_unknown_keys_are_treated_as_filters(self): - from bw_timex.database_metadata import split_scenario - - filters, build = split_scenario({"my_own_key": "value"}) - assert filters == {"my_own_key": "value"} - assert build == {} - - def test_none_scenario_gives_empty_dicts(self): - from bw_timex.database_metadata import split_scenario - - assert split_scenario(None) == ({}, {}) - - -class TestDatabaseMatchesScenario: - - def test_declared_and_equal_matches(self): - from bw_timex.database_metadata import database_matches_scenario - - assert database_matches_scenario({"pathway": "SSP2-PkBudg500"}, {"pathway": "SSP2-PkBudg500"}) - - def test_declared_and_different_does_not_match(self): - from bw_timex.database_metadata import database_matches_scenario - - assert not database_matches_scenario({"pathway": "SSP2-Base"}, {"pathway": "SSP2-PkBudg500"}) - - def test_undeclared_key_still_matches(self): - from bw_timex.database_metadata import database_matches_scenario - - assert database_matches_scenario({"representative_time": "2020-01-01"}, {"pathway": "SSP2-PkBudg500"}) - - def test_list_values_compare_order_insensitively(self): - from bw_timex.database_metadata import database_matches_scenario - - assert database_matches_scenario( - {"external_scenarios": ["b", "a"]}, {"external_scenarios": ["a", "b"]} - ) - - def test_empty_scenario_matches_everything(self): - from bw_timex.database_metadata import database_matches_scenario - - assert database_matches_scenario({"pathway": "SSP2-Base"}, None) - - -@pytest.mark.usefixtures("temporal_grouping_db_monthly") -class TestResolverIgnoresBuildKeys: - - def test_build_keys_do_not_reach_the_filter(self): - set_database_metadata( - "db_2022", - representative_time=datetime(2022, 1, 1), - pathway="SSP2-PkBudg500", - ) - resolved = resolve_database_dates_from_metadata( - {"pathway": "SSP2-PkBudg500", "years": [2022], "sectors": ["electricity"]} - ) - assert resolved == {"db_2022": datetime(2022, 1, 1)} -``` - -- [ ] **Step 2: Run the tests to verify they fail** - -Run: `pytest tests/test_database_metadata.py -k "SplitScenario or DatabaseMatchesScenario or ResolverIgnoresBuildKeys" -v` -Expected: FAIL with `ImportError: cannot import name 'split_scenario'` (and, for the last test, `ValueError: No database in this project declares the metadata key(s) ['sectors', 'years']`). - -- [ ] **Step 3: Implement** - -In `bw_timex/database_metadata.py`, below the existing `SCENARIO_SIGNATURE_KEYS` block, add: - -```python -#: Metadata keys premise writes that identify which scenario a database belongs -#: to, and that `TimexLCA(scenario=...)` filters on. -SCENARIO_FILTER_KEYS = ( - "iam_model", - "pathway", - "system_model", - "ecoinvent_version", -) - -#: Keys of a `scenario` mapping that describe how to *build* a missing vintage -#: rather than what to match. They never reach the metadata filter: `years` is a -#: list, and no database's metadata could ever equal it. -SCENARIO_BUILD_KEYS = ("years", "sectors", "source_database") - - -def split_scenario(scenario: dict | None) -> tuple[dict, dict]: - """Separate a `scenario` mapping into its filter keys and its build keys.""" - if not scenario: - return {}, {} - filters = {k: v for k, v in scenario.items() if k not in SCENARIO_BUILD_KEYS} - build = {k: v for k, v in scenario.items() if k in SCENARIO_BUILD_KEYS} - return filters, build - - -def database_matches_scenario(metadata: dict, scenario: dict | None) -> bool: - """Whether a database's metadata survives a `scenario` filter. - - A database is kept unless it *declares* a filtered key with a different - value: a hand-built vintage or a foreground carrying no scenario metadata - belongs to every scenario, not to none. - """ - if not scenario: - return True - return all( - key not in metadata or _values_match(metadata[key], wanted) - for key, wanted in scenario.items() - ) -``` - -Then rewrite the filtering block of `resolve_database_dates_from_metadata` to strip build keys and use the predicate: - -```python - candidates = _candidate_databases() - scenario, _ = split_scenario(scenario) - if scenario: - _check_filter_keys(scenario, candidates) - candidates = { - name: metadata - for name, metadata in candidates.items() - if database_matches_scenario(metadata, scenario) - } -``` - -- [ ] **Step 4: Run the tests to verify they pass** - -Run: `pytest tests/test_database_metadata.py -v` -Expected: PASS, including every pre-existing test in the file (the refactor must not change resolver behaviour). - -- [ ] **Step 5: Run the whole suite** - -Run: `pytest -q` -Expected: PASS, same count as before plus the new tests. - -- [ ] **Step 6: Commit** - -```bash -git add bw_timex/database_metadata.py tests/test_database_metadata.py -git commit -m "feat: separate scenario build keys from filter keys" -``` - ---- - -### Task 2: Find what exists, build nothing - -The steady-state path: a study re-run where every vintage is already there. It must not import premise, and it must be the same match rule the resolver uses. - -**Files:** -- Create: `bw_timex/scenario_builder.py` -- Test: `tests/test_scenario_builder.py` - -**Interfaces:** -- Consumes: `split_scenario`, `database_matches_scenario`, `_normalize_representative_time`, `REPRESENTATIVE_TIME`, `SCENARIOS`, `DYNAMIC` from `bw_timex.database_metadata`. -- Produces: - - `find_existing_vintages(filters: dict) -> dict[int, str]` — year → database name - - `ensure_scenario_databases(scenario: dict, premise_key: str | None = None, ecoinvent_credentials: tuple[str, str] | None = None) -> dict[str, datetime]` - - `_run_premise(...)` and `_import_ecoinvent(...)`, the seams later tasks fill in and every test monkeypatches - -- [ ] **Step 1: Write the failing tests** - -Create `tests/test_scenario_builder.py`: - -```python -"""Tests for finding or building the background vintages a scenario names. - -premise is never installed, never called and never imported here: the two -functions that touch it are monkeypatched with fakes that register small -Brightway databases carrying premise-style metadata. -""" - -import sys -from datetime import datetime - -import bw2data as bd -import pytest - -from bw_timex import set_database_metadata -from bw_timex.scenario_builder import ensure_scenario_databases, find_existing_vintages - -SCENARIO = { - "iam_model": "remind", - "pathway": "SSP2-PkBudg500", - "system_model": "cutoff", - "ecoinvent_version": "3.10.1", -} - - -def write_minimal_database(name): - """A one-process database, enough to be registered and carry metadata.""" - bd.Database(name).write( - { - (name, "C"): { - "name": "C", - "location": "somewhere", - "reference product": "C", - "exchanges": [ - {"amount": 1, "type": "production", "input": (name, "C")}, - ], - }, - }, - ) - - -def write_vintage(name, year, **extra): - """A database that looks like a premise export for `year`.""" - write_minimal_database(name) - set_database_metadata( - name, - representative_time=datetime(year, 1, 1), - premise_version="2.4.9.2", - **{**SCENARIO, **extra}, - ) - - -@pytest.fixture -def fake_premise(monkeypatch): - """Records calls to the premise seam and writes what premise would write.""" - calls = [] - - def fake_run_premise(**kwargs): - calls.append(kwargs) - years = [scenario["year"] for scenario in kwargs["scenarios"]] - for name, year in zip(kwargs["names"], years): - write_vintage(name, year) - - monkeypatch.setattr( - "bw_timex.scenario_builder._run_premise", fake_run_premise, raising=True - ) - return calls - - -@pytest.mark.usefixtures("temporal_grouping_db_monthly") -class TestFindExistingVintages: - - def test_matching_vintage_is_found_by_year(self): - write_vintage("ei_2030", 2030) - assert find_existing_vintages(SCENARIO) == {2030: "ei_2030"} - - def test_other_scenario_is_not_found(self): - write_vintage("ei_2030", 2030, pathway="SSP2-Base") - assert find_existing_vintages(SCENARIO) == {} - - def test_database_without_scenario_metadata_satisfies_its_year(self): - write_minimal_database("hand_built_2020") - set_database_metadata("hand_built_2020", representative_time=datetime(2020, 1, 1)) - assert find_existing_vintages(SCENARIO) == {2020: "hand_built_2020"} - - def test_dynamic_databases_are_ignored(self): - set_database_metadata("foreground", representative_time="dynamic") - assert find_existing_vintages(SCENARIO) == {} - - def test_multi_scenario_databases_are_ignored(self): - write_vintage("superstructure", 2030) - bd.databases["superstructure"]["scenarios"] = [{"year": 2030}, {"year": 2040}] - bd.databases.flush() - assert find_existing_vintages(SCENARIO) == {} - - -@pytest.mark.usefixtures("temporal_grouping_db_monthly") -class TestNothingToBuild: - - def test_all_years_present_builds_nothing(self, fake_premise): - write_vintage("ei_2030", 2030) - write_vintage("ei_2040", 2040) - ensure_scenario_databases({**SCENARIO, "years": [2030, 2040]}) - assert fake_premise == [] - - def test_all_years_present_returns_the_mapping(self, fake_premise): - write_vintage("ei_2030", 2030) - result = ensure_scenario_databases({**SCENARIO, "years": [2030]}) - assert result == {"ei_2030": datetime(2030, 1, 1)} - - def test_premise_is_not_imported(self, fake_premise, monkeypatch): - monkeypatch.delitem(sys.modules, "premise", raising=False) - write_vintage("ei_2030", 2030) - ensure_scenario_databases({**SCENARIO, "years": [2030]}) - assert "premise" not in sys.modules -``` - -- [ ] **Step 2: Run the tests to verify they fail** - -Run: `pytest tests/test_scenario_builder.py -v` -Expected: FAIL — `ModuleNotFoundError: No module named 'bw_timex.scenario_builder'`. - -- [ ] **Step 3: Implement** - -Create `bw_timex/scenario_builder.py`: - -```python -"""Find the background vintages a scenario names, or build them with premise. - -`TimexLCA(scenario=...)` selects background databases by their metadata. When -the project does not hold them yet, `ensure_scenario_databases` builds the -missing ones with premise instead of leaving the user at a dead end. - -premise and bw2io are imported inside `_run_premise` and `_import_ecoinvent` -only, so `bw_timex` keeps working without them installed and a run that finds -everything it needs never touches either. -""" - -from __future__ import annotations - -from datetime import datetime - -import bw2data as bd -from loguru import logger - -from .database_metadata import ( - DYNAMIC, - REPRESENTATIVE_TIME, - SCENARIOS, - _normalize_representative_time, - database_matches_scenario, - split_scenario, -) - - -def find_existing_vintages(filters: dict) -> dict[int, str]: - """Map each year the project already covers to the database covering it. - - A year is covered by a registered database whose `representative_time` - falls in it and that the scenario filter keeps. "Keeps" is the resolver's - own rule (`database_matches_scenario`): a database is dropped only if it - declares a filtered key with a different value. Any stricter rule would - build a second database for a year `TimexLCA` already resolves. - """ - found = {} - for name in bd.databases: - metadata = bd.databases[name] - if REPRESENTATIVE_TIME not in metadata or metadata.get(SCENARIOS): - continue - value = _normalize_representative_time(metadata[REPRESENTATIVE_TIME], name) - if value == DYNAMIC or not isinstance(value, datetime): - continue - if not database_matches_scenario(metadata, filters): - continue - found.setdefault(value.year, name) - return found - - -def _run_premise(**kwargs) -> None: - """Filled in by Task 5. Every premise call happens here and nowhere else.""" - raise NotImplementedError - - -def _import_ecoinvent(**kwargs) -> str: - """Filled in by Task 4. Every bw2io call happens here and nowhere else.""" - raise NotImplementedError - - -def ensure_scenario_databases( - scenario: dict, - premise_key: str | None = None, - ecoinvent_credentials: tuple[str, str] | None = None, -) -> dict[str, datetime]: - """ - Make sure every year of `scenario` has a background database, building what is missing. - - Parameters - ---------- - scenario : dict - The same mapping `TimexLCA` takes, plus the build keys `years` - (required), `sectors` and `source_database`. - premise_key : str, optional - premise decryption key. Falls back to `$PREMISE_KEY`. - ecoinvent_credentials : tuple, optional - `(username, password)`, used only if ecoinvent has to be imported. - Falls back to `$ECOINVENT_USERNAME` / `$ECOINVENT_PASSWORD`. - - Returns - ------- - dict - Database name to the point in time it represents, for the vintages - found or built. - """ - filters, build = split_scenario(scenario) - years = build["years"] - - existing = find_existing_vintages(filters) - missing = [year for year in years if year not in existing] - - if not missing: - logger.info( - f"All {len(years)} requested background vintage(s) already exist in " - f"this project. Nothing to build." - ) - return {existing[year]: datetime(year, 1, 1) for year in years} - - raise NotImplementedError # building is added in Tasks 3-5 -``` - -- [ ] **Step 4: Run the tests to verify they pass** - -Run: `pytest tests/test_scenario_builder.py -v` -Expected: PASS. - -- [ ] **Step 5: Commit** - -```bash -git add bw_timex/scenario_builder.py tests/test_scenario_builder.py -git commit -m "feat: find the background vintages a scenario already covers" -``` - ---- - -### Task 3: Argument validation and credentials - -Everything that can be rejected must be rejected here, before a multi-gigabyte build starts. - -**Files:** -- Modify: `bw_timex/validation.py` -- Modify: `bw_timex/scenario_builder.py` -- Test: `tests/test_scenario_builder.py` - -**Interfaces:** -- Consumes: `ensure_scenario_databases`, `SCENARIO_FILTER_KEYS`. -- Produces: - - `bw_timex.validation.ScenarioBuildInputs` (pydantic model, fields `scenario: dict`) - - `bw_timex.scenario_builder._resolve_premise_key(premise_key: str | None) -> str` - - `bw_timex.scenario_builder._resolve_ecoinvent_credentials(credentials: tuple | None) -> tuple[str, str]` - -- [ ] **Step 1: Write the failing tests** - -Append to `tests/test_scenario_builder.py`: - -```python -@pytest.mark.usefixtures("temporal_grouping_db_monthly") -class TestValidation: - - def test_missing_years_raises(self, fake_premise): - with pytest.raises(ValueError, match="years"): - ensure_scenario_databases(SCENARIO) - - def test_empty_years_raises(self, fake_premise): - with pytest.raises(ValueError, match="years"): - ensure_scenario_databases({**SCENARIO, "years": []}) - - def test_non_integer_year_raises(self, fake_premise): - with pytest.raises(ValueError, match="years"): - ensure_scenario_databases({**SCENARIO, "years": ["2030"]}) - - def test_missing_filter_keys_are_named(self, fake_premise): - with pytest.raises(ValueError, match="ecoinvent_version"): - ensure_scenario_databases( - {"iam_model": "remind", "pathway": "SSP2-PkBudg500", "system_model": "cutoff", - "years": [2030]} - ) - - def test_no_scenario_raises(self, fake_premise): - with pytest.raises(ValueError, match="scenario"): - ensure_scenario_databases(None) - - -@pytest.mark.usefixtures("temporal_grouping_db_monthly") -class TestCredentials: - - @pytest.fixture(autouse=True) - def _ecoinvent_present(self): - # Without a source database the run would stop at the ecoinvent - # credentials before ever reaching the premise key. - write_minimal_database("ecoinvent-3.10.1-cutoff") - write_minimal_database("ecoinvent-3.10.1-biosphere") - - def test_premise_key_argument_wins_over_environment(self, fake_premise, monkeypatch): - monkeypatch.setenv("PREMISE_KEY", "from-environment") - ensure_scenario_databases({**SCENARIO, "years": [2030]}, premise_key="explicit") - assert fake_premise[0]["key"] == "explicit" - - def test_premise_key_falls_back_to_environment(self, fake_premise, monkeypatch): - monkeypatch.setenv("PREMISE_KEY", "from-environment") - ensure_scenario_databases({**SCENARIO, "years": [2030]}) - assert fake_premise[0]["key"] == "from-environment" - - def test_missing_premise_key_names_the_variable(self, fake_premise, monkeypatch): - monkeypatch.delenv("PREMISE_KEY", raising=False) - with pytest.raises(ValueError, match="PREMISE_KEY"): - ensure_scenario_databases({**SCENARIO, "years": [2030]}) -``` - -Note: the three `TestCredentials` tests reach the build path, which Tasks 4 and 5 complete. They are expected to fail until Task 5 lands; run them with `-k` as noted in each task's verification step. - -- [ ] **Step 2: Run the tests to verify they fail** - -Run: `pytest tests/test_scenario_builder.py -k "TestValidation" -v` -Expected: FAIL — `NotImplementedError` instead of `ValueError`, and `AttributeError: 'NoneType' object has no attribute 'items'` for the `None` case. - -- [ ] **Step 3: Implement the validation model** - -Append to `bw_timex/validation.py`: - -```python -class ScenarioBuildInputs(BaseModel): - """Validates the scenario mapping handed to ensure_scenario_databases""" - - model_config = {"arbitrary_types_allowed": True} - - scenario: dict - - @field_validator("scenario") - @classmethod - def validate_scenario(cls, v: dict) -> dict: - from .database_metadata import SCENARIO_FILTER_KEYS - - if not v: - raise ValueError( - "scenario must be a non-empty dictionary describing the background " - "to build, e.g. {'iam_model': 'remind', 'pathway': 'SSP2-PkBudg500', " - "'system_model': 'cutoff', 'ecoinvent_version': '3.10.1', " - "'years': [2030, 2040]}." - ) - years = v.get("years") - if not years or not isinstance(years, (list, tuple)): - raise ValueError( - "scenario must contain a non-empty `years` list to build background " - "databases, e.g. scenario={..., 'years': [2030, 2040]}. premise " - "builds one database per year." - ) - if not all(isinstance(year, int) and not isinstance(year, bool) for year in years): - raise ValueError( - f"scenario `years` must be integer years, e.g. [2030, 2040], got " - f"{list(years)}." - ) - missing = [key for key in SCENARIO_FILTER_KEYS if key not in v] - if missing: - raise ValueError( - f"scenario is missing {missing}, which premise needs to build a " - f"database. Provide all of {list(SCENARIO_FILTER_KEYS)}." - ) - return v -``` - -- [ ] **Step 4: Implement credential resolution and wire in the validation** - -In `bw_timex/scenario_builder.py`, add `import os` to the imports, `from .validation import ScenarioBuildInputs`, and: - -```python -def _resolve_premise_key(premise_key: str | None) -> str: - key = premise_key or os.environ.get("PREMISE_KEY") - if not key: - raise ValueError( - "No premise decryption key. Pass `premise_key=...` or set the " - "environment variable PREMISE_KEY. The key is needed to read " - "premise's bundled IAM scenarios; see " - "https://premise.readthedocs.io for how to request one." - ) - return key - - -def _resolve_ecoinvent_credentials( - credentials: tuple[str, str] | None, -) -> tuple[str, str]: - if credentials: - username, password = credentials - else: - username = os.environ.get("ECOINVENT_USERNAME") - password = os.environ.get("ECOINVENT_PASSWORD") - missing = [ - name - for name, value in ( - ("ECOINVENT_USERNAME", username), - ("ECOINVENT_PASSWORD", password), - ) - if not value - ] - if missing: - raise ValueError( - f"No ecoinvent credentials, needed to import the source database " - f"premise builds from. Pass `ecoinvent_credentials=(username, " - f"password)` or set {' and '.join(missing)}." - ) - return username, password -``` - -and make `ensure_scenario_databases` validate first: - -```python - ScenarioBuildInputs(scenario=scenario) - filters, build = split_scenario(scenario) -``` - -`ScenarioBuildInputs(scenario=None)` raises a pydantic `ValidationError`, which subclasses `ValueError` and whose message contains `scenario`, satisfying `test_no_scenario_raises`. - -- [ ] **Step 5: Run the validation tests to verify they pass** - -Run: `pytest tests/test_scenario_builder.py -k "TestValidation or TestFindExistingVintages or TestNothingToBuild" -v` -Expected: PASS. (`TestCredentials` still fails at `NotImplementedError` — Task 5 completes it.) - -- [ ] **Step 6: Commit** - -```bash -git add bw_timex/validation.py bw_timex/scenario_builder.py tests/test_scenario_builder.py -git commit -m "feat: validate scenario build arguments and resolve credentials" -``` - ---- - -### Task 4: Source database, biosphere, and the overwrite guard - -premise builds *from* an ecoinvent already in the project. Finding it, importing it if absent, and refusing to overwrite an unrelated database are all decisions that must happen before the build. - -**Files:** -- Modify: `bw_timex/scenario_builder.py` -- Test: `tests/test_scenario_builder.py` - -**Interfaces:** -- Consumes: `_resolve_ecoinvent_credentials` (Task 3). -- Produces: - - `vintage_name(filters: dict, year: int) -> str` - - `_resolve_source_database(filters: dict, build: dict, ecoinvent_credentials) -> tuple[str, str]` returning `(source_database, biosphere_name)` - - `_check_no_collisions(names: list[str], filters: dict) -> None` - - `_import_ecoinvent(version: str, system_model: str, credentials: tuple[str, str]) -> str` (real body) - -- [ ] **Step 1: Write the failing tests** - -Append to `tests/test_scenario_builder.py`: - -```python -@pytest.fixture -def fake_ecoinvent_import(monkeypatch): - """Records ecoinvent imports and registers what bw2io would register.""" - calls = [] - - def fake_import(version, system_model, credentials): - calls.append((version, system_model, credentials)) - name = f"ecoinvent-{version}-{system_model}" - write_minimal_database(name) - write_minimal_database(f"ecoinvent-{version}-biosphere") - return name - - monkeypatch.setattr( - "bw_timex.scenario_builder._import_ecoinvent", fake_import, raising=True - ) - return calls - - -@pytest.mark.usefixtures("temporal_grouping_db_monthly") -class TestVintageName: - - def test_name_is_deterministic(self): - from bw_timex.scenario_builder import vintage_name - - assert ( - vintage_name(SCENARIO, 2030) - == "ei_cutoff_3.10.1_remind_SSP2-PkBudg500_2030" - ) - - -@pytest.mark.usefixtures("temporal_grouping_db_monthly") -class TestSourceDatabase: - - def test_existing_ecoinvent_is_used(self, fake_premise, fake_ecoinvent_import, monkeypatch): - monkeypatch.setenv("PREMISE_KEY", "key") - write_minimal_database("ecoinvent-3.10.1-cutoff") - write_minimal_database("ecoinvent-3.10.1-biosphere") - ensure_scenario_databases({**SCENARIO, "years": [2030]}) - assert fake_ecoinvent_import == [] - assert fake_premise[0]["source_database"] == "ecoinvent-3.10.1-cutoff" - assert fake_premise[0]["biosphere"] == "ecoinvent-3.10.1-biosphere" - - def test_missing_ecoinvent_is_imported(self, fake_premise, fake_ecoinvent_import, monkeypatch): - monkeypatch.setenv("PREMISE_KEY", "key") - monkeypatch.setenv("ECOINVENT_USERNAME", "user") - monkeypatch.setenv("ECOINVENT_PASSWORD", "secret") - ensure_scenario_databases({**SCENARIO, "years": [2030]}) - assert fake_ecoinvent_import == [("3.10.1", "cutoff", ("user", "secret"))] - - def test_missing_ecoinvent_without_credentials_raises(self, fake_premise, monkeypatch): - monkeypatch.setenv("PREMISE_KEY", "key") - monkeypatch.delenv("ECOINVENT_USERNAME", raising=False) - monkeypatch.delenv("ECOINVENT_PASSWORD", raising=False) - with pytest.raises(ValueError, match="ECOINVENT_USERNAME"): - ensure_scenario_databases({**SCENARIO, "years": [2030]}) - - def test_explicit_source_database_is_used(self, fake_premise, fake_ecoinvent_import, monkeypatch): - monkeypatch.setenv("PREMISE_KEY", "key") - write_minimal_database("my_own_ecoinvent") - write_minimal_database("biosphere3") - ensure_scenario_databases( - {**SCENARIO, "years": [2030], "source_database": "my_own_ecoinvent"} - ) - assert fake_ecoinvent_import == [] - assert fake_premise[0]["source_database"] == "my_own_ecoinvent" - assert fake_premise[0]["biosphere"] == "biosphere3" - - def test_unregistered_explicit_source_database_raises(self, fake_premise, monkeypatch): - monkeypatch.setenv("PREMISE_KEY", "key") - with pytest.raises(ValueError, match="no_such_db"): - ensure_scenario_databases( - {**SCENARIO, "years": [2030], "source_database": "no_such_db"} - ) - - -@pytest.mark.usefixtures("temporal_grouping_db_monthly") -class TestOverwriteGuard: - - def test_foreign_database_under_target_name_raises(self, fake_premise, monkeypatch): - monkeypatch.setenv("PREMISE_KEY", "key") - write_minimal_database("ei_cutoff_3.10.1_remind_SSP2-PkBudg500_2030") - with pytest.raises(ValueError, match="already exists"): - ensure_scenario_databases({**SCENARIO, "years": [2030]}) - - def test_nothing_is_built_when_a_name_collides(self, fake_premise, monkeypatch): - monkeypatch.setenv("PREMISE_KEY", "key") - write_minimal_database("ei_cutoff_3.10.1_remind_SSP2-PkBudg500_2030") - with pytest.raises(ValueError): - ensure_scenario_databases({**SCENARIO, "years": [2030, 2040]}) - assert fake_premise == [] -``` - -- [ ] **Step 2: Run the tests to verify they fail** - -Run: `pytest tests/test_scenario_builder.py -k "TestVintageName or TestSourceDatabase or TestOverwriteGuard" -v` -Expected: FAIL — `ImportError: cannot import name 'vintage_name'`, and `NotImplementedError` from the others. - -- [ ] **Step 3: Implement** - -In `bw_timex/scenario_builder.py`: - -```python -def vintage_name(filters: dict, year: int) -> str: - """The database name a built vintage gets.""" - return ( - f"ei_{filters['system_model']}_{filters['ecoinvent_version']}_" - f"{filters['iam_model']}_{filters['pathway']}_{year}" - ) - - -def _check_no_collisions(names: list[str], filters: dict) -> None: - """Refuse to build over a database that is not ours. - - `write_db_to_brightway` deletes and rewrites a database of the same name - without asking. A name that exists here belongs to someone else: a name - that matched the scenario would have satisfied its year already, and its - year would not be in the build list. - """ - colliding = [name for name in names if name in bd.databases] - if colliding: - raise ValueError( - f"Database(s) {colliding} already exist(s) in this project but do(es) " - f"not match scenario {filters!r}, and premise would overwrite them. " - f"Rename or delete them, or map them yourself with `database_dates`." - ) - - -def _resolve_source_database( - filters: dict, build: dict, ecoinvent_credentials -) -> tuple[str, str]: - """The ecoinvent database premise builds from, and its biosphere. - - Importing ecoinvent takes a while and needs a licence, so it happens only - when there is nothing to build from. - """ - version = filters["ecoinvent_version"] - system_model = filters["system_model"] - default_biosphere = f"ecoinvent-{version}-biosphere" - - source = build.get("source_database") - if source is not None: - if source not in bd.databases: - raise ValueError( - f"source_database '{source}' is not registered in this project. " - f"Available databases: {sorted(bd.databases)}." - ) - else: - source = f"ecoinvent-{version}-{system_model}" - if source not in bd.databases: - logger.info( - f"No database '{source}' in this project. Importing ecoinvent " - f"{version} ({system_model}) first; this takes a while and needs " - f"an ecoinvent licence." - ) - source = _import_ecoinvent( - version=version, - system_model=system_model, - credentials=_resolve_ecoinvent_credentials(ecoinvent_credentials), - ) - - for candidate in (default_biosphere, "biosphere3"): - if candidate in bd.databases: - return source, candidate - raise ValueError( - f"No biosphere database found: expected '{default_biosphere}' or " - f"'biosphere3'. premise needs one to link elementary flows." - ) -``` - -and give `_import_ecoinvent` its real body: - -```python -def _import_ecoinvent(version: str, system_model: str, credentials: tuple[str, str]) -> str: - """Import an ecoinvent release. The only place bw2io is called.""" - try: - from bw2io import import_ecoinvent_release - except ImportError as error: - raise ImportError( - 'bw2io is needed to import ecoinvent. Install it with: pip install ' - '"bw_timex[premise]"' - ) from error - - username, password = credentials - import_ecoinvent_release( - version=version, - system_model=system_model, - username=username, - password=password, - ) - return f"ecoinvent-{version}-{system_model}" -``` - -Replace the `raise NotImplementedError` at the end of `ensure_scenario_databases` with: - -```python - names = {year: vintage_name(filters, year) for year in missing} - _check_no_collisions(list(names.values()), filters) - - key = _resolve_premise_key(premise_key) - source_database, biosphere = _resolve_source_database( - filters, build, ecoinvent_credentials - ) - - raise NotImplementedError # the premise run is added in Task 5 -``` - -- [ ] **Step 4: Run the tests to verify they pass** - -Run: `pytest tests/test_scenario_builder.py -k "TestVintageName or TestOverwriteGuard" -v` -Expected: PASS. - -Run: `pytest tests/test_scenario_builder.py -k "TestSourceDatabase" -v` -Expected: `test_missing_ecoinvent_without_credentials_raises` and `test_unregistered_explicit_source_database_raises` PASS; the other three still fail at `NotImplementedError`, completed by Task 5. - -- [ ] **Step 5: Commit** - -```bash -git add bw_timex/scenario_builder.py tests/test_scenario_builder.py -git commit -m "feat: resolve the premise source database and guard against overwrites" -``` - ---- - -### Task 5: Run premise and record what was built - -**Files:** -- Modify: `bw_timex/scenario_builder.py` -- Test: `tests/test_scenario_builder.py` - -**Interfaces:** -- Consumes: everything from Tasks 2-4. -- Produces: `_run_premise(*, scenarios, source_database, source_version, system_model, biosphere, sectors, names, key) -> None` (keyword-only; every test fake matches this signature), and `ensure_scenario_databases` returning the full name → `datetime` mapping. - -- [ ] **Step 1: Write the failing tests** - -Append to `tests/test_scenario_builder.py`: - -```python -@pytest.mark.usefixtures("temporal_grouping_db_monthly") -class TestBuilding: - - @pytest.fixture(autouse=True) - def _ecoinvent_present(self, monkeypatch): - monkeypatch.setenv("PREMISE_KEY", "key") - write_minimal_database("ecoinvent-3.10.1-cutoff") - write_minimal_database("ecoinvent-3.10.1-biosphere") - - def test_only_missing_years_are_built(self, fake_premise): - write_vintage("ei_2030", 2030) - ensure_scenario_databases({**SCENARIO, "years": [2020, 2030, 2040]}) - assert len(fake_premise) == 1 - assert [s["year"] for s in fake_premise[0]["scenarios"]] == [2020, 2040] - - def test_premise_scenarios_carry_model_and_pathway(self, fake_premise): - ensure_scenario_databases({**SCENARIO, "years": [2030]}) - assert fake_premise[0]["scenarios"] == [ - {"model": "remind", "pathway": "SSP2-PkBudg500", "year": 2030} - ] - - def test_source_version_and_system_model_are_passed(self, fake_premise): - ensure_scenario_databases({**SCENARIO, "years": [2030]}) - assert fake_premise[0]["source_version"] == "3.10.1" - assert fake_premise[0]["system_model"] == "cutoff" - - def test_names_line_up_with_scenarios(self, fake_premise): - ensure_scenario_databases({**SCENARIO, "years": [2030, 2040]}) - assert fake_premise[0]["names"] == [ - "ei_cutoff_3.10.1_remind_SSP2-PkBudg500_2030", - "ei_cutoff_3.10.1_remind_SSP2-PkBudg500_2040", - ] - - def test_all_sectors_by_default(self, fake_premise): - ensure_scenario_databases({**SCENARIO, "years": [2030]}) - assert fake_premise[0]["sectors"] is None - - def test_narrowed_sectors_are_passed_through(self, fake_premise): - ensure_scenario_databases( - {**SCENARIO, "years": [2030], "sectors": ["electricity", "steel"]} - ) - assert fake_premise[0]["sectors"] == ["electricity", "steel"] - - def test_narrowed_sectors_are_recorded_in_metadata(self, fake_premise): - ensure_scenario_databases( - {**SCENARIO, "years": [2030], "sectors": ["electricity"]} - ) - name = "ei_cutoff_3.10.1_remind_SSP2-PkBudg500_2030" - assert bd.databases[name]["sectors"] == ["electricity"] - - def test_no_sectors_metadata_when_all_sectors(self, fake_premise): - ensure_scenario_databases({**SCENARIO, "years": [2030]}) - name = "ei_cutoff_3.10.1_remind_SSP2-PkBudg500_2030" - assert "sectors" not in bd.databases[name] - - def test_built_and_existing_vintages_are_returned(self, fake_premise): - write_vintage("ei_2030", 2030) - result = ensure_scenario_databases({**SCENARIO, "years": [2030, 2040]}) - assert result == { - "ei_2030": datetime(2030, 1, 1), - "ei_cutoff_3.10.1_remind_SSP2-PkBudg500_2040": datetime(2040, 1, 1), - } - - def test_missing_representative_time_after_write_raises(self, monkeypatch): - def fake_run_premise(**kwargs): - for name in kwargs["names"]: - write_minimal_database(name) # no metadata: an old premise - - monkeypatch.setattr( - "bw_timex.scenario_builder._run_premise", fake_run_premise, raising=True - ) - with pytest.raises(RuntimeError, match="2.4.9.2"): - ensure_scenario_databases({**SCENARIO, "years": [2030]}) - - -@pytest.mark.usefixtures("temporal_grouping_db_monthly") -class TestPremiseNotInstalled: - - def test_import_error_names_the_extra(self, monkeypatch): - from bw_timex import scenario_builder - - monkeypatch.setenv("PREMISE_KEY", "key") - write_minimal_database("ecoinvent-3.10.1-cutoff") - write_minimal_database("ecoinvent-3.10.1-biosphere") - monkeypatch.setitem(sys.modules, "premise", None) # forces ImportError - with pytest.raises(ImportError, match=r'bw_timex\[premise\]'): - scenario_builder.ensure_scenario_databases({**SCENARIO, "years": [2030]}) -``` - -- [ ] **Step 2: Run the tests to verify they fail** - -Run: `pytest tests/test_scenario_builder.py -k "TestBuilding or TestPremiseNotInstalled" -v` -Expected: FAIL with `NotImplementedError`. - -- [ ] **Step 3: Implement the seam** - -Replace the `_run_premise` stub in `bw_timex/scenario_builder.py`: - -```python -def _run_premise( - *, - scenarios: list[dict], - source_database: str, - source_version: str, - system_model: str, - biosphere: str, - sectors: list[str] | None, - names: list[str], - key: str, -) -> None: - """Build and write one prospective database per scenario. - - The only place premise is called. One `NewDatabase` for all scenarios, not - one per year: premise caches the extracted source database, so separate - runs would re-extract ecoinvent every time. - """ - try: - from premise import NewDatabase - except ImportError as error: - raise ImportError( - 'premise is needed to build background databases. Install it with: ' - 'pip install "bw_timex[premise]"' - ) from error - - ndb = NewDatabase( - scenarios=scenarios, - source_db=source_database, - source_version=source_version, - system_model=system_model, - biosphere_name=biosphere, - key=key, - ) - if sectors: - ndb.update(sectors) - else: - ndb.update() - ndb.write_db_to_brightway(name=names) -``` - -- [ ] **Step 4: Implement the build path** - -Replace the trailing `raise NotImplementedError` in `ensure_scenario_databases` with: - -```python - sectors = build.get("sectors") - logger.info( - f"Building {len(missing)} background database(s) for year(s) {missing} with " - f"premise ({filters['iam_model']}, {filters['pathway']}, " - f"{'all sectors' if not sectors else ', '.join(sectors)}). Each is a full " - f"copy of ecoinvent, so expect tens of minutes and roughly 2-4 GB per year." - ) - - _run_premise( - scenarios=[ - { - "model": filters["iam_model"], - "pathway": filters["pathway"], - "year": year, - } - for year in missing - ], - source_database=source_database, - source_version=filters["ecoinvent_version"], - system_model=filters["system_model"], - biosphere=biosphere, - sectors=sectors, - names=[names[year] for year in missing], - key=key, - ) - - for year in missing: - name = names[year] - if REPRESENTATIVE_TIME not in bd.databases.get(name, {}): - raise RuntimeError( - f"premise wrote '{name}' without `{REPRESENTATIVE_TIME}` metadata, so " - f"`TimexLCA` cannot tell what point in time it represents. This " - f"metadata is written by premise >= 2.4.9.2; check your installed " - f"version, or set it yourself with `bw_timex.set_database_metadata`." - ) - if sectors: - # premise does not record which sectors were updated, and two runs of - # the same pathway with different sectors would otherwise look identical - # to the scenario filter. - set_database_metadata(name, sectors=list(sectors)) - - logger.info(f"Built {len(missing)} background database(s).") - - resolved = {existing[year]: datetime(year, 1, 1) for year in years if year in existing} - resolved.update({names[year]: datetime(year, 1, 1) for year in missing}) - return resolved -``` - -Add `set_database_metadata` to the `.database_metadata` import at the top of the module. - -- [ ] **Step 5: Run the tests to verify they pass** - -Run: `pytest tests/test_scenario_builder.py -v` -Expected: PASS — every class in the file, including `TestCredentials` and `TestSourceDatabase` from Tasks 3 and 4. - -- [ ] **Step 6: Commit** - -```bash -git add bw_timex/scenario_builder.py tests/test_scenario_builder.py -git commit -m "feat: build missing background vintages with premise" -``` - ---- - -### Task 6: Wire it into TimexLCA - -**Files:** -- Modify: `bw_timex/timex_lca.py:129-201` (signature, docstring, `__init__` body) and `bw_timex/timex_lca.py:283-352` (`_resolve_database_dates`) -- Modify: `bw_timex/validation.py` (`TimexLCAInputs`) -- Modify: `bw_timex/__init__.py` -- Test: `tests/test_scenario_builder.py` - -**Interfaces:** -- Consumes: `ensure_scenario_databases` (Task 5). -- Produces: `TimexLCA(demand, method, database_dates=None, scenario=None, create_missing=False, premise_key=None, ecoinvent_credentials=None, use_global_lci_cache=True)`; `bw_timex.ensure_scenario_databases`. - -- [ ] **Step 1: Write the failing tests** - -Append to `tests/test_scenario_builder.py`: - -```python -@pytest.mark.usefixtures("temporal_grouping_db_monthly") -class TestTimexLCAIntegration: - - @pytest.fixture(autouse=True) - def _ecoinvent_present(self, monkeypatch): - monkeypatch.setenv("PREMISE_KEY", "key") - write_minimal_database("ecoinvent-3.10.1-cutoff") - write_minimal_database("ecoinvent-3.10.1-biosphere") - - def test_public_export(self): - import bw_timex - - assert bw_timex.ensure_scenario_databases is ensure_scenario_databases - - def test_create_missing_builds_and_resolves(self, fake_premise): - from bw_timex import TimexLCA - - tlca = TimexLCA( - demand={("foreground", "A"): 1}, - method=("GWP", "example"), - scenario={**SCENARIO, "years": [2030]}, - create_missing=True, - ) - assert ( - tlca.database_dates["ei_cutoff_3.10.1_remind_SSP2-PkBudg500_2030"] - == datetime(2030, 1, 1) - ) - assert tlca.database_dates["foreground"] == "dynamic" - - def test_default_does_not_build(self, fake_premise): - from bw_timex import TimexLCA - - with pytest.raises(ValueError): - TimexLCA( - demand={("foreground", "A"): 1}, - method=("GWP", "example"), - scenario={**SCENARIO, "years": [2030]}, - ) - assert fake_premise == [] - - def test_error_without_create_missing_mentions_it(self, fake_premise): - from bw_timex import TimexLCA - - with pytest.raises(ValueError, match="create_missing"): - TimexLCA( - demand={("foreground", "A"): 1}, - method=("GWP", "example"), - scenario={**SCENARIO, "years": [2030]}, - ) - - def test_create_missing_with_database_dates_raises(self, fake_premise): - from bw_timex import TimexLCA - - with pytest.raises(ValueError, match="database_dates"): - TimexLCA( - demand={("foreground", "A"): 1}, - method=("GWP", "example"), - database_dates={"foreground": "dynamic"}, - create_missing=True, - ) - assert fake_premise == [] - - def test_create_missing_without_scenario_raises(self, fake_premise): - from bw_timex import TimexLCA - - with pytest.raises(ValueError, match="scenario"): - TimexLCA( - demand={("foreground", "A"): 1}, - method=("GWP", "example"), - create_missing=True, - ) - assert fake_premise == [] -``` - -The `temporal_grouping_db_monthly` fixture registers a `foreground` database with a process `A` and the method `("GWP", "example")`; check the fixture and use whatever demand key and method it actually provides, rather than inventing one. - -- [ ] **Step 2: Run the tests to verify they fail** - -Run: `pytest tests/test_scenario_builder.py -k "TestTimexLCAIntegration" -v` -Expected: FAIL — `AttributeError: module 'bw_timex' has no attribute 'ensure_scenario_databases'` and `TypeError: __init__() got an unexpected keyword argument 'create_missing'`. - -- [ ] **Step 3: Extend the validation model** - -In `bw_timex/validation.py`, add to `TimexLCAInputs`: - -```python - create_missing: bool = False - premise_key: Optional[str] = None - ecoinvent_credentials: Optional[tuple] = None -``` - -and a validator on the model: - -```python - @model_validator(mode="after") - def validate_create_missing(self) -> "TimexLCAInputs": - if not self.create_missing: - return self - if self.database_dates is not None: - raise ValueError( - "`create_missing` builds the background databases a `scenario` " - "describes, and only applies when `database_dates` is not given. " - "Pass one or the other." - ) - if self.scenario is None: - raise ValueError( - "`create_missing=True` needs a `scenario` describing what to build, " - "e.g. scenario={'iam_model': 'remind', 'pathway': 'SSP2-PkBudg500', " - "'system_model': 'cutoff', 'ecoinvent_version': '3.10.1', " - "'years': [2030, 2040]}." - ) - if self.ecoinvent_credentials is not None and len(self.ecoinvent_credentials) != 2: - raise ValueError( - "`ecoinvent_credentials` must be a (username, password) tuple." - ) - return self -``` - -- [ ] **Step 4: Plumb the arguments through TimexLCA** - -In `bw_timex/timex_lca.py`, extend the signature: - -```python - def __init__( - self, - demand: dict, - method: tuple, - database_dates: dict = None, - scenario: dict = None, - create_missing: bool = False, - premise_key: str = None, - ecoinvent_credentials: tuple = None, - use_global_lci_cache: bool = True, - ) -> None: -``` - -Add to the docstring's Parameters section, after the `scenario` entry: - -``` - create_missing : bool, optional - If True, background databases the `scenario` names but that this - project does not hold yet are built with `premise`, and ecoinvent - is imported first if it is missing. The `scenario` then also needs - a `years` list (and may narrow `sectors` or name a - `source_database`). Needs the optional dependency: - `pip install "bw_timex[premise]"`. Building takes tens of minutes - and roughly 2-4 GB per year. Default is False, which raises - instead of building. Cannot be combined with `database_dates`. - premise_key : str, optional - premise decryption key, used only when building. Falls back to the - environment variable `PREMISE_KEY`. - ecoinvent_credentials : tuple, optional - `(username, password)`, used only when ecoinvent itself has to be - imported. Falls back to the environment variables - `ECOINVENT_USERNAME` and `ECOINVENT_PASSWORD`. -``` - -Replace the assignment block at the start of `__init__` (currently `self.demand = demand` through the `TimexLCAInputs(...)` call) with validation first, then the build, then resolution: - -```python - self.demand = demand - self.method = method - self.scenario = scenario - - TimexLCAInputs( - demand=demand, - method=method, - database_dates=database_dates, - scenario=scenario, - create_missing=create_missing, - premise_key=premise_key, - ecoinvent_credentials=ecoinvent_credentials, - ) - - if create_missing: - from .scenario_builder import ensure_scenario_databases - - ensure_scenario_databases( - scenario, - premise_key=premise_key, - ecoinvent_credentials=ecoinvent_credentials, - ) - - self.database_dates = self._resolve_database_dates( - demand=demand, database_dates=database_dates, scenario=scenario - ) -``` - -Note the reordering: `TimexLCAInputs` now runs *before* `_resolve_database_dates`, because a bad `create_missing` combination must be rejected before anything is built. The old call passed `self.database_dates`; the new one passes the raw `database_dates` argument, which is what the model validates (`validate_demand_in_dynamic_databases` short-circuits when it is `None`, as it does today for the metadata path). - -In `_resolve_database_dates`, extend the "matched no database" error with a line about building: - -```python - raise ValueError( - f"scenario={scenario!r} matched no database in this project. " - f"Values actually declared for its key(s) by this project's " - f"databases: {details}. Check for a typo in the filter, or pass " - f"`create_missing=True` (with a `years` list in the scenario) to " - f"build the databases with premise." - ) -``` - -- [ ] **Step 5: Export the helper** - -In `bw_timex/__init__.py`, add `from .scenario_builder import ensure_scenario_databases` next to the `database_metadata` import, and `"ensure_scenario_databases"` to `__all__` in the `utils` block (alphabetically, before `get_exchange`). - -`scenario_builder` imports no third-party library at module level, so this does not make premise a hard dependency. - -- [ ] **Step 6: Run the tests to verify they pass** - -Run: `pytest tests/test_scenario_builder.py -v` -Expected: PASS. - -Run: `pytest tests/test_public_api.py tests/test_timex_lca.py tests/test_database_metadata.py -v` -Expected: PASS. `test_public_api.py` may assert on `__all__`; update it there if it enumerates the exports. - -- [ ] **Step 7: Run the whole suite** - -Run: `pytest -q` -Expected: PASS. - -- [ ] **Step 8: Commit** - -```bash -git add bw_timex/timex_lca.py bw_timex/validation.py bw_timex/__init__.py tests/ -git commit -m "feat: let TimexLCA build the background databases a scenario names" -``` - ---- - -### Task 7: Packaging and documentation - -**Files:** -- Modify: `pyproject.toml` -- Modify: `docs/content/installation.md` -- Modify: `docs/content/getting_started/quickstart.md` -- Modify: `CHANGES.md` - -**Interfaces:** -- Consumes: the finished feature. -- Produces: the `bw_timex[premise]` extra. - -- [ ] **Step 1: Add the optional dependency** - -In `pyproject.toml`, under `[project.optional-dependencies]`, next to the existing `solvers` entry: - -```toml -premise = ["premise[bw25]>=2.4.9.2"] -``` - -- [ ] **Step 2: Verify the extra resolves** - -Run: `python -m pip install --dry-run ".[premise]" 2>&1 | tail -20` -Expected: a resolution that includes `premise>=2.4.9.2` and a `bw2data` ≥ 4.6. If the resolver reports a conflict with bw_timex's `bw2data>=4.6`, stop and report it rather than loosening the pin. - -- [ ] **Step 3: Document installation** - -In `docs/content/installation.md`, after the existing install instructions: - -```markdown -## Building background databases automatically - -`TimexLCA` can build the prospective background databases a scenario names, using -[premise](https://github.com/polca/premise). That needs the optional extra: - -```bash -pip install "bw_timex[premise]" -``` - -and credentials, which `bw_timex` reads from the environment unless you pass them: - -| variable | needed for | -|---|---| -| `PREMISE_KEY` | decrypting premise's bundled IAM scenarios | -| `ECOINVENT_USERNAME`, `ECOINVENT_PASSWORD` | importing ecoinvent, if the project has none yet | -``` - -- [ ] **Step 4: Document usage** - -In `docs/content/getting_started/quickstart.md`, next to the existing `scenario` documentation: - -````markdown -If the project does not hold the scenario's databases yet, `bw_timex` can build them -with premise instead of raising. Add the years to the scenario and pass -`create_missing=True`: - -```python -tlca = TimexLCA( - demand={("foreground", "ev"): 1}, - method=("EF v3.1", "climate change", "global warming potential (GWP100)"), - scenario={ - "iam_model": "remind", - "pathway": "SSP2-PkBudg500", - "system_model": "cutoff", - "ecoinvent_version": "3.10.1", - "years": [2020, 2030, 2040], - }, - create_missing=True, -) -``` - -Only missing years are built, so running this again builds nothing. ecoinvent is -imported first if the project has none. Each vintage is a full copy of ecoinvent: -expect tens of minutes and roughly 2-4 GB per year. Two optional scenario keys tune -the build: `sectors` narrows what premise updates (all sectors by default), and -`source_database` names the ecoinvent to build from, if it is not the one -`import_ecoinvent_release` writes. -```` - -- [ ] **Step 5: Changelog** - -Add one bullet at the top of the current unreleased section of `CHANGES.md`: - -```markdown -* Added `TimexLCA(scenario={..., "years": [...]}, create_missing=True)`, which builds background databases the scenario names but the project does not hold, using premise (optional extra: `pip install "bw_timex[premise]"`). Only missing years are built, ecoinvent is imported first if absent, and credentials are read from `PREMISE_KEY` / `ECOINVENT_USERNAME` / `ECOINVENT_PASSWORD` unless passed explicitly. Also available standalone as `bw_timex.ensure_scenario_databases` -``` - -- [ ] **Step 6: Verify the docs build** - -Run: `python docs/convert_notebooks.py --help 2>/dev/null; ls docs` -Expected: no error. If the project has a docs build command in `zensical.toml` or CI, run that instead and expect a clean build. - -- [ ] **Step 7: Run the whole suite once more** - -Run: `pytest -q` -Expected: PASS. - -- [ ] **Step 8: Commit** - -```bash -git add pyproject.toml docs CHANGES.md -git commit -m "docs: document building background databases with premise" -``` - ---- - -## Manual verification (not part of CI) - -Once, by hand, in a project with an ecoinvent licence and a premise key: - -```python -import bw2data as bd -from bw_timex import ensure_scenario_databases - -bd.projects.set_current("premise_autocreate_check") -ensure_scenario_databases( - { - "iam_model": "remind", - "pathway": "SSP2-PkBudg500", - "system_model": "cutoff", - "ecoinvent_version": "3.10.1", - "years": [2030], - "sectors": ["electricity"], # keep the check short - } -) -print(bd.databases["ei_cutoff_3.10.1_remind_SSP2-PkBudg500_2030"]) -``` - -Confirm: the database exists, its metadata carries `representative_time`, -`iam_model`, `pathway`, `system_model`, `ecoinvent_version` from premise plus -`sectors` from us, and a second call to the same function builds nothing. diff --git a/docs/superpowers/specs/2026-08-03-multiple-databases-per-date-design.md b/docs/superpowers/specs/2026-08-03-multiple-databases-per-date-design.md deleted file mode 100644 index f618b16c..00000000 --- a/docs/superpowers/specs/2026-08-03-multiple-databases-per-date-design.md +++ /dev/null @@ -1,174 +0,0 @@ -# Multiple background databases per point in time - -## Problem - -`bw_timex` maps each background database to one point in time via `database_dates`. -Internally it assumes the reverse mapping is also unique: `TimelineBuilder` builds -`reversed_database_dates = {date: database}` (`timeline_builder.py:549`) and the -background-traversal extractor builds the same inversion -(`edge_extractor.py:125`). Two databases sharing a date collapse into one, so only -one of them is ever reachable. - -This forces users to keep modified copies of background processes inside the -background database they were copied from, which violates the practice of leaving -ecoinvent/premise databases untouched. The electric-vehicle premise notebooks do -exactly that: they write `..., without EOL` copies back into -`ei312_REMIND-EU_SSP2_NDC_`. - -## Goal - -Allow several static databases to share a date, so a study can keep its modified -background processes in its own databases — one per vintage — while the untouched -vintages stay read-only. - -## Non-goals - -- Changing the `database_dates` signature or any other public API. -- Overlay/shadow semantics, database priority lists, or explicit vintage groups. - These are only needed for same-name overrides, which this design rejects with an - error instead (see "Ambiguity"). -- Relaxing the cross-vintage matching key. Matching stays - `(name, reference product, location)`. - -## Design - -### Public interface - -Unchanged. `database_dates` is `{database_name: datetime | "dynamic"}`; its keys are -database names, so duplicate datetime values are already syntactically valid. They -simply have to stop breaking internally. - -```python -database_dates = { - "ei312_2020": datetime(2020, 1, 1), - "ei312_2030": datetime(2030, 1, 1), - "ev_background_2020": datetime(2020, 1, 1), # modified copies - "ev_background_2030": datetime(2030, 1, 1), - "foreground": "dynamic", -} -``` - -`validation.py` already accepts this — it validates each value independently and -never checks uniqueness — so no validation change is required. - -### Resolution rule: per producer, by content - -Temporal market shares are currently computed per *producer date* and then remapped -to database names through the global date -> database inversion. They become -per *producer*: - -1. Candidate databases for a producer = the static databases that contain a - `(name, reference product, location)` match for it. -2. The producer's interpolation runs over the dates of its candidates only. -3. Each resulting date maps back to the database through that producer's own - candidate map. - -With the example above, `glider production, passenger car, without EOL` exists only -in `ev_background_2020` and `ev_background_2030`, so it interpolates over those. -`market group for electricity, low voltage` exists only in `ei312_2020` and -`ei312_2030`, so it interpolates over those. No configuration is needed for either. - -### Ambiguity: same triplet, same date - -If a producer matches in more than one database at the *same* date — e.g. a copy -that keeps the original's name, reference product and location — the model is -genuinely ambiguous. Raise a `ValueError` naming the producer and the colliding -databases, and state the fix: give the copy a distinct name, reference product or -location. - -The check runs lazily, only for producers that actually become temporal markets, so -it never scans the full background. - -### Partial coverage - -A producer that exists in fewer vintages than are configured (e.g. a copy made once, -into a single dated database) interpolates over the vintages it does exist in. With -one candidate that means a constant `{database: 1}` — a time-invariant background -process, which is legitimate. - -Emit one `logger.warning` per such producer, naming it and the databases used, so a -forgotten vintage copy is visible. - -### Components - -**`TimelineBuilder.add_column_temporal_market_shares_to_timeline`** -(`timeline_builder.py:508-596`) - -Replace `self.reversed_database_dates` with a per-producer map -`{producer_id: {date: database_name}}`, built in a single pass over `self.nodes` -metadata restricted to the market producers (the leaf background frontier — normally -a handful of nodes). `self.nodes` holds `LazyActivity` proxies whose `name`, -`reference product` and `location` come from scalar columns, so the pass does not -unpickle data blobs. - -Interpolation helpers (`find_closest_date`, -`get_weights_for_interpolation_between_nearest_years`) are unchanged; each producer -passes its own `sorted_dates`. Weights are memoized per -`(candidate_dates_tuple, producer_date)` so producers sharing a candidate set do not -recompute. - -`add_interpolation_weights_at_intersection_to_background` -(`timeline_builder.py:667`) is the only other reader of -`reversed_database_dates` and has no call site; delete it. - -**`edge_extractor.VariantSplitMixin._variant_shares_for_date`** -(`edge_extractor.py:115-137`) - -Takes the producer's `node_id` in addition to the date. Candidates come from -`interdatabase_activity_mapping[node_id]` intersected with the static databases; -under `traverse_background` the full mapping is already built up front -(`TimexLCA.add_full_interdatabase_activity_mapping`). Same ambiguity error and -partial-coverage warning. `_resolve_in_variant` is unchanged. The single call site is -`edge_extractor.py:379`, inside `_emit_variant_split*`, which already has `node_id`. - -**`TimexLCA.add_interdatabase_activity_mapping_from_timeline`** -(`timex_lca.py:1568`) - -The triplet pass in the timeline builder produces exactly the `{producer: {db: id}}` -data this method recomputes after the timeline is built. Populate -`interdatabase_activity_mapping` from the timeline builder's pass and let this method -reuse it, so the scan happens once. Net effect on setup time is neutral or better. - -**`TimexLCA.add_full_interdatabase_activity_mapping`** (`timex_lca.py:1537`) - -Picks one arbitrary anchor per triplet across all static databases -(`tuples_dict.setdefault`). Under a same-date collision the ambiguity error fires -first, so the arbitrary choice is never reached; no change needed. - -### Deliberately unchanged - -These already key on database name rather than on date, and work as soon as the -shares dict carries the right names: - -- `matrix_modifier.py:287` — iterates `{database: share}` and looks the producer up - per database through `interdatabase_activity_mapping.find_match`. -- `helper_classes.InterDatabaseMapping` — `{anchor_id: {database: id}}`. -- `activity_time_mapping` — keyed `((database, code), date_hash)`, unique across - same-date databases because codes differ. -- `TimexLCA.prepare_base_lca_inputs` / `create_node_collections` — reach the new - databases through `find_graph_dependents` from the foreground, as with any other - linked background database. - -## Testing - -Test-first. New fixture: `background_2020` / `background_2030` plus -`background_mod_2020` / `background_mod_2030` at identical dates, with the foreground -linked to the modified copies. - -1. Shares of a copied producer route to the `_mod_` family; shares of an untouched - producer route to the plain family; weights match the single-database-per-date - values. -2. A producer with the same triplet in two databases at one date raises `ValueError` - naming both databases. -3. A producer present in a single vintage yields `{database: 1}` at every date and - logs a warning. -4. The same routing holds with `traverse_background=True`. -5. Existing unique-date tests pass unchanged (regression). - -## Follow-up - -Update `notebooks/example_electric_vehicle_premise_simple.ipynb`, the full -`example_electric_vehicle_premise.ipynb`, and -`notebooks/teaching/teaching_example_ev_premise.ipynb` to write the `without EOL` -copies into `ev_background_` databases instead of into the premise databases, -and document duplicate dates in the `database_dates` docs. diff --git a/docs/superpowers/specs/2026-08-21-representative-time-metadata-design.md b/docs/superpowers/specs/2026-08-21-representative-time-metadata-design.md deleted file mode 100644 index 773202ed..00000000 --- a/docs/superpowers/specs/2026-08-21-representative-time-metadata-design.md +++ /dev/null @@ -1,284 +0,0 @@ -# Representative time as database metadata - -## Problem - -`TimexLCA` learns what a background database represents in one way only: the -`database_dates` argument the user hand-writes at every call site. - -```python -database_dates = { - "ei310_remind_SSP2-PkBudg500_2030": datetime(2030, 1, 1), - "ei310_remind_SSP2-PkBudg500_2040": datetime(2040, 1, 1), - "ei310_remind_SSP2-PkBudg500_2050": datetime(2050, 1, 1), - "foreground": "dynamic", -} -``` - -The information is already in the database — a premise export knows the year it was -built for — but it lives only in the database *name*, so every study re-types it, and -a typo either raises (`Database 'x' not available`) or, worse, silently maps a vintage -to the wrong year. - -premise [PR #303](https://github.com/polca/premise/pull/303) (merged to `master`, not -in 2.4.9.2) closes the gap on the producing side: exported Brightway databases now -carry what they represent in their `bd.databases[name]` metadata. - -```python -{ - # written by brightway - "format": "Ecoinvent XML", "depends": [...], "backend": "sqlite", - "number": 43648, "modified": "...", "processed": "...", - # written by premise - "premise_version": "2.4.9.1", - "iam_model": "remind", - "pathway": "SSP2-PkBudg500", - "representative_time": "2050-01-01T00:00:00", - "ecoinvent_version": "3.10.1", - "system_model": "cutoff", -} -``` - -Multi-scenario exports (superstructure, scenario arrays) instead carry a `scenarios` -list of such mappings, and a top-level `representative_time` only when all their -scenarios share a year. User (external) scenarios are listed under -`external_scenarios`. - -## Goal - -`TimexLCA` reads the databases' own metadata by default, so the common case needs no -timing argument at all: - -```python -tlca = TimexLCA(demand={("foreground", "A"): 1}, method=("GWP", "example")) -``` - -A project holding several IAM scenarios stays unambiguous: `TimexLCA` refuses to guess -and tells the user how to pick. - -```python -tlca = TimexLCA( - demand={("foreground", "A"): 1}, - method=("GWP", "example"), - scenario={"pathway": "SSP2-PkBudg500"}, -) -``` - -Databases that carry no metadata (hand-built vintages, the foreground) get it from a -one-line helper instead of a repeated argument. - -## Non-goals - -- Removing or deprecating `database_dates`. It stays, unchanged in meaning, and - scripts that pass it behave exactly as they do today. -- Reading anything from database *names*. No year parsing, no naming convention. -- Making superstructure / scenario-array databases usable in `TimexLCA`. They are - recognised and skipped, not supported. -- Writing metadata on import of ecoinvent or any other database. Only the explicit - helper writes. -- Changing how a resolved `database_dates` mapping is used downstream. Everything - after resolution — timeline, temporal markets, matrix modification — is untouched. - -## Design - -### Public interface - -```python -TimexLCA( - demand: dict, - method: tuple, - database_dates: dict = None, - scenario: dict = None, - use_global_lci_cache: bool = True, -) -``` - -`scenario` is a mapping of database metadata key to required value. Any key that -appears in database metadata is allowed — `iam_model`, `pathway`, `system_model`, -`ecoinvent_version`, `premise_version`, and whatever premise adds later. It is a dict -rather than a set of explicit keywords so that the signature stays closed (a -misspelled `use_global_lci_cache` raises `TypeError` instead of being swallowed as a -filter), the call site reads as background selection, and one dict can be reused -across a comparison loop. - -```python -from bw_timex import set_database_metadata - -set_database_metadata("db_2030", representative_time=datetime(2030, 1, 1)) -set_database_metadata( - "my_2050_variant", - representative_time="2050-01-01", - iam_model="remind", - pathway="SSP2-PkBudg500", -) -``` - -### Resolution - -`TimexLCA.__init__` resolves `self.database_dates` before anything else, in -`_resolve_database_dates`. Two mutually exclusive branches: - -**`database_dates` given.** It is the whole mapping. Metadata is not read, `scenario` -must be `None` (passing both raises `ValueError`), and demand databases missing from -it raise in validation as they do today. This keeps every existing script -bit-for-bit unchanged: a legacy call in a project that also holds ten premise -vintages must not silently pull those ten in. - -**`database_dates` not given.** Resolve from metadata: - -1. **Candidates.** Every database in `bd.databases` whose metadata has a - `representative_time`. -2. **Skip multi-scenario databases.** A candidate that also has a non-empty - `scenarios` list is dropped with a `logger.info` naming it. `bw_timex` needs one - technosphere per point in time and cannot pick a scenario out of a superstructure - database. Such a database can still be used by naming it in `database_dates`. -3. **Filter by `scenario`.** A candidate is dropped only if it *declares* a filtered - key with a different value. A candidate that does not declare the key at all is - kept — a hand-built 2020 database, an untouched ecoinvent, or the foreground has no - `pathway`, and filtering it out would break every mixed setup. - `external_scenarios` (a list) compares order-insensitively as a set; all other - values compare with `==` after `str` coercion of both sides. -4. **Ambiguity check.** Over the surviving candidates that declare at least one - scenario key, build a signature from - `("iam_model", "pathway", "system_model", "ecoinvent_version", "external_scenarios")` - (missing key → `None`). Bookkeeping keys such as `premise_version` are deliberately - not part of the signature: re-running premise must not look like a second scenario. - More than one distinct signature raises `ValueError`, reporting only the keys whose - values actually differ: - - ``` - Several background scenarios found in this project: - pathway='SSP2-PkBudg500': ei310_remind_SSP2-PkBudg500_2030, - ei310_remind_SSP2-PkBudg500_2040, - ei310_remind_SSP2-PkBudg500_2050 - pathway='SSP2-Base': ei310_remind_SSP2-Base_2030, - ei310_remind_SSP2-Base_2040, - ei310_remind_SSP2-Base_2050 - Select one, e.g. scenario={'pathway': 'SSP2-PkBudg500'}, or map the databases - explicitly with database_dates. - ``` - - Databases that declare no scenario key at all never appear in this check and are - always kept. -5. **Normalize values.** `datetime` passes through; a string parses with - `datetime.fromisoformat`; the literal `"dynamic"` passes through. Anything else - raises `ValueError` naming the database, the key and the offending value. -6. **Demand databases.** Every database holding a demand key that is not already - mapped is added as `"dynamic"`. -7. **Nothing found.** If no database carries `representative_time`, log the existing - "no remapping will be done" message and fall back to today's behaviour: demand - databases marked `"dynamic"`. - -An unknown filter key — one that no candidate database declares — raises rather than -filtering everything away, listing the keys and values present in the project. That is -what buys back the autocomplete a dict does not give. - -### Module layout - -Discovery, filtering and the setter live in a new module, -`bw_timex/database_metadata.py`. `timex_lca.py` is already large and this is a -self-contained responsibility with its own tests; `utils.py` holds exchange- and -plot-level helpers. `TimexLCA` imports `resolve_database_dates_from_metadata` from it, -and `set_database_metadata` is re-exported from the `bw_timex` top-level namespace. - -### Setter helper - -`bw_timex.database_metadata.set_database_metadata(database, **metadata)`, re-exported -from `bw_timex`: - -- `database` may be a name or a `bd.Database`; unregistered → `ValueError`. -- `representative_time` accepts a `datetime` (serialized with `.isoformat()`), an ISO - string (validated by round-tripping through `fromisoformat`), or `"dynamic"`. - Brightway metadata is stored as JSON, so a `datetime` object left in it breaks - `bd.databases.flush()`; converting is the point of the helper. -- Any other key is written as given, after a JSON-serializability check. -- Writes into `bd.databases[name]` and calls `bd.databases.flush()`, so the value - survives a project reload. -- Returns the resulting metadata mapping. - -### Validation - -`TimexLCAInputs` gains `scenario: Optional[dict]`, validating that keys are strings -and values are scalars or lists of scalars, and that `scenario` and `database_dates` -are not both given. The metadata-side errors (unparseable value, ambiguity, unknown -filter key) are raised in `_resolve_database_dates`, which owns the metadata, not in -the pydantic model. - -`set_database_metadata` gets its own `DatabaseMetadataInputs` model in -`validation.py`, matching how the other user-facing helpers validate. - -## Interactions and limits - -- **Several databases per date** ([#205](https://github.com/brightway-lca/bw_timex/pull/205)) - still works: metadata discovery can map two databases to the same - `representative_time`, which is exactly the modified-copy case. Two full ecoinvent - copies of the same vintage (e.g. from two premise runs) collide on process identity - and raise there, as designed; the fix is a `scenario` filter on `premise_version` or - an explicit `database_dates`. -- **Setup cost.** `TimexLCA.__init__` loads node metadata for every database in - `database_dates`, so auto-discovery costs one node-metadata load per matching - database. A project holding vintages from an unrelated study pays for them; the - escape hatches are `scenario` or `database_dates`. -- **premise version.** The metadata is written by premise `master` (post-2.4.9.2). - Databases written by older premise carry nothing, and the docs say so; those users - either write metadata with the helper or keep using `database_dates`. - -## Documentation - -- `docs/content/getting_started/quickstart.md`: step 3 becomes "the databases already - know when they are"; `database_dates` shown once as the explicit alternative; the - cheat-sheet row for background timing updated. -- `docs/content/getting_started/adding_temporal_information.md` and - `build_process_timeline.md`: update the passages that name `database_dates`. -- New section in `docs/content/getting_started/` on what a database represents: - the metadata keys, `set_database_metadata`, scenario selection and its error, the - premise-version caveat, and `database_dates` as the explicit override. -- New `docs/api/database_metadata.md` (`::: bw_timex.database_metadata`), added to the - API nav in `zensical.toml`. -- `CHANGES.md`: entry under `[Unreleased]`. - -## Notebooks - -Every notebook that builds its own databases writes metadata with -`set_database_metadata` and drops the `database_dates` argument; the premise notebooks -rely on premise-written metadata and show `scenario` where a project holds more than -one pathway. - -- `notebooks/tutorials/1_getting_started.ipynb` -- `notebooks/tutorials/2_electric_vehicle_from_scratch.ipynb` -- `notebooks/tutorials/3_dynamic_characterization.ipynb` -- `notebooks/tutorials/4_import_model_from_excel.ipynb` -- `notebooks/advanced/background_temporal_distributions.ipynb` -- `notebooks/advanced/background_temporal_distributions_premise.ipynb` -- `notebooks/advanced/uncertainty_with_datapackages.ipynb` -- `notebooks/teaching/ev_walkthrough_premise.ipynb` -- `notebooks/teaching/exercise_ev_vs_petrol_solutions.ipynb` -- `notebooks/examples/electric_vehicle_premise.ipynb` -- `notebooks/examples/electric_vehicle_premise_detailed.ipynb` -- `notebooks/development/benchmarking.ipynb` - -`notebooks/examples/paper_case_study.ipynb` is **not** touched: it reproduces a -published study and must keep its exact code. - -## Testing - -New `tests/test_database_metadata.py`, on the existing small fixtures: - -- Timing resolved from `representative_time` metadata with no `database_dates`. -- ISO string and `datetime` metadata values both resolve; `"dynamic"` in metadata - marks a database dynamic; a garbage value raises naming the database. -- Demand database defaults to `"dynamic"` when its metadata says nothing. -- `database_dates` is exclusive: a project full of metadata-carrying databases plus an - explicit `database_dates` resolves to exactly that mapping. -- `database_dates` together with `scenario` raises. -- `scenario` filter selects one pathway out of two; databases without scenario - metadata survive the filter. -- Two scenario sets and no `scenario` raises, and the message names the differing key - and both values. -- Same scenario written by two premise versions does not raise (bookkeeping keys are - outside the signature). -- A database carrying `scenarios` is skipped, and named in the log. -- An unknown filter key raises listing the available keys. -- `set_database_metadata` round-trips through `bd.databases.flush()` and a re-read; - a `datetime` lands as an ISO string; an unregistered database raises. -- End-to-end: an existing scenario test rewritten to use metadata gives the same - score as the `database_dates` version. diff --git a/docs/superpowers/specs/2026-08-23-premise-scenario-autocreation-design.md b/docs/superpowers/specs/2026-08-23-premise-scenario-autocreation-design.md deleted file mode 100644 index d5f0c237..00000000 --- a/docs/superpowers/specs/2026-08-23-premise-scenario-autocreation-design.md +++ /dev/null @@ -1,282 +0,0 @@ -# Creating missing scenario databases with premise - -## Problem - -`TimexLCA(scenario={...})` selects background databases that already exist in the -project. If they do not, it raises and lists what the project actually holds: - -``` -scenario={'pathway': 'SSP2-PkBudg500'} matched no database in this project. -Values actually declared for its key(s) by this project's databases: -'pathway': ['SSP2-Base']. Check for a typo in the filter. -``` - -For a typo that is the right answer. For the other case — the user simply has not -built that scenario yet — it is a dead end that sends them to a premise notebook, -where they hand-write a `NewDatabase` call whose `model`, `pathway`, `system_model` -and `ecoinvent_version` must match, key for key, the `scenario` filter they will pass -to `TimexLCA` afterwards. The same facts are typed twice, in two libraries, and only -the second one checks them. - -## Goal - -One recipe, stated once, that either finds the vintages or builds them: - -```python -tlca = TimexLCA( - demand={("foreground", "ev"): 1}, - method=("EF v3.1", "climate change", "global warming potential (GWP100)"), - scenario={ - "iam_model": "remind", - "pathway": "SSP2-PkBudg500", - "system_model": "cutoff", - "ecoinvent_version": "3.10.1", - "years": [2020, 2030, 2040], - }, - create_missing=True, -) -``` - -Missing years are built with premise, missing ecoinvent is imported first, and the -resulting databases carry the metadata that makes them findable on the next run — so -the second call to the same code builds nothing. - -## Non-goals - -- Changing the default. `create_missing` defaults to `False`; without it the - behaviour is exactly today's, down to the error text (plus one added line saying - auto-building exists). -- Deferring `scenario` to `lci()`, or a settings-object / `run()` API. Everything - after `_resolve_database_dates` in `__init__` — base LCA, node collections, time - mapping — depends on the resolved database set, so resolution stays eager. A - lazier `TimexLCA` is a separate change. -- Making premise a hard dependency. It is an optional extra, imported only when a - build actually happens. -- Reproducing premise's own API surface. Only the arguments needed to build a - vintage set for a `TimexLCA` are exposed; anything more exotic (external - scenarios, `system_model_args`, incremental databases, custom IAM files) is built - in premise directly and then simply found by the scenario filter. -- Superstructure / scenario-array exports. Unchanged: recognised and skipped. - -## Design - -### Public interface - -```python -TimexLCA( - demand: dict, - method: tuple, - database_dates: dict = None, - scenario: dict = None, - create_missing: bool = False, - premise_key: str = None, - ecoinvent_credentials: tuple[str, str] = None, - use_global_lci_cache: bool = True, -) -``` - -and the same thing standalone, for users who would rather build first and calculate -later: - -```python -from bw_timex import ensure_scenario_databases - -ensure_scenario_databases(scenario, premise_key=None, ecoinvent_credentials=None) -``` - -`ensure_scenario_databases` returns the name → `datetime` mapping of the vintages it -found or built. `TimexLCA` ignores that return value and re-reads the metadata through -the existing `_resolve_database_dates`, so there is exactly one code path that decides -what a database represents. - -#### Build keys - -`scenario` gains three keys that describe *how to build*, as opposed to *what to -match*: - -| key | meaning | required | -|---|---|---| -| `years` | vintages to find or build, as integer years, e.g. `[2020, 2030, 2040]` — premise takes a year, not a date | yes, when `create_missing=True` | -| `sectors` | premise sectors to update; absent → all | no | -| `source_database` | name of the ecoinvent database to build from | no | - -They are stripped before the metadata filter runs. They must be: `years` is a list -and no database's `representative_time` could ever equal it, so leaving it in would -match nothing. The four filter keys (`iam_model`, `pathway`, `system_model`, -`ecoinvent_version`) keep their current meaning and are additionally what premise is -called with. - -Putting the years in `scenario` rather than in a separate argument keeps one object -describing one background: it can be built once and reused across a comparison loop, -and there is no way to pass years that disagree with the filter they belong to. - -### Resolution and build - -`ensure_scenario_databases`: - -1. **Split** `scenario` into filter keys and build keys. -2. **Match per year.** A year is satisfied by a registered database whose - `representative_time` falls in that year and that the scenario filter would keep. - "Would keep" is the resolver's own rule, not a stricter one: a database is kept - unless it *declares* a filter key with a different value, so a hand-built 2020 - vintage carrying only `representative_time` satisfies 2020 and is not rebuilt. - Sharing the rule with `resolve_database_dates_from_metadata` is what guarantees - that a database counted here is a database the resolver accepts afterwards — - any stricter match would build a second 2020 database that the resolver then - maps to the same date as the first. -3. **Early return.** No missing years → return, without importing premise. This is - the steady-state path: every re-run of a study takes it, and it must stay free. -4. **Source database.** `scenario["source_database"]` if given; otherwise - `ecoinvent-{ecoinvent_version}-{system_model}`, the name - `bw2io.import_ecoinvent_release` writes. Absent from the project → import it with - `bw2io.import_ecoinvent_release(version, system_model, username, password)`. The - biosphere name is derived the same way (`ecoinvent-{version}-biosphere`) and - passed to premise explicitly, since premise defaults to `"biosphere3"`, which a - namespaced ecoinvent import does not create. -5. **One premise run for every missing year.** - `NewDatabase(scenarios=[{"model": iam_model, "pathway": pathway, "year": y} for y - in missing], source_db=..., source_version=..., system_model=..., - biosphere_name=..., key=...)`. One run, not one per year: premise caches the - extracted source database, so N runs would re-extract ecoinvent N times. -6. **Update.** `ndb.update(sectors)` if `sectors` was given, else `ndb.update()`. -7. **Write.** `write_db_to_brightway(name=[...])` with deterministic names, - `ei_{system_model}_{ecoinvent_version}_{iam_model}_{pathway}_{year}`. -8. **Metadata.** premise >= 2.4.9.2 writes `representative_time`, `iam_model`, - `pathway`, `system_model`, `ecoinvent_version` and `premise_version` on export, so - nothing is duplicated here. Two additions only: - - `sectors` is written with `set_database_metadata` when it was narrowed, because - premise does not record it and two runs of the same pathway with different - sectors would otherwise be indistinguishable to the scenario filter. - - after the write, assert `representative_time` is present on each new database. - Missing → `RuntimeError` naming the premise pin. One check instead of - version-sniffing branches. - -**Overwrite guard.** Before step 5, every target name is checked against -`bd.databases`. A name that exists but whose metadata does not match the scenario -aborts the build with a `ValueError`: `write_db_to_brightway` deletes and rewrites a -colliding name silently, and that database is someone's multi-gigabyte work. A name -that exists *and* matches cannot occur — step 2 would have counted its year as -satisfied. - -**Announcement.** Before step 4, a `logger.info` states which years will be built, -which sectors, that each vintage is a full ecoinvent copy of roughly 2-4 GB, and that -this takes tens of minutes. The user asked for it, but not necessarily knowing that. - -### Module layout - -A new module, `bw_timex/scenario_builder.py`, holding `ensure_scenario_databases` and -the two seams below. `database_metadata.py` stays what its name says: reading and -writing metadata, no side effects, no third-party imports. `timex_lca.py` is already -2500 lines and gains only the argument plumbing. - -`ensure_scenario_databases` is re-exported from the `bw_timex` top-level namespace, -next to `set_database_metadata`. - -### Third-party seams - -All premise and bw2io contact happens in two module-level functions that do nothing -else: - -```python -def _run_premise(scenarios, source_database, source_version, system_model, - biosphere, sectors, names, key) -> None -def _import_ecoinvent(version, system_model, credentials) -> str -``` - -Both import their library inside the function body. Tests monkeypatch them (see -Testing). Keeping them free of logic is what makes the rest of the module testable -without premise installed. - -### Dependency - -```toml -[project.optional-dependencies] -premise = ["premise[bw25]>=2.4.9.2"] -``` - -2.4.9.2 is both the current PyPI release and the first that writes the metadata this -design relies on, so the floor costs nobody anything. - -The `[bw25]` extra is not optional: bare `premise` declares `bw2data` and `bw2io` -unpinned, and pins them to Brightway 2.5 versions (`bw2data>=4.3`, `bw2io>=0.9.4`) -only under that extra. Without it a resolver may satisfy them with the bw2-era pins -(`bw2data==3.6.6`), which contradicts bw_timex's `bw2data>=4.6`. `bw2io` arrives -through the same extra, so it needs no entry of its own. - -### Credentials - -Resolved in `ensure_scenario_databases`, once, before any expensive work: - -| value | argument | environment fallback | -|---|---|---| -| premise decryption key | `premise_key` | `PREMISE_KEY` | -| ecoinvent user | `ecoinvent_credentials[0]` | `ECOINVENT_USERNAME` | -| ecoinvent password | `ecoinvent_credentials[1]` | `ECOINVENT_PASSWORD` | - -An explicit argument always wins. The environment fallback exists so that notebooks -that get committed do not carry secrets in a cell; the explicit argument exists so -that a one-off interactive call does not require setting the environment first. -Ecoinvent credentials are read only on the branch that actually imports ecoinvent. - -### Validation and failure modes - -Everything below raises before premise is imported, and long before anything is -written: - -| condition | raises | -|---|---| -| premise not installed | `ImportError`: `pip install "bw_timex[premise]"` | -| `create_missing=True`, no `years` | `ValueError` | -| `create_missing=True` and `database_dates` given | `ValueError`, mirroring the existing `scenario` + `database_dates` rule | -| `create_missing=True`, scenario missing any of the four filter keys | `ValueError` naming them; premise needs all four | -| no premise key | `ValueError` naming `PREMISE_KEY` | -| ecoinvent absent and no credentials | `ValueError` naming `ECOINVENT_USERNAME` / `ECOINVENT_PASSWORD` | -| target name exists with foreign metadata | `ValueError`, nothing written | -| `representative_time` absent after write | `RuntimeError` naming the premise pin | -| `create_missing=False` and the filter matched nothing | today's `ValueError`, plus one line on how to auto-build | - -`create_missing=True` with `scenario=None` raises as well: there is no recipe to -build from. - -A new pydantic model in `validation.py`, `ScenarioBuildInputs`, carries the argument -checks, matching the existing `TimexLCAInputs` pattern. `TimexLCAInputs` gains -`create_missing`, `premise_key` and `ecoinvent_credentials`. - -## Testing - -premise cannot run in CI: it needs a decryption key, a licensed ecoinvent, and tens -of minutes and gigabytes per vintage. The two seams above are therefore monkeypatched -with fakes that register small in-memory databases carrying premise-style metadata. -Everything on this side of the seam — splitting, matching, guarding, credential -resolution, metadata — is exercised for real. - -- **Reuse**: all years already present → `_run_premise` never called, and `premise` - never enters `sys.modules`. -- **Partial**: years `[2020, 2030, 2040]` with 2030 present → `_run_premise` called - once, with exactly `[2020, 2040]`. -- **Sectors**: a narrowed list reaches the fake and lands in the new databases' - metadata; absent → the fake is called with `sectors=None` (meaning `update()`). -- **Metadata round-trip**: after a fake build, `TimexLCA.database_dates` resolves the - built databases to the right datetimes through the normal resolver. -- **Overwrite guard**: a colliding name with foreign metadata raises and - `_run_premise` is never called. -- **Source database**: an explicit `source_database` is passed through; a missing - ecoinvent triggers `_import_ecoinvent`; a present one does not. -- **Credentials**: explicit argument beats environment; missing values raise naming - the exact variable (with a monkeypatched `os.environ`). -- **Validation**: no `years`; missing filter keys; `create_missing` together with - `database_dates`; `create_missing` without `scenario`. -- **`ImportError`** when premise is absent, **`RuntimeError`** when the fake omits - `representative_time`. -- **Integration**: an existing fixture project plus `TimexLCA(..., create_missing=True)` - with faked seams completes `__init__` and calculates against the built vintages. - -A real premise run stays manual: one documented snippet, run by hand, recorded in the -docs. - -## Documentation - -- `docs/content/getting_started`: a short section on letting `TimexLCA` build the - background, next to the existing scenario-filter documentation. -- `installation.md`: the `bw_timex[premise]` extra and the two environment variables. -- `CHANGES.md`: one entry. diff --git a/notebooks/advanced/scenario_comparison.ipynb b/notebooks/advanced/scenario_comparison.ipynb index 0f069598..6817da3a 100644 --- a/notebooks/advanced/scenario_comparison.ipynb +++ b/notebooks/advanced/scenario_comparison.ipynb @@ -9,7 +9,7 @@ "\n", "`TimexLCASettings` holds everything one calculation needs - the demand, the\n", "method, the background selection, and every timeline/LCI/LCIA option - so one\n", - "object is also the record of what was run. `TimexLCA.from_settings(...).run()`\n", + "object is also the record of what was run. `TimexLCA(...).run()`\n", "executes it, `run()` can be called again with overrides, and\n", "`TimexLCA.compare()` runs a list of them into one table.\n", "\n", @@ -133,694 +133,287 @@ }, { "cell_type": "code", - "execution_count": 2, - "id": "7c2fa9ce", - "metadata": { - "execution": { - "iopub.execute_input": "2026-08-23T20:21:32.495604Z", - "iopub.status.busy": "2026-08-23T20:21:32.495482Z", - "iopub.status.idle": "2026-08-23T20:22:00.367566Z", - "shell.execute_reply": "2026-08-23T20:22:00.367115Z" - } - }, - "outputs": [ - { - "name": "stderr", - "output_type": "stream", - "text": [ - "\u001b[32m2026-08-23 22:21:32.496\u001b[0m | \u001b[1mINFO \u001b[0m | \u001b[36mbw_timex.timex_lca\u001b[0m:\u001b[36m__init__\u001b[0m:\u001b[36m275\u001b[0m - \u001b[1mInitializing TimexLCA object...\u001b[0m\n" - ] - }, - { - "name": "stderr", - "output_type": "stream", - "text": [ - "\u001b[32m2026-08-23 22:21:32.500\u001b[0m | \u001b[1mINFO \u001b[0m | \u001b[36mbw_timex.timex_lca\u001b[0m:\u001b[36m__init__\u001b[0m:\u001b[36m305\u001b[0m - \u001b[1mCalculating base LCA...\u001b[0m\n" - ] - }, - { - "name": "stderr", - "output_type": "stream", - "text": [ - "/Users/timodiepers/Documents/Coding/bw_timex/.venv/lib/python3.12/site-packages/scikits/umfpack/umfpack.py:737: UmfpackWarning: (almost) singular matrix! (estimated cond. number: 3.90e+13)\n", - " warnings.warn(msg, UmfpackWarning)\n", - "\u001b[32m2026-08-23 22:21:33.210\u001b[0m | \u001b[1mINFO \u001b[0m | \u001b[36mbw_timex.timex_lca\u001b[0m:\u001b[36m__init__\u001b[0m:\u001b[36m322\u001b[0m - \u001b[1mCollecting node infos...\u001b[0m\n" - ] - }, - { - "name": "stderr", - "output_type": "stream", - "text": [ - "\u001b[32m2026-08-23 22:21:33.286\u001b[0m | \u001b[1mINFO \u001b[0m | \u001b[36mbw_timex.timex_lca\u001b[0m:\u001b[36m__init__\u001b[0m:\u001b[36m334\u001b[0m - \u001b[1mLoading node metadata from 10 database(s)...\u001b[0m\n" - ] - }, - { - "name": "stderr", - "output_type": "stream", - "text": [ - "\u001b[32m2026-08-23 22:21:42.824\u001b[0m | \u001b[1mINFO \u001b[0m | \u001b[36mbw_timex.timex_lca\u001b[0m:\u001b[36m__init__\u001b[0m:\u001b[36m371\u001b[0m - \u001b[1mTimexLCA initialized.\u001b[0m\n" - ] - }, - { - "name": "stderr", - "output_type": "stream", - "text": [ - "\u001b[32m2026-08-23 22:21:42.824\u001b[0m | \u001b[1mINFO \u001b[0m | \u001b[36mbw_timex.timex_lca\u001b[0m:\u001b[36mrun\u001b[0m:\u001b[36m529\u001b[0m - \u001b[1mStarting TimexLCA.run() pipeline...\u001b[0m\n" - ] - }, - { - "name": "stderr", - "output_type": "stream", - "text": [ - "\u001b[32m2026-08-23 22:21:42.824\u001b[0m | \u001b[1mINFO \u001b[0m | \u001b[36mbw_timex.timex_lca\u001b[0m:\u001b[36mrun\u001b[0m:\u001b[36m536\u001b[0m - \u001b[1mStep 1/4: Building timeline...\u001b[0m\n" - ] - }, - { - "name": "stderr", - "output_type": "stream", - "text": [ - "\u001b[32m2026-08-23 22:21:42.825\u001b[0m | \u001b[1mINFO \u001b[0m | \u001b[36mbw_timex.timex_lca\u001b[0m:\u001b[36mbuild_timeline\u001b[0m:\u001b[36m938\u001b[0m - \u001b[1mNo edge filter function provided. Skipping all edges in background databases.\u001b[0m\n" - ] - }, - { - "name": "stderr", - "output_type": "stream", - "text": [ - "\u001b[32m2026-08-23 22:21:50.671\u001b[0m | \u001b[1mINFO \u001b[0m | \u001b[36mbw_timex.timex_lca\u001b[0m:\u001b[36mbuild_timeline\u001b[0m:\u001b[36m959\u001b[0m - \u001b[1mCreating activity time mapping...\u001b[0m\n" - ] - }, - { - "name": "stderr", - "output_type": "stream", - "text": [ - "\u001b[32m2026-08-23 22:21:50.839\u001b[0m | \u001b[1mINFO \u001b[0m | \u001b[36mbw_timex.timeline_builder\u001b[0m:\u001b[36m__init__\u001b[0m:\u001b[36m112\u001b[0m - \u001b[1mTraversing supply chain graph...\u001b[0m\n" - ] - }, - { - "name": "stdout", - "output_type": "stream", - "text": [ - "Starting graph traversal\n" - ] - }, - { - "name": "stderr", - "output_type": "stream", - "text": [ - "/Users/timodiepers/Documents/Coding/bw_timex/.venv/lib/python3.12/site-packages/bw_graph_tools/graph_traversal/new_node_each_visit.py:351: UserWarning: Graph traversal covered only 0.2% of the total LCA score. Consider lowering the `cutoff` (currently 1e-09) to improve coverage.\n", - " warnings.warn(\n", - "\u001b[32m2026-08-23 22:21:57.876\u001b[0m | \u001b[1mINFO \u001b[0m | \u001b[36mbw_timex.timeline_builder\u001b[0m:\u001b[36mbuild_timeline\u001b[0m:\u001b[36m183\u001b[0m - \u001b[1mBuilding timeline...\u001b[0m\n" - ] - }, - { - "name": "stderr", - "output_type": "stream", - "text": [ - "\u001b[32m2026-08-23 22:21:57.995\u001b[0m | \u001b[1mINFO \u001b[0m | \u001b[36mbw_timex.timeline_builder\u001b[0m:\u001b[36mget_weights_for_interpolation_between_nearest_years\u001b[0m:\u001b[36m808\u001b[0m - \u001b[1mReference date 2018-01-01 00:00:00 is lower than all provided dates. Data will be taken from the closest higher year.\u001b[0m\n" - ] - }, - { - "name": "stderr", - "output_type": "stream", - "text": [ - "\u001b[32m2026-08-23 22:21:58.026\u001b[0m | \u001b[1mINFO \u001b[0m | \u001b[36mbw_timex.timex_lca\u001b[0m:\u001b[36mrun\u001b[0m:\u001b[36m551\u001b[0m - \u001b[1mStep 2/4: Calculating LCI...\u001b[0m\n" - ] - }, - { - "name": "stdout", - "output_type": "stream", - "text": [ - "Calculation count: 103\n" - ] - }, - { - "name": "stderr", - "output_type": "stream", - "text": [ - "\u001b[32m2026-08-23 22:21:58.378\u001b[0m | \u001b[1mINFO \u001b[0m | \u001b[36mbw_timex.timex_lca\u001b[0m:\u001b[36mlci\u001b[0m:\u001b[36m1120\u001b[0m - \u001b[1mExpanding matrices...\u001b[0m\n" - ] - }, - { - "name": "stderr", - "output_type": "stream", - "text": [ - "\u001b[32m2026-08-23 22:21:58.390\u001b[0m | \u001b[1mINFO \u001b[0m | \u001b[36mbw_timex.timex_lca\u001b[0m:\u001b[36mlci\u001b[0m:\u001b[36m1139\u001b[0m - \u001b[1mCalculating dynamic inventory...\u001b[0m\n" - ] - }, - { - "name": "stderr", - "output_type": "stream", - "text": [ - "/Users/timodiepers/Documents/Coding/bw_timex/.venv/lib/python3.12/site-packages/scikits/umfpack/umfpack.py:737: UmfpackWarning: (almost) singular matrix! (estimated cond. number: 1.65e+13)\n", - " warnings.warn(msg, UmfpackWarning)\n", - "\u001b[32m2026-08-23 22:22:00.362\u001b[0m | \u001b[1mINFO \u001b[0m | \u001b[36mbw_timex.timex_lca\u001b[0m:\u001b[36mrun\u001b[0m:\u001b[36m560\u001b[0m - \u001b[1mStep 3/4: Calculating static LCIA...\u001b[0m\n" - ] - }, - { - "name": "stderr", - "output_type": "stream", - "text": [ - "\u001b[32m2026-08-23 22:22:00.365\u001b[0m | \u001b[1mINFO \u001b[0m | \u001b[36mbw_timex.timex_lca\u001b[0m:\u001b[36mrun\u001b[0m:\u001b[36m578\u001b[0m - \u001b[1mStep 4/4: Skipping dynamic LCIA (disabled).\u001b[0m\n" - ] - }, - { - "name": "stderr", - "output_type": "stream", - "text": [ - "\u001b[32m2026-08-23 22:22:00.365\u001b[0m | \u001b[1mINFO \u001b[0m | \u001b[36mbw_timex.timex_lca\u001b[0m:\u001b[36mrun\u001b[0m:\u001b[36m580\u001b[0m - \u001b[1mTimexLCA.run() completed successfully.\u001b[0m\n" - ] - }, - { - "name": "stdout", - "output_type": "stream", - "text": [ - "base score: 23247\n", - "static score: 25463\n" - ] - } - ], - "source": [ - "settings = TimexLCASettings(\n", - " demand=demand,\n", - " method=method,\n", - " starting_datetime=datetime(2020, 6, 1), # bought in 2020, driven from then on\n", - " build_dynamic_biosphere=False, # only the static score is needed here\n", - " dynamic_lcia_enabled=False,\n", - " label=\"bought 2020\",\n", - ")\n", - "\n", - "tlca = TimexLCA.from_settings(settings).run()\n", - "print(\"base score: \", round(tlca.base_score))\n", - "print(\"static score:\", round(tlca.static_score))" - ] - }, - { - "cell_type": "markdown", - "id": "664d1c58", - "metadata": {}, - "source": [ - "The time-explicit score is higher than the base score because the EV's\n", - "30,000 kWh are spread over its lifetime and resolved against the REMIND-EU\n", - "vintage each year of use falls into, rather than the 2020 database the\n", - "exchange nominally points at.\n", - "\n", - "The traversal's score-coverage warning is expected: it only measures the\n", - "*foreground* graph it walks, while most of this system's impact sits in the\n", - "background, which is resolved by solving the expanded matrix instead. The\n", - "score is unchanged across a wide range of `cutoff` values." - ] - }, - { - "cell_type": "markdown", - "id": "f724a7c0", - "metadata": {}, - "source": [ - "`run()` again on the same object, changing only what should change -\n", - "the settings object itself is left untouched, and the base LCA is reused:" - ] - }, - { - "cell_type": "code", - "execution_count": 3, - "id": "9200b3fd", - "metadata": { - "execution": { - "iopub.execute_input": "2026-08-23T20:22:00.368930Z", - "iopub.status.busy": "2026-08-23T20:22:00.368827Z", - "iopub.status.idle": "2026-08-23T20:22:07.094503Z", - "shell.execute_reply": "2026-08-23T20:22:07.094016Z" - } - }, - "outputs": [ - { - "name": "stderr", - "output_type": "stream", - "text": [ - "\u001b[32m2026-08-23 22:22:00.369\u001b[0m | \u001b[1mINFO \u001b[0m | \u001b[36mbw_timex.timex_lca\u001b[0m:\u001b[36mrun\u001b[0m:\u001b[36m529\u001b[0m - \u001b[1mStarting TimexLCA.run() pipeline...\u001b[0m\n" - ] - }, - { - "name": "stderr", - "output_type": "stream", - "text": [ - "\u001b[32m2026-08-23 22:22:00.369\u001b[0m | \u001b[1mINFO \u001b[0m | \u001b[36mbw_timex.timex_lca\u001b[0m:\u001b[36mrun\u001b[0m:\u001b[36m536\u001b[0m - \u001b[1mStep 1/4: Building timeline...\u001b[0m\n" - ] - }, - { - "name": "stderr", - "output_type": "stream", - "text": [ - "\u001b[32m2026-08-23 22:22:00.370\u001b[0m | \u001b[1mINFO \u001b[0m | \u001b[36mbw_timex.timex_lca\u001b[0m:\u001b[36mbuild_timeline\u001b[0m:\u001b[36m938\u001b[0m - \u001b[1mNo edge filter function provided. Skipping all edges in background databases.\u001b[0m\n" - ] - }, - { - "name": "stderr", - "output_type": "stream", - "text": [ - "\u001b[32m2026-08-23 22:22:00.370\u001b[0m | \u001b[1mINFO \u001b[0m | \u001b[36mbw_timex.timex_lca\u001b[0m:\u001b[36mbuild_timeline\u001b[0m:\u001b[36m959\u001b[0m - \u001b[1mCreating activity time mapping...\u001b[0m\n" - ] - }, - { - "name": "stderr", - "output_type": "stream", - "text": [ - "\u001b[32m2026-08-23 22:22:00.528\u001b[0m | \u001b[1mINFO \u001b[0m | \u001b[36mbw_timex.timeline_builder\u001b[0m:\u001b[36m__init__\u001b[0m:\u001b[36m112\u001b[0m - \u001b[1mTraversing supply chain graph...\u001b[0m\n" - ] - }, - { - "name": "stdout", - "output_type": "stream", - "text": [ - "Starting graph traversal\n" - ] - }, - { - "name": "stderr", - "output_type": "stream", - "text": [ - "/Users/timodiepers/Documents/Coding/bw_timex/.venv/lib/python3.12/site-packages/bw_graph_tools/graph_traversal/new_node_each_visit.py:351: UserWarning: Graph traversal covered only 0.2% of the total LCA score. Consider lowering the `cutoff` (currently 1e-09) to improve coverage.\n", - " warnings.warn(\n", - "\u001b[32m2026-08-23 22:22:03.698\u001b[0m | \u001b[1mINFO \u001b[0m | \u001b[36mbw_timex.timeline_builder\u001b[0m:\u001b[36mbuild_timeline\u001b[0m:\u001b[36m183\u001b[0m - \u001b[1mBuilding timeline...\u001b[0m\n" - ] - }, - { - "name": "stderr", - "output_type": "stream", - "text": [ - "\u001b[32m2026-08-23 22:22:03.805\u001b[0m | \u001b[1mINFO \u001b[0m | \u001b[36mbw_timex.timeline_builder\u001b[0m:\u001b[36mget_weights_for_interpolation_between_nearest_years\u001b[0m:\u001b[36m815\u001b[0m - \u001b[1mReference date 2073-01-01 00:00:00 is higher than all provided dates. Data will be taken from the closest lower year.\u001b[0m\n" - ] - }, - { - "name": "stderr", - "output_type": "stream", - "text": [ - "\u001b[32m2026-08-23 22:22:03.834\u001b[0m | \u001b[1mINFO \u001b[0m | \u001b[36mbw_timex.timex_lca\u001b[0m:\u001b[36mrun\u001b[0m:\u001b[36m551\u001b[0m - \u001b[1mStep 2/4: Calculating LCI...\u001b[0m\n" - ] - }, - { - "name": "stdout", - "output_type": "stream", - "text": [ - "Calculation count: 103\n" - ] - }, - { - "name": "stderr", - "output_type": "stream", - "text": [ - "\u001b[32m2026-08-23 22:22:04.308\u001b[0m | \u001b[1mINFO \u001b[0m | \u001b[36mbw_timex.timex_lca\u001b[0m:\u001b[36mlci\u001b[0m:\u001b[36m1120\u001b[0m - \u001b[1mExpanding matrices...\u001b[0m\n" - ] - }, - { - "name": "stderr", - "output_type": "stream", - "text": [ - "\u001b[32m2026-08-23 22:22:04.337\u001b[0m | \u001b[1mINFO \u001b[0m | \u001b[36mbw_timex.timex_lca\u001b[0m:\u001b[36mlci\u001b[0m:\u001b[36m1139\u001b[0m - \u001b[1mCalculating dynamic inventory...\u001b[0m\n" - ] - }, - { - "name": "stderr", - "output_type": "stream", - "text": [ - "/Users/timodiepers/Documents/Coding/bw_timex/.venv/lib/python3.12/site-packages/scikits/umfpack/umfpack.py:737: UmfpackWarning: (almost) singular matrix! (estimated cond. number: 2.10e+13)\n", - " warnings.warn(msg, UmfpackWarning)\n", - "\u001b[32m2026-08-23 22:22:07.089\u001b[0m | \u001b[1mINFO \u001b[0m | \u001b[36mbw_timex.timex_lca\u001b[0m:\u001b[36mrun\u001b[0m:\u001b[36m560\u001b[0m - \u001b[1mStep 3/4: Calculating static LCIA...\u001b[0m\n" - ] - }, - { - "name": "stderr", - "output_type": "stream", - "text": [ - "\u001b[32m2026-08-23 22:22:07.091\u001b[0m | \u001b[1mINFO \u001b[0m | \u001b[36mbw_timex.timex_lca\u001b[0m:\u001b[36mrun\u001b[0m:\u001b[36m578\u001b[0m - \u001b[1mStep 4/4: Skipping dynamic LCIA (disabled).\u001b[0m\n" - ] - }, - { - "name": "stderr", - "output_type": "stream", - "text": [ - "\u001b[32m2026-08-23 22:22:07.092\u001b[0m | \u001b[1mINFO \u001b[0m | \u001b[36mbw_timex.timex_lca\u001b[0m:\u001b[36mrun\u001b[0m:\u001b[36m580\u001b[0m - \u001b[1mTimexLCA.run() completed successfully.\u001b[0m\n" - ] - }, - { - "name": "stdout", - "output_type": "stream", - "text": [ - "static score, bought 2075: 8375\n", - "base LCA reused: True\n" - ] - } - ], - "source": [ - "base_lca_id = id(tlca.base_lca)\n", - "\n", - "tlca.run(starting_datetime=datetime(2075, 6, 1)) # same EV, bought decades later\n", - "print(\"static score, bought 2075:\", round(tlca.static_score))\n", - "print(\"base LCA reused:\", id(tlca.base_lca) == base_lca_id)" - ] - }, - { - "cell_type": "markdown", - "id": "27eeb73a", - "metadata": {}, - "source": [ - "A third of the 2020 purchase's impact, on the same EV - that is REMIND-EU's grid decarbonizing under the vehicle." - ] - }, - { - "cell_type": "markdown", - "id": "76cb0ec3", - "metadata": {}, - "source": [ - "## Several calculations: `compare()`" - ] - }, - { - "cell_type": "markdown", - "id": "2c8ee43d", - "metadata": {}, - "source": [ - "`compare()` takes a list of settings and returns a `ComparisonResult`\n", - "whose `summary` holds one row each - the scores next to every setting that\n", - "produced them, so the table is its own record of what was run. It builds one\n", - "`TimexLCA` per distinct background, so the purchase years below share a\n", - "single object:" - ] - }, - { - "cell_type": "code", - "execution_count": 4, - "id": "e328ddc0", - "metadata": { - "execution": { - "iopub.execute_input": "2026-08-23T20:22:07.096140Z", - "iopub.status.busy": "2026-08-23T20:22:07.096013Z", - "iopub.status.idle": "2026-08-23T20:22:39.490162Z", - "shell.execute_reply": "2026-08-23T20:22:39.489738Z" - } - }, - "outputs": [ - { - "name": "stderr", - "output_type": "stream", - "text": [ - "\u001b[32m2026-08-23 22:22:07.096\u001b[0m | \u001b[1mINFO \u001b[0m | \u001b[36mbw_timex.timex_lca\u001b[0m:\u001b[36m__init__\u001b[0m:\u001b[36m275\u001b[0m - \u001b[1mInitializing TimexLCA object...\u001b[0m\n" - ] - }, - { - "name": "stderr", - "output_type": "stream", - "text": [ - "\u001b[32m2026-08-23 22:22:07.098\u001b[0m | \u001b[1mINFO \u001b[0m | \u001b[36mbw_timex.timex_lca\u001b[0m:\u001b[36m__init__\u001b[0m:\u001b[36m305\u001b[0m - \u001b[1mCalculating base LCA...\u001b[0m\n" - ] - }, - { - "name": "stderr", - "output_type": "stream", - "text": [ - "/Users/timodiepers/Documents/Coding/bw_timex/.venv/lib/python3.12/site-packages/scikits/umfpack/umfpack.py:737: UmfpackWarning: (almost) singular matrix! (estimated cond. number: 3.90e+13)\n", - " warnings.warn(msg, UmfpackWarning)\n", - "\u001b[32m2026-08-23 22:22:07.769\u001b[0m | \u001b[1mINFO \u001b[0m | \u001b[36mbw_timex.timex_lca\u001b[0m:\u001b[36m__init__\u001b[0m:\u001b[36m322\u001b[0m - \u001b[1mCollecting node infos...\u001b[0m\n" - ] - }, - { - "name": "stderr", - "output_type": "stream", - "text": [ - "\u001b[32m2026-08-23 22:22:07.804\u001b[0m | \u001b[1mINFO \u001b[0m | \u001b[36mbw_timex.timex_lca\u001b[0m:\u001b[36m__init__\u001b[0m:\u001b[36m334\u001b[0m - \u001b[1mLoading node metadata from 10 database(s)...\u001b[0m\n" - ] - }, - { - "name": "stderr", - "output_type": "stream", - "text": [ - "\u001b[32m2026-08-23 22:22:07.874\u001b[0m | \u001b[1mINFO \u001b[0m | \u001b[36mbw_timex.timex_lca\u001b[0m:\u001b[36m__init__\u001b[0m:\u001b[36m371\u001b[0m - \u001b[1mTimexLCA initialized.\u001b[0m\n" - ] - }, - { - "name": "stderr", - "output_type": "stream", - "text": [ - "\u001b[32m2026-08-23 22:22:07.875\u001b[0m | \u001b[1mINFO \u001b[0m | \u001b[36mbw_timex.timex_lca\u001b[0m:\u001b[36mcompare\u001b[0m:\u001b[36m708\u001b[0m - \u001b[1mComparison 1/3: bought 2020\u001b[0m\n" - ] - }, - { - "name": "stderr", - "output_type": "stream", - "text": [ - "\u001b[32m2026-08-23 22:22:07.875\u001b[0m | \u001b[1mINFO \u001b[0m | \u001b[36mbw_timex.timex_lca\u001b[0m:\u001b[36mrun\u001b[0m:\u001b[36m529\u001b[0m - \u001b[1mStarting TimexLCA.run() pipeline...\u001b[0m\n" - ] - }, - { - "name": "stderr", - "output_type": "stream", - "text": [ - "\u001b[32m2026-08-23 22:22:07.875\u001b[0m | \u001b[1mINFO \u001b[0m | \u001b[36mbw_timex.timex_lca\u001b[0m:\u001b[36mrun\u001b[0m:\u001b[36m536\u001b[0m - \u001b[1mStep 1/4: Building timeline...\u001b[0m\n" - ] - }, - { - "name": "stderr", - "output_type": "stream", - "text": [ - "\u001b[32m2026-08-23 22:22:07.875\u001b[0m | \u001b[1mINFO \u001b[0m | \u001b[36mbw_timex.timex_lca\u001b[0m:\u001b[36mbuild_timeline\u001b[0m:\u001b[36m938\u001b[0m - \u001b[1mNo edge filter function provided. Skipping all edges in background databases.\u001b[0m\n" - ] - }, - { - "name": "stderr", - "output_type": "stream", - "text": [ - "\u001b[32m2026-08-23 22:22:15.590\u001b[0m | \u001b[1mINFO \u001b[0m | \u001b[36mbw_timex.timex_lca\u001b[0m:\u001b[36mbuild_timeline\u001b[0m:\u001b[36m959\u001b[0m - \u001b[1mCreating activity time mapping...\u001b[0m\n" - ] - }, - { - "name": "stderr", - "output_type": "stream", - "text": [ - "\u001b[32m2026-08-23 22:22:15.753\u001b[0m | \u001b[1mINFO \u001b[0m | \u001b[36mbw_timex.timeline_builder\u001b[0m:\u001b[36m__init__\u001b[0m:\u001b[36m112\u001b[0m - \u001b[1mTraversing supply chain graph...\u001b[0m\n" - ] - }, - { - "name": "stdout", - "output_type": "stream", - "text": [ - "Starting graph traversal\n" - ] - }, - { - "name": "stderr", - "output_type": "stream", - "text": [ - "/Users/timodiepers/Documents/Coding/bw_timex/.venv/lib/python3.12/site-packages/bw_graph_tools/graph_traversal/new_node_each_visit.py:351: UserWarning: Graph traversal covered only 0.2% of the total LCA score. Consider lowering the `cutoff` (currently 1e-09) to improve coverage.\n", - " warnings.warn(\n", - "\u001b[32m2026-08-23 22:22:22.780\u001b[0m | \u001b[1mINFO \u001b[0m | \u001b[36mbw_timex.timeline_builder\u001b[0m:\u001b[36mbuild_timeline\u001b[0m:\u001b[36m183\u001b[0m - \u001b[1mBuilding timeline...\u001b[0m\n" - ] - }, - { - "name": "stderr", - "output_type": "stream", - "text": [ - "\u001b[32m2026-08-23 22:22:22.883\u001b[0m | \u001b[1mINFO \u001b[0m | \u001b[36mbw_timex.timeline_builder\u001b[0m:\u001b[36mget_weights_for_interpolation_between_nearest_years\u001b[0m:\u001b[36m808\u001b[0m - \u001b[1mReference date 2018-01-01 00:00:00 is lower than all provided dates. Data will be taken from the closest higher year.\u001b[0m\n" - ] - }, - { - "name": "stderr", - "output_type": "stream", - "text": [ - "\u001b[32m2026-08-23 22:22:22.916\u001b[0m | \u001b[1mINFO \u001b[0m | \u001b[36mbw_timex.timex_lca\u001b[0m:\u001b[36mrun\u001b[0m:\u001b[36m551\u001b[0m - \u001b[1mStep 2/4: Calculating LCI...\u001b[0m\n" - ] - }, - { - "name": "stdout", - "output_type": "stream", - "text": [ - "Calculation count: 103\n" - ] - }, - { - "name": "stderr", - "output_type": "stream", - "text": [ - "\u001b[32m2026-08-23 22:22:23.269\u001b[0m | \u001b[1mINFO \u001b[0m | \u001b[36mbw_timex.timex_lca\u001b[0m:\u001b[36mlci\u001b[0m:\u001b[36m1120\u001b[0m - \u001b[1mExpanding matrices...\u001b[0m\n" - ] - }, - { - "name": "stderr", - "output_type": "stream", - "text": [ - "\u001b[32m2026-08-23 22:22:23.281\u001b[0m | \u001b[1mINFO \u001b[0m | \u001b[36mbw_timex.timex_lca\u001b[0m:\u001b[36mlci\u001b[0m:\u001b[36m1139\u001b[0m - \u001b[1mCalculating dynamic inventory...\u001b[0m\n" - ] - }, - { - "name": "stderr", - "output_type": "stream", - "text": [ - "/Users/timodiepers/Documents/Coding/bw_timex/.venv/lib/python3.12/site-packages/scikits/umfpack/umfpack.py:737: UmfpackWarning: (almost) singular matrix! (estimated cond. number: 1.65e+13)\n", - " warnings.warn(msg, UmfpackWarning)\n", - "\u001b[32m2026-08-23 22:22:25.261\u001b[0m | \u001b[1mINFO \u001b[0m | \u001b[36mbw_timex.timex_lca\u001b[0m:\u001b[36mrun\u001b[0m:\u001b[36m560\u001b[0m - \u001b[1mStep 3/4: Calculating static LCIA...\u001b[0m\n" - ] - }, - { - "name": "stderr", - "output_type": "stream", - "text": [ - "\u001b[32m2026-08-23 22:22:25.264\u001b[0m | \u001b[1mINFO \u001b[0m | \u001b[36mbw_timex.timex_lca\u001b[0m:\u001b[36mrun\u001b[0m:\u001b[36m578\u001b[0m - \u001b[1mStep 4/4: Skipping dynamic LCIA (disabled).\u001b[0m\n" - ] - }, - { - "name": "stderr", - "output_type": "stream", - "text": [ - "\u001b[32m2026-08-23 22:22:25.264\u001b[0m | \u001b[1mINFO \u001b[0m | \u001b[36mbw_timex.timex_lca\u001b[0m:\u001b[36mrun\u001b[0m:\u001b[36m580\u001b[0m - \u001b[1mTimexLCA.run() completed successfully.\u001b[0m\n" - ] - }, - { - "name": "stderr", - "output_type": "stream", - "text": [ - "\u001b[32m2026-08-23 22:22:25.265\u001b[0m | \u001b[1mINFO \u001b[0m | \u001b[36mbw_timex.timex_lca\u001b[0m:\u001b[36mcompare\u001b[0m:\u001b[36m708\u001b[0m - \u001b[1mComparison 2/3: bought 2040\u001b[0m\n" - ] - }, - { - "name": "stderr", - "output_type": "stream", - "text": [ - "\u001b[32m2026-08-23 22:22:25.265\u001b[0m | \u001b[1mINFO \u001b[0m | \u001b[36mbw_timex.timex_lca\u001b[0m:\u001b[36mrun\u001b[0m:\u001b[36m529\u001b[0m - \u001b[1mStarting TimexLCA.run() pipeline...\u001b[0m\n" - ] - }, - { - "name": "stderr", - "output_type": "stream", - "text": [ - "\u001b[32m2026-08-23 22:22:25.265\u001b[0m | \u001b[1mINFO \u001b[0m | \u001b[36mbw_timex.timex_lca\u001b[0m:\u001b[36mrun\u001b[0m:\u001b[36m536\u001b[0m - \u001b[1mStep 1/4: Building timeline...\u001b[0m\n" - ] - }, - { - "name": "stderr", - "output_type": "stream", - "text": [ - "\u001b[32m2026-08-23 22:22:25.266\u001b[0m | \u001b[1mINFO \u001b[0m | \u001b[36mbw_timex.timex_lca\u001b[0m:\u001b[36mbuild_timeline\u001b[0m:\u001b[36m938\u001b[0m - \u001b[1mNo edge filter function provided. Skipping all edges in background databases.\u001b[0m\n" - ] - }, - { - "name": "stderr", - "output_type": "stream", - "text": [ - "\u001b[32m2026-08-23 22:22:25.266\u001b[0m | \u001b[1mINFO \u001b[0m | \u001b[36mbw_timex.timex_lca\u001b[0m:\u001b[36mbuild_timeline\u001b[0m:\u001b[36m959\u001b[0m - \u001b[1mCreating activity time mapping...\u001b[0m\n" - ] - }, - { - "name": "stderr", - "output_type": "stream", - "text": [ - "\u001b[32m2026-08-23 22:22:25.428\u001b[0m | \u001b[1mINFO \u001b[0m | \u001b[36mbw_timex.timeline_builder\u001b[0m:\u001b[36m__init__\u001b[0m:\u001b[36m112\u001b[0m - \u001b[1mTraversing supply chain graph...\u001b[0m\n" - ] - }, - { - "name": "stdout", - "output_type": "stream", - "text": [ - "Starting graph traversal\n" - ] - }, - { - "name": "stderr", - "output_type": "stream", - "text": [ - "/Users/timodiepers/Documents/Coding/bw_timex/.venv/lib/python3.12/site-packages/bw_graph_tools/graph_traversal/new_node_each_visit.py:351: UserWarning: Graph traversal covered only 0.2% of the total LCA score. Consider lowering the `cutoff` (currently 1e-09) to improve coverage.\n", - " warnings.warn(\n", - "\u001b[32m2026-08-23 22:22:28.595\u001b[0m | \u001b[1mINFO \u001b[0m | \u001b[36mbw_timex.timeline_builder\u001b[0m:\u001b[36mbuild_timeline\u001b[0m:\u001b[36m183\u001b[0m - \u001b[1mBuilding timeline...\u001b[0m\n" - ] - }, - { - "name": "stderr", - "output_type": "stream", - "text": [ - "\u001b[32m2026-08-23 22:22:28.731\u001b[0m | \u001b[1mINFO \u001b[0m | \u001b[36mbw_timex.timex_lca\u001b[0m:\u001b[36mrun\u001b[0m:\u001b[36m551\u001b[0m - \u001b[1mStep 2/4: Calculating LCI...\u001b[0m\n" - ] - }, - { - "name": "stdout", - "output_type": "stream", - "text": [ - "Calculation count: 103\n" - ] - }, - { - "name": "stderr", - "output_type": "stream", - "text": [ - "\u001b[32m2026-08-23 22:22:29.318\u001b[0m | \u001b[1mINFO \u001b[0m | \u001b[36mbw_timex.timex_lca\u001b[0m:\u001b[36mlci\u001b[0m:\u001b[36m1120\u001b[0m - \u001b[1mExpanding matrices...\u001b[0m\n" - ] - }, + "execution_count": 18, + "id": "56eb0c58", + "metadata": {}, + "outputs": [ { "name": "stderr", "output_type": "stream", "text": [ - "\u001b[32m2026-08-23 22:22:29.356\u001b[0m | \u001b[1mINFO \u001b[0m | \u001b[36mbw_timex.timex_lca\u001b[0m:\u001b[36mlci\u001b[0m:\u001b[36m1139\u001b[0m - \u001b[1mCalculating dynamic inventory...\u001b[0m\n" + "\u001b[32m2026-08-24 11:52:07.954\u001b[0m | \u001b[1mINFO \u001b[0m | \u001b[36mbw_timex.timex_lca\u001b[0m:\u001b[36m__init__\u001b[0m:\u001b[36m421\u001b[0m - \u001b[1mInitializing TimexLCA object...\u001b[0m\n", + "\u001b[32m2026-08-24 11:52:07.971\u001b[0m | \u001b[1mINFO \u001b[0m | \u001b[36mbw_timex.timex_lca\u001b[0m:\u001b[36m__init__\u001b[0m:\u001b[36m508\u001b[0m - \u001b[1mCalculating base LCA...\u001b[0m\n", + "/Users/timodiepers/Documents/Coding/bw_timex/.venv/lib/python3.12/site-packages/scikits/umfpack/umfpack.py:737: UmfpackWarning: (almost) singular matrix! (estimated cond. number: 3.90e+13)\n", + " warnings.warn(msg, UmfpackWarning)\n", + "\u001b[32m2026-08-24 11:52:08.788\u001b[0m | \u001b[1mINFO \u001b[0m | \u001b[36mbw_timex.timex_lca\u001b[0m:\u001b[36m__init__\u001b[0m:\u001b[36m525\u001b[0m - \u001b[1mCollecting node infos...\u001b[0m\n", + "\u001b[32m2026-08-24 11:52:08.835\u001b[0m | \u001b[1mINFO \u001b[0m | \u001b[36mbw_timex.timex_lca\u001b[0m:\u001b[36m__init__\u001b[0m:\u001b[36m537\u001b[0m - \u001b[1mLoading node metadata from 10 database(s)...\u001b[0m\n", + "\u001b[32m2026-08-24 11:52:09.003\u001b[0m | \u001b[1mINFO \u001b[0m | \u001b[36mbw_timex.timex_lca\u001b[0m:\u001b[36m__init__\u001b[0m:\u001b[36m574\u001b[0m - \u001b[1mTimexLCA initialized.\u001b[0m\n", + "\u001b[32m2026-08-24 11:52:09.004\u001b[0m | \u001b[1mINFO \u001b[0m | \u001b[36mbw_timex.timex_lca\u001b[0m:\u001b[36mrun\u001b[0m:\u001b[36m738\u001b[0m - \u001b[1mStarting TimexLCA.run() pipeline...\u001b[0m\n", + "\u001b[32m2026-08-24 11:52:09.005\u001b[0m | \u001b[1mINFO \u001b[0m | \u001b[36mbw_timex.timex_lca\u001b[0m:\u001b[36mrun\u001b[0m:\u001b[36m745\u001b[0m - \u001b[1mStep 1/4: Building timeline...\u001b[0m\n", + "\u001b[32m2026-08-24 11:52:09.009\u001b[0m | \u001b[1mINFO \u001b[0m | \u001b[36mbw_timex.timex_lca\u001b[0m:\u001b[36mbuild_timeline\u001b[0m:\u001b[36m1177\u001b[0m - \u001b[1mNo edge filter function provided. Skipping all edges in background databases.\u001b[0m\n" + ] + }, + { + "ename": "KeyboardInterrupt", + "evalue": "", + "output_type": "error", + "traceback": [ + "\u001b[31m---------------------------------------------------------------------------\u001b[39m", + "\u001b[31mKeyboardInterrupt\u001b[39m Traceback (most recent call last)", + "\u001b[36mCell\u001b[39m\u001b[36m \u001b[39m\u001b[32mIn[18]\u001b[39m\u001b[32m, line 1\u001b[39m\n\u001b[32m----> \u001b[39m\u001b[32m1\u001b[39m \u001b[43mTimexLCA\u001b[49m\u001b[43m(\u001b[49m\u001b[43msettings\u001b[49m\u001b[43m)\u001b[49m\u001b[43m.\u001b[49m\u001b[43mrun\u001b[49m\u001b[43m(\u001b[49m\u001b[43m)\u001b[49m\n", + "\u001b[36mFile \u001b[39m\u001b[32m~/Documents/Coding/bw_timex/bw_timex/timex_lca.py:746\u001b[39m, in \u001b[36mTimexLCA.run\u001b[39m\u001b[34m(self, settings, **overrides)\u001b[39m\n\u001b[32m 744\u001b[39m \u001b[38;5;66;03m# Build timeline\u001b[39;00m\n\u001b[32m 745\u001b[39m logger.info(\u001b[33m\"\u001b[39m\u001b[33mStep 1/4: Building timeline...\u001b[39m\u001b[33m\"\u001b[39m)\n\u001b[32m--> \u001b[39m\u001b[32m746\u001b[39m \u001b[38;5;28;43mself\u001b[39;49m\u001b[43m.\u001b[49m\u001b[43mbuild_timeline\u001b[49m\u001b[43m(\u001b[49m\n\u001b[32m 747\u001b[39m \u001b[43m \u001b[49m\u001b[43mstarting_datetime\u001b[49m\u001b[43m=\u001b[49m\u001b[43msettings\u001b[49m\u001b[43m.\u001b[49m\u001b[43mstarting_datetime\u001b[49m\u001b[43m,\u001b[49m\n\u001b[32m 748\u001b[39m \u001b[43m \u001b[49m\u001b[43mtemporal_grouping\u001b[49m\u001b[43m=\u001b[49m\u001b[43msettings\u001b[49m\u001b[43m.\u001b[49m\u001b[43mtemporal_grouping\u001b[49m\u001b[43m,\u001b[49m\n\u001b[32m 749\u001b[39m \u001b[43m \u001b[49m\u001b[43minterpolation_type\u001b[49m\u001b[43m=\u001b[49m\u001b[43msettings\u001b[49m\u001b[43m.\u001b[49m\u001b[43minterpolation_type\u001b[49m\u001b[43m,\u001b[49m\n\u001b[32m 750\u001b[39m \u001b[43m \u001b[49m\u001b[43medge_filter_function\u001b[49m\u001b[43m=\u001b[49m\u001b[43msettings\u001b[49m\u001b[43m.\u001b[49m\u001b[43medge_filter_function\u001b[49m\u001b[43m,\u001b[49m\n\u001b[32m 751\u001b[39m \u001b[43m \u001b[49m\u001b[43mcutoff\u001b[49m\u001b[43m=\u001b[49m\u001b[43msettings\u001b[49m\u001b[43m.\u001b[49m\u001b[43mcutoff\u001b[49m\u001b[43m,\u001b[49m\n\u001b[32m 752\u001b[39m \u001b[43m \u001b[49m\u001b[43mmax_calc\u001b[49m\u001b[43m=\u001b[49m\u001b[43msettings\u001b[49m\u001b[43m.\u001b[49m\u001b[43mmax_calc\u001b[49m\u001b[43m,\u001b[49m\n\u001b[32m 753\u001b[39m \u001b[43m \u001b[49m\u001b[43mgraph_traversal\u001b[49m\u001b[43m=\u001b[49m\u001b[43msettings\u001b[49m\u001b[43m.\u001b[49m\u001b[43mgraph_traversal\u001b[49m\u001b[43m,\u001b[49m\n\u001b[32m 754\u001b[39m \u001b[43m \u001b[49m\u001b[43mtraverse_background\u001b[49m\u001b[43m=\u001b[49m\u001b[43msettings\u001b[49m\u001b[43m.\u001b[49m\u001b[43mtraverse_background\u001b[49m\u001b[43m,\u001b[49m\n\u001b[32m 755\u001b[39m \u001b[43m \u001b[49m\u001b[43m*\u001b[49m\u001b[43msettings\u001b[49m\u001b[43m.\u001b[49m\u001b[43mtimeline_args\u001b[49m\u001b[43m,\u001b[49m\n\u001b[32m 756\u001b[39m \u001b[43m \u001b[49m\u001b[43m*\u001b[49m\u001b[43m*\u001b[49m\u001b[43msettings\u001b[49m\u001b[43m.\u001b[49m\u001b[43mtimeline_kwargs\u001b[49m\u001b[43m,\u001b[49m\n\u001b[32m 757\u001b[39m \u001b[43m\u001b[49m\u001b[43m)\u001b[49m\n\u001b[32m 759\u001b[39m \u001b[38;5;66;03m# Calculate LCI\u001b[39;00m\n\u001b[32m 760\u001b[39m logger.info(\u001b[33m\"\u001b[39m\u001b[33mStep 2/4: Calculating LCI...\u001b[39m\u001b[33m\"\u001b[39m)\n", + "\u001b[36mFile \u001b[39m\u001b[32m~/Documents/Coding/bw_timex/bw_timex/timex_lca.py:1183\u001b[39m, in \u001b[36mTimexLCA.build_timeline\u001b[39m\u001b[34m(self, starting_datetime, temporal_grouping, interpolation_type, edge_filter_function, cutoff, max_calc, graph_traversal, traverse_background, *args, **kwargs)\u001b[39m\n\u001b[32m 1181\u001b[39m skippable = \u001b[38;5;28mset\u001b[39m()\n\u001b[32m 1182\u001b[39m \u001b[38;5;28;01mfor\u001b[39;00m db \u001b[38;5;129;01min\u001b[39;00m \u001b[38;5;28mself\u001b[39m.database_dates_static.keys():\n\u001b[32m-> \u001b[39m\u001b[32m1183\u001b[39m \u001b[43mskippable\u001b[49m\u001b[43m.\u001b[49m\u001b[43mupdate\u001b[49m\u001b[43m(\u001b[49m\u001b[43mnode\u001b[49m\u001b[43m.\u001b[49m\u001b[43mid\u001b[49m\u001b[43m \u001b[49m\u001b[38;5;28;43;01mfor\u001b[39;49;00m\u001b[43m \u001b[49m\u001b[43mnode\u001b[49m\u001b[43m \u001b[49m\u001b[38;5;129;43;01min\u001b[39;49;00m\u001b[43m \u001b[49m\u001b[43mbd\u001b[49m\u001b[43m.\u001b[49m\u001b[43mDatabase\u001b[49m\u001b[43m(\u001b[49m\u001b[43mdb\u001b[49m\u001b[43m)\u001b[49m\u001b[43m)\u001b[49m\n\u001b[32m 1184\u001b[39m \u001b[38;5;28mself\u001b[39m._default_edge_filter_function = skippable.\u001b[34m__contains__\u001b[39m\n\u001b[32m 1185\u001b[39m \u001b[38;5;28mself\u001b[39m.edge_filter_function = \u001b[38;5;28mself\u001b[39m._default_edge_filter_function\n", + "\u001b[36mFile \u001b[39m\u001b[32m~/Documents/Coding/bw_timex/bw_timex/timex_lca.py:1183\u001b[39m, in \u001b[36m\u001b[39m\u001b[34m(.0)\u001b[39m\n\u001b[32m 1181\u001b[39m skippable = \u001b[38;5;28mset\u001b[39m()\n\u001b[32m 1182\u001b[39m \u001b[38;5;28;01mfor\u001b[39;00m db \u001b[38;5;129;01min\u001b[39;00m \u001b[38;5;28mself\u001b[39m.database_dates_static.keys():\n\u001b[32m-> \u001b[39m\u001b[32m1183\u001b[39m skippable.update\u001b[43m(\u001b[49m\u001b[43mnode\u001b[49m\u001b[43m.\u001b[49m\u001b[43mid\u001b[49m\u001b[43m \u001b[49m\u001b[38;5;28;43;01mfor\u001b[39;49;00m\u001b[43m \u001b[49m\u001b[43mnode\u001b[49m\u001b[43m \u001b[49m\u001b[38;5;129;43;01min\u001b[39;49;00m\u001b[43m \u001b[49m\u001b[43mbd\u001b[49m\u001b[43m.\u001b[49m\u001b[43mDatabase\u001b[49m\u001b[43m(\u001b[49m\u001b[43mdb\u001b[49m\u001b[43m)\u001b[49m\u001b[43m)\u001b[49m\n\u001b[32m 1184\u001b[39m \u001b[38;5;28mself\u001b[39m._default_edge_filter_function = skippable.\u001b[34m__contains__\u001b[39m\n\u001b[32m 1185\u001b[39m \u001b[38;5;28mself\u001b[39m.edge_filter_function = \u001b[38;5;28mself\u001b[39m._default_edge_filter_function\n", + "\u001b[36mFile \u001b[39m\u001b[32m~/Documents/Coding/bw_timex/.venv/lib/python3.12/site-packages/bw2data/backends/base.py:493\u001b[39m, in \u001b[36mSQLiteBackend.__iter__\u001b[39m\u001b[34m(self)\u001b[39m\n\u001b[32m 492\u001b[39m \u001b[38;5;28;01mdef\u001b[39;00m\u001b[38;5;250m \u001b[39m\u001b[34m__iter__\u001b[39m(\u001b[38;5;28mself\u001b[39m):\n\u001b[32m--> \u001b[39m\u001b[32m493\u001b[39m \u001b[43m \u001b[49m\u001b[38;5;28;43;01mfor\u001b[39;49;00m\u001b[43m \u001b[49m\u001b[43mds\u001b[49m\u001b[43m \u001b[49m\u001b[38;5;129;43;01min\u001b[39;49;00m\u001b[43m \u001b[49m\u001b[38;5;28;43mself\u001b[39;49m\u001b[43m.\u001b[49m\u001b[43m_get_queryset\u001b[49m\u001b[43m(\u001b[49m\u001b[43m)\u001b[49m\u001b[43m:\u001b[49m\n\u001b[32m 494\u001b[39m \u001b[43m \u001b[49m\u001b[38;5;28;43;01myield\u001b[39;49;00m\u001b[43m \u001b[49m\u001b[38;5;28;43mself\u001b[39;49m\u001b[43m.\u001b[49m\u001b[43mnode_class\u001b[49m\u001b[43m(\u001b[49m\u001b[43mds\u001b[49m\u001b[43m)\u001b[49m\n", + "\u001b[36mFile \u001b[39m\u001b[32m~/Documents/Coding/bw_timex/.venv/lib/python3.12/site-packages/peewee.py:7417\u001b[39m, in \u001b[36mBaseModelSelect.__iter__\u001b[39m\u001b[34m(self)\u001b[39m\n\u001b[32m 7415\u001b[39m \u001b[38;5;28;01mdef\u001b[39;00m\u001b[38;5;250m \u001b[39m\u001b[34m__iter__\u001b[39m(\u001b[38;5;28mself\u001b[39m):\n\u001b[32m 7416\u001b[39m \u001b[38;5;28;01mif\u001b[39;00m \u001b[38;5;129;01mnot\u001b[39;00m \u001b[38;5;28mself\u001b[39m._cursor_wrapper:\n\u001b[32m-> \u001b[39m\u001b[32m7417\u001b[39m \u001b[38;5;28;43mself\u001b[39;49m\u001b[43m.\u001b[49m\u001b[43mexecute\u001b[49m\u001b[43m(\u001b[49m\u001b[43m)\u001b[49m\n\u001b[32m 7418\u001b[39m \u001b[38;5;28;01mreturn\u001b[39;00m \u001b[38;5;28miter\u001b[39m(\u001b[38;5;28mself\u001b[39m._cursor_wrapper)\n", + "\u001b[36mFile \u001b[39m\u001b[32m~/Documents/Coding/bw_timex/.venv/lib/python3.12/site-packages/peewee.py:2010\u001b[39m, in \u001b[36mdatabase_required..inner\u001b[39m\u001b[34m(self, database, *args, **kwargs)\u001b[39m\n\u001b[32m 2007\u001b[39m \u001b[38;5;28;01mif\u001b[39;00m \u001b[38;5;129;01mnot\u001b[39;00m database:\n\u001b[32m 2008\u001b[39m \u001b[38;5;28;01mraise\u001b[39;00m InterfaceError(\u001b[33m'\u001b[39m\u001b[33mQuery must be bound to a database in order \u001b[39m\u001b[33m'\u001b[39m\n\u001b[32m 2009\u001b[39m \u001b[33m'\u001b[39m\u001b[33mto call \u001b[39m\u001b[33m\"\u001b[39m\u001b[38;5;132;01m%s\u001b[39;00m\u001b[33m\"\u001b[39m\u001b[33m.\u001b[39m\u001b[33m'\u001b[39m % method.\u001b[34m__name__\u001b[39m)\n\u001b[32m-> \u001b[39m\u001b[32m2010\u001b[39m \u001b[38;5;28;01mreturn\u001b[39;00m \u001b[43mmethod\u001b[49m\u001b[43m(\u001b[49m\u001b[38;5;28;43mself\u001b[39;49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43mdatabase\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43m*\u001b[49m\u001b[43margs\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43m*\u001b[49m\u001b[43m*\u001b[49m\u001b[43mkwargs\u001b[49m\u001b[43m)\u001b[49m\n", + "\u001b[36mFile \u001b[39m\u001b[32m~/Documents/Coding/bw_timex/.venv/lib/python3.12/site-packages/peewee.py:2081\u001b[39m, in \u001b[36mBaseQuery.execute\u001b[39m\u001b[34m(self, database)\u001b[39m\n\u001b[32m 2079\u001b[39m \u001b[38;5;129m@database_required\u001b[39m\n\u001b[32m 2080\u001b[39m \u001b[38;5;28;01mdef\u001b[39;00m\u001b[38;5;250m \u001b[39m\u001b[34mexecute\u001b[39m(\u001b[38;5;28mself\u001b[39m, database):\n\u001b[32m-> \u001b[39m\u001b[32m2081\u001b[39m \u001b[38;5;28;01mreturn\u001b[39;00m \u001b[38;5;28;43mself\u001b[39;49m\u001b[43m.\u001b[49m\u001b[43m_execute\u001b[49m\u001b[43m(\u001b[49m\u001b[43mdatabase\u001b[49m\u001b[43m)\u001b[49m\n", + "\u001b[36mFile \u001b[39m\u001b[32m~/Documents/Coding/bw_timex/.venv/lib/python3.12/site-packages/peewee.py:2254\u001b[39m, in \u001b[36mSelectBase._execute\u001b[39m\u001b[34m(self, database)\u001b[39m\n\u001b[32m 2252\u001b[39m \u001b[38;5;28;01mdef\u001b[39;00m\u001b[38;5;250m \u001b[39m\u001b[34m_execute\u001b[39m(\u001b[38;5;28mself\u001b[39m, database):\n\u001b[32m 2253\u001b[39m \u001b[38;5;28;01mif\u001b[39;00m \u001b[38;5;28mself\u001b[39m._cursor_wrapper \u001b[38;5;129;01mis\u001b[39;00m \u001b[38;5;28;01mNone\u001b[39;00m:\n\u001b[32m-> \u001b[39m\u001b[32m2254\u001b[39m cursor = \u001b[43mdatabase\u001b[49m\u001b[43m.\u001b[49m\u001b[43mexecute\u001b[49m\u001b[43m(\u001b[49m\u001b[38;5;28;43mself\u001b[39;49m\u001b[43m)\u001b[49m\n\u001b[32m 2255\u001b[39m \u001b[38;5;28mself\u001b[39m._cursor_wrapper = \u001b[38;5;28mself\u001b[39m._get_cursor_wrapper(cursor)\n\u001b[32m 2256\u001b[39m \u001b[38;5;28;01mreturn\u001b[39;00m \u001b[38;5;28mself\u001b[39m._cursor_wrapper\n", + "\u001b[36mFile \u001b[39m\u001b[32m~/Documents/Coding/bw_timex/.venv/lib/python3.12/site-packages/peewee.py:3312\u001b[39m, in \u001b[36mDatabase.execute\u001b[39m\u001b[34m(self, query, **context_options)\u001b[39m\n\u001b[32m 3310\u001b[39m ctx = \u001b[38;5;28mself\u001b[39m.get_sql_context(**context_options)\n\u001b[32m 3311\u001b[39m sql, params = ctx.sql(query).query()\n\u001b[32m-> \u001b[39m\u001b[32m3312\u001b[39m \u001b[38;5;28;01mreturn\u001b[39;00m \u001b[38;5;28;43mself\u001b[39;49m\u001b[43m.\u001b[49m\u001b[43mexecute_sql\u001b[49m\u001b[43m(\u001b[49m\u001b[43msql\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43mparams\u001b[49m\u001b[43m)\u001b[49m\n", + "\u001b[36mFile \u001b[39m\u001b[32m~/Documents/Coding/bw_timex/.venv/lib/python3.12/site-packages/peewee.py:3306\u001b[39m, in \u001b[36mDatabase.execute_sql\u001b[39m\u001b[34m(self, sql, params)\u001b[39m\n\u001b[32m 3304\u001b[39m \u001b[38;5;28;01mwith\u001b[39;00m __exception_wrapper__:\n\u001b[32m 3305\u001b[39m cursor = \u001b[38;5;28mself\u001b[39m.cursor()\n\u001b[32m-> \u001b[39m\u001b[32m3306\u001b[39m \u001b[43mcursor\u001b[49m\u001b[43m.\u001b[49m\u001b[43mexecute\u001b[49m\u001b[43m(\u001b[49m\u001b[43msql\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43mparams\u001b[49m\u001b[43m \u001b[49m\u001b[38;5;129;43;01mor\u001b[39;49;00m\u001b[43m \u001b[49m\u001b[43m(\u001b[49m\u001b[43m)\u001b[49m\u001b[43m)\u001b[49m\n\u001b[32m 3307\u001b[39m \u001b[38;5;28;01mreturn\u001b[39;00m cursor\n", + "\u001b[31mKeyboardInterrupt\u001b[39m: " ] - }, + } + ], + "source": [ + "TimexLCA(settings).run()" + ] + }, + { + "cell_type": "code", + "execution_count": 19, + "id": "7c2fa9ce", + "metadata": { + "execution": { + "iopub.execute_input": "2026-08-23T20:21:32.495604Z", + "iopub.status.busy": "2026-08-23T20:21:32.495482Z", + "iopub.status.idle": "2026-08-23T20:22:00.367566Z", + "shell.execute_reply": "2026-08-23T20:22:00.367115Z" + } + }, + "outputs": [ { "name": "stderr", "output_type": "stream", "text": [ - "/Users/timodiepers/Documents/Coding/bw_timex/.venv/lib/python3.12/site-packages/scikits/umfpack/umfpack.py:737: UmfpackWarning: (almost) singular matrix! (estimated cond. number: 9.21e+13)\n", - " warnings.warn(msg, UmfpackWarning)\n", - "\u001b[32m2026-08-23 22:22:32.637\u001b[0m | \u001b[1mINFO \u001b[0m | \u001b[36mbw_timex.timex_lca\u001b[0m:\u001b[36mrun\u001b[0m:\u001b[36m560\u001b[0m - \u001b[1mStep 3/4: Calculating static LCIA...\u001b[0m\n" + "\u001b[32m2026-08-24 11:52:25.188\u001b[0m | \u001b[1mINFO \u001b[0m | \u001b[36mbw_timex.timex_lca\u001b[0m:\u001b[36m__init__\u001b[0m:\u001b[36m421\u001b[0m - \u001b[1mInitializing TimexLCA object...\u001b[0m\n" ] }, { - "name": "stderr", - "output_type": "stream", - "text": [ - "\u001b[32m2026-08-23 22:22:32.639\u001b[0m | \u001b[1mINFO \u001b[0m | \u001b[36mbw_timex.timex_lca\u001b[0m:\u001b[36mrun\u001b[0m:\u001b[36m578\u001b[0m - \u001b[1mStep 4/4: Skipping dynamic LCIA (disabled).\u001b[0m\n" + "ename": "ValueError", + "evalue": "No database in this project declares the metadata key(s) ['ecoinvent_version', 'iam_model', 'pathway', 'system_model']. Keys declared by the databases of this project: overwrite, read_only, representative_time. Add the metadata with `bw_timex.set_database_metadata`, check the spelling of your `scenario` filter, or build the databases with premise - `bw_timex.ensure_scenario_databases(scenario)`, or `TimexLCA(..., create_missing=True)`, both with a `years` list in the scenario.", + "output_type": "error", + "traceback": [ + "\u001b[31m---------------------------------------------------------------------------\u001b[39m", + "\u001b[31mValueError\u001b[39m Traceback (most recent call last)", + "\u001b[36mCell\u001b[39m\u001b[36m \u001b[39m\u001b[32mIn[19]\u001b[39m\u001b[32m, line 22\u001b[39m\n\u001b[32m 1\u001b[39m settings = TimexLCASettings(\n\u001b[32m 2\u001b[39m demand=demand,\n\u001b[32m 3\u001b[39m method=method, \n\u001b[32m (...)\u001b[39m\u001b[32m 19\u001b[39m \n\u001b[32m 20\u001b[39m )\n\u001b[32m---> \u001b[39m\u001b[32m22\u001b[39m tlca = \u001b[43mTimexLCA\u001b[49m\u001b[43m(\u001b[49m\u001b[43msettings\u001b[49m\u001b[43m)\u001b[49m.run()\n\u001b[32m 23\u001b[39m \u001b[38;5;28mprint\u001b[39m(\u001b[33m\"\u001b[39m\u001b[33mbase score: \u001b[39m\u001b[33m\"\u001b[39m, \u001b[38;5;28mround\u001b[39m(tlca.base_score))\n\u001b[32m 24\u001b[39m \u001b[38;5;28mprint\u001b[39m(\u001b[33m\"\u001b[39m\u001b[33mstatic score:\u001b[39m\u001b[33m\"\u001b[39m, \u001b[38;5;28mround\u001b[39m(tlca.static_score))\n", + "\u001b[36mFile \u001b[39m\u001b[32m~/Documents/Coding/bw_timex/bw_timex/timex_lca.py:475\u001b[39m, in \u001b[36mTimexLCA.__init__\u001b[39m\u001b[34m(self, demand, method, database_dates, scenario, create_missing, premise_key, ecoinvent_credentials, use_global_lci_cache)\u001b[39m\n\u001b[32m 467\u001b[39m \u001b[38;5;28;01mfrom\u001b[39;00m\u001b[38;5;250m \u001b[39m\u001b[34;01m.\u001b[39;00m\u001b[34;01mscenario_builder\u001b[39;00m\u001b[38;5;250m \u001b[39m\u001b[38;5;28;01mimport\u001b[39;00m ensure_scenario_databases\n\u001b[32m 469\u001b[39m ensure_scenario_databases(\n\u001b[32m 470\u001b[39m scenario,\n\u001b[32m 471\u001b[39m premise_key=premise_key,\n\u001b[32m 472\u001b[39m ecoinvent_credentials=ecoinvent_credentials,\n\u001b[32m 473\u001b[39m )\n\u001b[32m--> \u001b[39m\u001b[32m475\u001b[39m \u001b[38;5;28mself\u001b[39m.database_dates = \u001b[38;5;28;43mself\u001b[39;49m\u001b[43m.\u001b[49m\u001b[43m_resolve_database_dates\u001b[49m\u001b[43m(\u001b[49m\n\u001b[32m 476\u001b[39m \u001b[43m \u001b[49m\u001b[43mdemand\u001b[49m\u001b[43m=\u001b[49m\u001b[43mdemand\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43mdatabase_dates\u001b[49m\u001b[43m=\u001b[49m\u001b[43mdatabase_dates\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43mscenario\u001b[49m\u001b[43m=\u001b[49m\u001b[43mscenario\u001b[49m\n\u001b[32m 477\u001b[39m \u001b[43m\u001b[49m\u001b[43m)\u001b[49m\n\u001b[32m 479\u001b[39m \u001b[38;5;66;03m# Settings this object was built from, if any, and the raw values of the\u001b[39;00m\n\u001b[32m 480\u001b[39m \u001b[38;5;66;03m# fields that pick the background. Kept as passed (not as resolved), so\u001b[39;00m\n\u001b[32m 481\u001b[39m \u001b[38;5;66;03m# `run` can tell whether a settings object asks for the same background.\u001b[39;00m\n\u001b[32m 482\u001b[39m \u001b[38;5;28mself\u001b[39m.settings = settings\n", + "\u001b[36mFile \u001b[39m\u001b[32m~/Documents/Coding/bw_timex/bw_timex/timex_lca.py:1013\u001b[39m, in \u001b[36mTimexLCA._resolve_database_dates\u001b[39m\u001b[34m(demand, database_dates, scenario)\u001b[39m\n\u001b[32m 1006\u001b[39m \u001b[38;5;28;01mraise\u001b[39;00m \u001b[38;5;167;01mValueError\u001b[39;00m(\n\u001b[32m 1007\u001b[39m \u001b[33m\"\u001b[39m\u001b[33m`scenario` selects background databases by their metadata and \u001b[39m\u001b[33m\"\u001b[39m\n\u001b[32m 1008\u001b[39m \u001b[33m\"\u001b[39m\u001b[33monly applies when `database_dates` is not given. Pass one or \u001b[39m\u001b[33m\"\u001b[39m\n\u001b[32m 1009\u001b[39m \u001b[33m\"\u001b[39m\u001b[33mthe other.\u001b[39m\u001b[33m\"\u001b[39m\n\u001b[32m 1010\u001b[39m )\n\u001b[32m 1011\u001b[39m \u001b[38;5;28;01mreturn\u001b[39;00m \u001b[38;5;28mdict\u001b[39m(database_dates)\n\u001b[32m-> \u001b[39m\u001b[32m1013\u001b[39m resolved = \u001b[43mresolve_database_dates_from_metadata\u001b[49m\u001b[43m(\u001b[49m\u001b[43mscenario\u001b[49m\u001b[43m)\u001b[49m\n\u001b[32m 1015\u001b[39m \u001b[38;5;66;03m# Only the filter keys are matched against metadata; the build keys\u001b[39;00m\n\u001b[32m 1016\u001b[39m \u001b[38;5;66;03m# (`years`, `sectors`, ...) describe what to build and are never\u001b[39;00m\n\u001b[32m 1017\u001b[39m \u001b[38;5;66;03m# declared by any database, so reporting them as unmatched metadata\u001b[39;00m\n\u001b[32m 1018\u001b[39m \u001b[38;5;66;03m# would send the user looking for a key that cannot exist.\u001b[39;00m\n\u001b[32m 1019\u001b[39m filters, _ = split_scenario(scenario)\n", + "\u001b[36mFile \u001b[39m\u001b[32m~/Documents/Coding/bw_timex/bw_timex/database_metadata.py:361\u001b[39m, in \u001b[36mresolve_database_dates_from_metadata\u001b[39m\u001b[34m(scenario)\u001b[39m\n\u001b[32m 359\u001b[39m scenario, _ = split_scenario(scenario)\n\u001b[32m 360\u001b[39m \u001b[38;5;28;01mif\u001b[39;00m scenario:\n\u001b[32m--> \u001b[39m\u001b[32m361\u001b[39m \u001b[43m_check_filter_keys\u001b[49m\u001b[43m(\u001b[49m\u001b[43mscenario\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43mcandidates\u001b[49m\u001b[43m)\u001b[49m\n\u001b[32m 362\u001b[39m candidates = {\n\u001b[32m 363\u001b[39m name: metadata\n\u001b[32m 364\u001b[39m \u001b[38;5;28;01mfor\u001b[39;00m name, metadata \u001b[38;5;129;01min\u001b[39;00m candidates.items()\n\u001b[32m 365\u001b[39m \u001b[38;5;28;01mif\u001b[39;00m database_matches_scenario(metadata, scenario)\n\u001b[32m 366\u001b[39m }\n\u001b[32m 367\u001b[39m _check_unambiguous(candidates)\n", + "\u001b[36mFile \u001b[39m\u001b[32m~/Documents/Coding/bw_timex/bw_timex/database_metadata.py:238\u001b[39m, in \u001b[36m_check_filter_keys\u001b[39m\u001b[34m(scenario, candidates)\u001b[39m\n\u001b[32m 236\u001b[39m \u001b[38;5;28;01mreturn\u001b[39;00m\n\u001b[32m 237\u001b[39m available = \u001b[33m\"\u001b[39m\u001b[33m, \u001b[39m\u001b[33m\"\u001b[39m.join(\u001b[38;5;28msorted\u001b[39m(declared)) \u001b[38;5;129;01mor\u001b[39;00m \u001b[33m\"\u001b[39m\u001b[33mnone\u001b[39m\u001b[33m\"\u001b[39m\n\u001b[32m--> \u001b[39m\u001b[32m238\u001b[39m \u001b[38;5;28;01mraise\u001b[39;00m \u001b[38;5;167;01mValueError\u001b[39;00m(\n\u001b[32m 239\u001b[39m \u001b[33mf\u001b[39m\u001b[33m\"\u001b[39m\u001b[33mNo database in this project declares the metadata key(s) \u001b[39m\u001b[33m\"\u001b[39m\n\u001b[32m 240\u001b[39m \u001b[33mf\u001b[39m\u001b[33m\"\u001b[39m\u001b[38;5;132;01m{\u001b[39;00munknown\u001b[38;5;132;01m}\u001b[39;00m\u001b[33m. Keys declared by the databases of this project: \u001b[39m\u001b[38;5;132;01m{\u001b[39;00mavailable\u001b[38;5;132;01m}\u001b[39;00m\u001b[33m. \u001b[39m\u001b[33m\"\u001b[39m\n\u001b[32m 241\u001b[39m \u001b[33mf\u001b[39m\u001b[33m\"\u001b[39m\u001b[33mAdd the metadata with `bw_timex.set_database_metadata`, check the \u001b[39m\u001b[33m\"\u001b[39m\n\u001b[32m 242\u001b[39m \u001b[33mf\u001b[39m\u001b[33m\"\u001b[39m\u001b[33mspelling of your `scenario` filter, or build the databases with \u001b[39m\u001b[33m\"\u001b[39m\n\u001b[32m 243\u001b[39m \u001b[33mf\u001b[39m\u001b[33m\"\u001b[39m\u001b[33mpremise - `bw_timex.ensure_scenario_databases(scenario)`, or \u001b[39m\u001b[33m\"\u001b[39m\n\u001b[32m 244\u001b[39m \u001b[33mf\u001b[39m\u001b[33m\"\u001b[39m\u001b[33m`TimexLCA(..., create_missing=True)`, both with a `years` list in \u001b[39m\u001b[33m\"\u001b[39m\n\u001b[32m 245\u001b[39m \u001b[33mf\u001b[39m\u001b[33m\"\u001b[39m\u001b[33mthe scenario.\u001b[39m\u001b[33m\"\u001b[39m\n\u001b[32m 246\u001b[39m )\n", + "\u001b[31mValueError\u001b[39m: No database in this project declares the metadata key(s) ['ecoinvent_version', 'iam_model', 'pathway', 'system_model']. Keys declared by the databases of this project: overwrite, read_only, representative_time. Add the metadata with `bw_timex.set_database_metadata`, check the spelling of your `scenario` filter, or build the databases with premise - `bw_timex.ensure_scenario_databases(scenario)`, or `TimexLCA(..., create_missing=True)`, both with a `years` list in the scenario." ] - }, + } + ], + "source": [ + "\n", + "settings = TimexLCASettings(\n", + " demand=demand,\n", + " method=method, \n", + " scenario={\n", + " \"iam_model\": \"remind\",\n", + " \"pathway\": \"SSP2-NDC\",\n", + " \"system_model\": \"cutoff\",\n", + " \"ecoinvent_version\": \"3.12\",\n", + " \"years\": [2020, 2030, 2040, 2050, 2075, 2100],\n", + " },\n", + " timeline={\"starting_datetime\": datetime(2020, 6, 1), \"graph_traversal\": \"bfs\"}, # bought in 2020, driven from then on\n", + " lci={\n", + " \"build_dynamic_biosphere\": False, # only the static score is needed here\n", + " },\n", + " lcia={\n", + " \"dynamic_lcia_enabled\": False\n", + " },\n", + " label=\"bought 2020\",\n", + " \n", + ")\n", + "\n", + "tlca = TimexLCA(settings).run()\n", + "print(\"base score: \", round(tlca.base_score))\n", + "print(\"static score:\", round(tlca.static_score))" + ] + }, + { + "cell_type": "markdown", + "id": "664d1c58", + "metadata": {}, + "source": [ + "The time-explicit score is higher than the base score because the EV's\n", + "30,000 kWh are spread over its lifetime and resolved against the REMIND-EU\n", + "vintage each year of use falls into, rather than the 2020 database the\n", + "exchange nominally points at.\n", + "\n", + "The traversal's score-coverage warning is expected: it only measures the\n", + "*foreground* graph it walks, while most of this system's impact sits in the\n", + "background, which is resolved by solving the expanded matrix instead. The\n", + "score is unchanged across a wide range of `cutoff` values." + ] + }, + { + "cell_type": "markdown", + "id": "f724a7c0", + "metadata": {}, + "source": [ + "`run()` again on the same object, changing only what should change -\n", + "the settings object itself is left untouched, and the base LCA is reused:" + ] + }, + { + "cell_type": "code", + "execution_count": 4, + "id": "9200b3fd", + "metadata": { + "execution": { + "iopub.execute_input": "2026-08-23T20:22:00.368930Z", + "iopub.status.busy": "2026-08-23T20:22:00.368827Z", + "iopub.status.idle": "2026-08-23T20:22:07.094503Z", + "shell.execute_reply": "2026-08-23T20:22:07.094016Z" + } + }, + "outputs": [ { "name": "stderr", "output_type": "stream", "text": [ - "\u001b[32m2026-08-23 22:22:32.640\u001b[0m | \u001b[1mINFO \u001b[0m | \u001b[36mbw_timex.timex_lca\u001b[0m:\u001b[36mrun\u001b[0m:\u001b[36m580\u001b[0m - \u001b[1mTimexLCA.run() completed successfully.\u001b[0m\n" + "\u001b[32m2026-08-24 09:46:12.251\u001b[0m | \u001b[1mINFO \u001b[0m | \u001b[36mbw_timex.timex_lca\u001b[0m:\u001b[36mrun\u001b[0m:\u001b[36m738\u001b[0m - \u001b[1mStarting TimexLCA.run() pipeline...\u001b[0m\n", + "\u001b[32m2026-08-24 09:46:12.254\u001b[0m | \u001b[1mINFO \u001b[0m | \u001b[36mbw_timex.timex_lca\u001b[0m:\u001b[36mrun\u001b[0m:\u001b[36m745\u001b[0m - \u001b[1mStep 1/4: Building timeline...\u001b[0m\n", + "\u001b[32m2026-08-24 09:46:12.258\u001b[0m | \u001b[1mINFO \u001b[0m | \u001b[36mbw_timex.timex_lca\u001b[0m:\u001b[36mrun\u001b[0m:\u001b[36m760\u001b[0m - \u001b[1mStep 2/4: Calculating LCI...\u001b[0m\n", + "\u001b[32m2026-08-24 09:46:12.982\u001b[0m | \u001b[1mINFO \u001b[0m | \u001b[36mbw_timex.timex_lca\u001b[0m:\u001b[36mlci\u001b[0m:\u001b[36m1350\u001b[0m - \u001b[1mExpanding matrices...\u001b[0m\n", + "\u001b[32m2026-08-24 09:46:13.068\u001b[0m | \u001b[1mINFO \u001b[0m | \u001b[36mbw_timex.timex_lca\u001b[0m:\u001b[36mlci\u001b[0m:\u001b[36m1369\u001b[0m - \u001b[1mCalculating dynamic inventory...\u001b[0m\n", + "\u001b[32m2026-08-24 09:46:17.519\u001b[0m | \u001b[1mINFO \u001b[0m | \u001b[36mbw_timex.timex_lca\u001b[0m:\u001b[36mrun\u001b[0m:\u001b[36m769\u001b[0m - \u001b[1mStep 3/4: Calculating static LCIA...\u001b[0m\n", + "\u001b[32m2026-08-24 09:46:17.526\u001b[0m | \u001b[1mINFO \u001b[0m | \u001b[36mbw_timex.timex_lca\u001b[0m:\u001b[36mrun\u001b[0m:\u001b[36m776\u001b[0m - \u001b[1mStep 4/4: Skipping dynamic LCIA (disabled).\u001b[0m\n", + "\u001b[32m2026-08-24 09:46:17.527\u001b[0m | \u001b[1mINFO \u001b[0m | \u001b[36mbw_timex.timex_lca\u001b[0m:\u001b[36mrun\u001b[0m:\u001b[36m799\u001b[0m - \u001b[1mTimexLCA.run() completed successfully.\u001b[0m\n" ] }, { - "name": "stderr", + "name": "stdout", "output_type": "stream", "text": [ - "\u001b[32m2026-08-23 22:22:32.640\u001b[0m | \u001b[1mINFO \u001b[0m | \u001b[36mbw_timex.timex_lca\u001b[0m:\u001b[36mcompare\u001b[0m:\u001b[36m708\u001b[0m - \u001b[1mComparison 3/3: bought 2075\u001b[0m\n" + "static score, bought 2075: 8375\n", + "base LCA reused: True\n" ] - }, + } + ], + "source": [ + "base_lca_id = id(tlca.base_lca)\n", + "\n", + "tlca.run(starting_datetime=datetime(2075, 6, 1)) # same EV, bought decades later\n", + "print(\"static score, bought 2075:\", round(tlca.static_score))\n", + "print(\"base LCA reused:\", id(tlca.base_lca) == base_lca_id)" + ] + }, + { + "cell_type": "markdown", + "id": "27eeb73a", + "metadata": {}, + "source": [ + "A third of the 2020 purchase's impact, on the same EV - that is REMIND-EU's grid decarbonizing under the vehicle." + ] + }, + { + "cell_type": "markdown", + "id": "76cb0ec3", + "metadata": {}, + "source": [ + "## Several calculations: `compare()`" + ] + }, + { + "cell_type": "markdown", + "id": "2c8ee43d", + "metadata": {}, + "source": [ + "`compare()` takes a list of settings and returns a `ComparisonResult`\n", + "whose `summary` holds one row each - the scores next to every setting that\n", + "produced them, so the table is its own record of what was run. It builds one\n", + "`TimexLCA` per distinct background, so the purchase years below share a\n", + "single object:" + ] + }, + { + "cell_type": "code", + "execution_count": 5, + "id": "e328ddc0", + "metadata": { + "execution": { + "iopub.execute_input": "2026-08-23T20:22:07.096140Z", + "iopub.status.busy": "2026-08-23T20:22:07.096013Z", + "iopub.status.idle": "2026-08-23T20:22:39.490162Z", + "shell.execute_reply": "2026-08-23T20:22:39.489738Z" + } + }, + "outputs": [ { "name": "stderr", "output_type": "stream", "text": [ - "\u001b[32m2026-08-23 22:22:32.640\u001b[0m | \u001b[1mINFO \u001b[0m | \u001b[36mbw_timex.timex_lca\u001b[0m:\u001b[36mrun\u001b[0m:\u001b[36m529\u001b[0m - \u001b[1mStarting TimexLCA.run() pipeline...\u001b[0m\n" + "\u001b[32m2026-08-24 09:46:44.216\u001b[0m | \u001b[1mINFO \u001b[0m | \u001b[36mbw_timex.timex_lca\u001b[0m:\u001b[36m__init__\u001b[0m:\u001b[36m421\u001b[0m - \u001b[1mInitializing TimexLCA object...\u001b[0m\n", + "\u001b[32m2026-08-24 09:46:44.224\u001b[0m | \u001b[1mINFO \u001b[0m | \u001b[36mbw_timex.timex_lca\u001b[0m:\u001b[36m__init__\u001b[0m:\u001b[36m508\u001b[0m - \u001b[1mCalculating base LCA...\u001b[0m\n", + "/Users/timodiepers/Documents/Coding/bw_timex/.venv/lib/python3.12/site-packages/scikits/umfpack/umfpack.py:737: UmfpackWarning: (almost) singular matrix! (estimated cond. number: 3.90e+13)\n", + " warnings.warn(msg, UmfpackWarning)\n", + "\u001b[32m2026-08-24 09:46:45.218\u001b[0m | \u001b[1mINFO \u001b[0m | \u001b[36mbw_timex.timex_lca\u001b[0m:\u001b[36m__init__\u001b[0m:\u001b[36m525\u001b[0m - \u001b[1mCollecting node infos...\u001b[0m\n", + "\u001b[32m2026-08-24 09:46:45.274\u001b[0m | \u001b[1mINFO \u001b[0m | \u001b[36mbw_timex.timex_lca\u001b[0m:\u001b[36m__init__\u001b[0m:\u001b[36m537\u001b[0m - \u001b[1mLoading node metadata from 10 database(s)...\u001b[0m\n", + "\u001b[32m2026-08-24 09:46:45.541\u001b[0m | \u001b[1mINFO \u001b[0m | \u001b[36mbw_timex.timex_lca\u001b[0m:\u001b[36m__init__\u001b[0m:\u001b[36m574\u001b[0m - \u001b[1mTimexLCA initialized.\u001b[0m\n", + "\u001b[32m2026-08-24 09:46:45.541\u001b[0m | \u001b[1mINFO \u001b[0m | \u001b[36mbw_timex.timex_lca\u001b[0m:\u001b[36mcompare\u001b[0m:\u001b[36m948\u001b[0m - \u001b[1mComparison 1/3: bought 2020\u001b[0m\n", + "\u001b[32m2026-08-24 09:46:45.542\u001b[0m | \u001b[1mINFO \u001b[0m | \u001b[36mbw_timex.timex_lca\u001b[0m:\u001b[36mrun\u001b[0m:\u001b[36m738\u001b[0m - \u001b[1mStarting TimexLCA.run() pipeline...\u001b[0m\n", + "\u001b[32m2026-08-24 09:46:45.542\u001b[0m | \u001b[1mINFO \u001b[0m | \u001b[36mbw_timex.timex_lca\u001b[0m:\u001b[36mrun\u001b[0m:\u001b[36m745\u001b[0m - \u001b[1mStep 1/4: Building timeline...\u001b[0m\n", + "\u001b[32m2026-08-24 09:46:45.543\u001b[0m | \u001b[1mINFO \u001b[0m | \u001b[36mbw_timex.timex_lca\u001b[0m:\u001b[36mbuild_timeline\u001b[0m:\u001b[36m1177\u001b[0m - \u001b[1mNo edge filter function provided. Skipping all edges in background databases.\u001b[0m\n", + "\u001b[32m2026-08-24 09:47:04.812\u001b[0m | \u001b[1mINFO \u001b[0m | \u001b[36mbw_timex.timex_lca\u001b[0m:\u001b[36mbuild_timeline\u001b[0m:\u001b[36m1198\u001b[0m - \u001b[1mCreating activity time mapping...\u001b[0m\n", + "\u001b[32m2026-08-24 09:47:05.135\u001b[0m | \u001b[1mINFO \u001b[0m | \u001b[36mbw_timex.timeline_builder\u001b[0m:\u001b[36m__init__\u001b[0m:\u001b[36m112\u001b[0m - \u001b[1mTraversing supply chain graph...\u001b[0m\n" ] }, { - "name": "stderr", + "name": "stdout", "output_type": "stream", "text": [ - "\u001b[32m2026-08-23 22:22:32.640\u001b[0m | \u001b[1mINFO \u001b[0m | \u001b[36mbw_timex.timex_lca\u001b[0m:\u001b[36mrun\u001b[0m:\u001b[36m536\u001b[0m - \u001b[1mStep 1/4: Building timeline...\u001b[0m\n" + "Starting graph traversal\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ - "\u001b[32m2026-08-23 22:22:32.641\u001b[0m | \u001b[1mINFO \u001b[0m | \u001b[36mbw_timex.timex_lca\u001b[0m:\u001b[36mbuild_timeline\u001b[0m:\u001b[36m938\u001b[0m - \u001b[1mNo edge filter function provided. Skipping all edges in background databases.\u001b[0m\n" + "/Users/timodiepers/Documents/Coding/bw_timex/.venv/lib/python3.12/site-packages/bw_graph_tools/graph_traversal/new_node_each_visit.py:351: UserWarning: Graph traversal covered only 0.2% of the total LCA score. Consider lowering the `cutoff` (currently 1e-09) to improve coverage.\n", + " warnings.warn(\n", + "\u001b[32m2026-08-24 09:47:15.143\u001b[0m | \u001b[1mINFO \u001b[0m | \u001b[36mbw_timex.timeline_builder\u001b[0m:\u001b[36mbuild_timeline\u001b[0m:\u001b[36m183\u001b[0m - \u001b[1mBuilding timeline...\u001b[0m\n" ] }, { - "name": "stderr", + "name": "stdout", "output_type": "stream", "text": [ - "\u001b[32m2026-08-23 22:22:32.641\u001b[0m | \u001b[1mINFO \u001b[0m | \u001b[36mbw_timex.timex_lca\u001b[0m:\u001b[36mbuild_timeline\u001b[0m:\u001b[36m959\u001b[0m - \u001b[1mCreating activity time mapping...\u001b[0m\n" + "Calculation count: 103\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ - "\u001b[32m2026-08-23 22:22:32.818\u001b[0m | \u001b[1mINFO \u001b[0m | \u001b[36mbw_timex.timeline_builder\u001b[0m:\u001b[36m__init__\u001b[0m:\u001b[36m112\u001b[0m - \u001b[1mTraversing supply chain graph...\u001b[0m\n" + "\u001b[32m2026-08-24 09:47:15.362\u001b[0m | \u001b[1mINFO \u001b[0m | \u001b[36mbw_timex.timeline_builder\u001b[0m:\u001b[36mget_weights_for_interpolation_between_nearest_years\u001b[0m:\u001b[36m808\u001b[0m - \u001b[1mReference date 2018-01-01 00:00:00 is lower than all provided dates. Data will be taken from the closest higher year.\u001b[0m\n", + "\u001b[32m2026-08-24 09:47:15.476\u001b[0m | \u001b[1mINFO \u001b[0m | \u001b[36mbw_timex.timex_lca\u001b[0m:\u001b[36mrun\u001b[0m:\u001b[36m760\u001b[0m - \u001b[1mStep 2/4: Calculating LCI...\u001b[0m\n", + "\u001b[32m2026-08-24 09:47:15.985\u001b[0m | \u001b[1mINFO \u001b[0m | \u001b[36mbw_timex.timex_lca\u001b[0m:\u001b[36mlci\u001b[0m:\u001b[36m1350\u001b[0m - \u001b[1mExpanding matrices...\u001b[0m\n", + "\u001b[32m2026-08-24 09:47:16.010\u001b[0m | \u001b[1mINFO \u001b[0m | \u001b[36mbw_timex.timex_lca\u001b[0m:\u001b[36mlci\u001b[0m:\u001b[36m1369\u001b[0m - \u001b[1mCalculating dynamic inventory...\u001b[0m\n", + "/Users/timodiepers/Documents/Coding/bw_timex/.venv/lib/python3.12/site-packages/scikits/umfpack/umfpack.py:737: UmfpackWarning: (almost) singular matrix! (estimated cond. number: 1.65e+13)\n", + " warnings.warn(msg, UmfpackWarning)\n", + "\u001b[32m2026-08-24 09:47:19.042\u001b[0m | \u001b[1mINFO \u001b[0m | \u001b[36mbw_timex.timex_lca\u001b[0m:\u001b[36mrun\u001b[0m:\u001b[36m769\u001b[0m - \u001b[1mStep 3/4: Calculating static LCIA...\u001b[0m\n", + "\u001b[32m2026-08-24 09:47:19.047\u001b[0m | \u001b[1mINFO \u001b[0m | \u001b[36mbw_timex.timex_lca\u001b[0m:\u001b[36mrun\u001b[0m:\u001b[36m776\u001b[0m - \u001b[1mStep 4/4: Skipping dynamic LCIA (disabled).\u001b[0m\n", + "\u001b[32m2026-08-24 09:47:19.048\u001b[0m | \u001b[1mINFO \u001b[0m | \u001b[36mbw_timex.timex_lca\u001b[0m:\u001b[36mrun\u001b[0m:\u001b[36m799\u001b[0m - \u001b[1mTimexLCA.run() completed successfully.\u001b[0m\n", + "\u001b[32m2026-08-24 09:47:19.053\u001b[0m | \u001b[1mINFO \u001b[0m | \u001b[36mbw_timex.timex_lca\u001b[0m:\u001b[36mcompare\u001b[0m:\u001b[36m948\u001b[0m - \u001b[1mComparison 2/3: bought 2040\u001b[0m\n", + "\u001b[32m2026-08-24 09:47:19.054\u001b[0m | \u001b[1mINFO \u001b[0m | \u001b[36mbw_timex.timex_lca\u001b[0m:\u001b[36mrun\u001b[0m:\u001b[36m738\u001b[0m - \u001b[1mStarting TimexLCA.run() pipeline...\u001b[0m\n", + "\u001b[32m2026-08-24 09:47:19.054\u001b[0m | \u001b[1mINFO \u001b[0m | \u001b[36mbw_timex.timex_lca\u001b[0m:\u001b[36mrun\u001b[0m:\u001b[36m745\u001b[0m - \u001b[1mStep 1/4: Building timeline...\u001b[0m\n", + "\u001b[32m2026-08-24 09:47:19.056\u001b[0m | \u001b[1mINFO \u001b[0m | \u001b[36mbw_timex.timex_lca\u001b[0m:\u001b[36mbuild_timeline\u001b[0m:\u001b[36m1177\u001b[0m - \u001b[1mNo edge filter function provided. Skipping all edges in background databases.\u001b[0m\n", + "\u001b[32m2026-08-24 09:47:19.056\u001b[0m | \u001b[1mINFO \u001b[0m | \u001b[36mbw_timex.timex_lca\u001b[0m:\u001b[36mbuild_timeline\u001b[0m:\u001b[36m1198\u001b[0m - \u001b[1mCreating activity time mapping...\u001b[0m\n", + "\u001b[32m2026-08-24 09:47:19.569\u001b[0m | \u001b[1mINFO \u001b[0m | \u001b[36mbw_timex.timeline_builder\u001b[0m:\u001b[36m__init__\u001b[0m:\u001b[36m112\u001b[0m - \u001b[1mTraversing supply chain graph...\u001b[0m\n" ] }, { @@ -836,65 +429,72 @@ "text": [ "/Users/timodiepers/Documents/Coding/bw_timex/.venv/lib/python3.12/site-packages/bw_graph_tools/graph_traversal/new_node_each_visit.py:351: UserWarning: Graph traversal covered only 0.2% of the total LCA score. Consider lowering the `cutoff` (currently 1e-09) to improve coverage.\n", " warnings.warn(\n", - "\u001b[32m2026-08-23 22:22:36.008\u001b[0m | \u001b[1mINFO \u001b[0m | \u001b[36mbw_timex.timeline_builder\u001b[0m:\u001b[36mbuild_timeline\u001b[0m:\u001b[36m183\u001b[0m - \u001b[1mBuilding timeline...\u001b[0m\n" + "\u001b[32m2026-08-24 09:47:23.917\u001b[0m | \u001b[1mINFO \u001b[0m | \u001b[36mbw_timex.timeline_builder\u001b[0m:\u001b[36mbuild_timeline\u001b[0m:\u001b[36m183\u001b[0m - \u001b[1mBuilding timeline...\u001b[0m\n" ] }, { - "name": "stderr", + "name": "stdout", "output_type": "stream", "text": [ - "\u001b[32m2026-08-23 22:22:36.115\u001b[0m | \u001b[1mINFO \u001b[0m | \u001b[36mbw_timex.timeline_builder\u001b[0m:\u001b[36mget_weights_for_interpolation_between_nearest_years\u001b[0m:\u001b[36m815\u001b[0m - \u001b[1mReference date 2073-01-01 00:00:00 is higher than all provided dates. Data will be taken from the closest lower year.\u001b[0m\n" + "Calculation count: 103\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ - "\u001b[32m2026-08-23 22:22:36.158\u001b[0m | \u001b[1mINFO \u001b[0m | \u001b[36mbw_timex.timex_lca\u001b[0m:\u001b[36mrun\u001b[0m:\u001b[36m551\u001b[0m - \u001b[1mStep 2/4: Calculating LCI...\u001b[0m\n" + "\u001b[32m2026-08-24 09:47:24.102\u001b[0m | \u001b[1mINFO \u001b[0m | \u001b[36mbw_timex.timex_lca\u001b[0m:\u001b[36mrun\u001b[0m:\u001b[36m760\u001b[0m - \u001b[1mStep 2/4: Calculating LCI...\u001b[0m\n", + "\u001b[32m2026-08-24 09:47:25.000\u001b[0m | \u001b[1mINFO \u001b[0m | \u001b[36mbw_timex.timex_lca\u001b[0m:\u001b[36mlci\u001b[0m:\u001b[36m1350\u001b[0m - \u001b[1mExpanding matrices...\u001b[0m\n", + "\u001b[32m2026-08-24 09:47:25.099\u001b[0m | \u001b[1mINFO \u001b[0m | \u001b[36mbw_timex.timex_lca\u001b[0m:\u001b[36mlci\u001b[0m:\u001b[36m1369\u001b[0m - \u001b[1mCalculating dynamic inventory...\u001b[0m\n", + "/Users/timodiepers/Documents/Coding/bw_timex/.venv/lib/python3.12/site-packages/scikits/umfpack/umfpack.py:737: UmfpackWarning: (almost) singular matrix! (estimated cond. number: 9.21e+13)\n", + " warnings.warn(msg, UmfpackWarning)\n", + "\u001b[32m2026-08-24 09:47:29.952\u001b[0m | \u001b[1mINFO \u001b[0m | \u001b[36mbw_timex.timex_lca\u001b[0m:\u001b[36mrun\u001b[0m:\u001b[36m769\u001b[0m - \u001b[1mStep 3/4: Calculating static LCIA...\u001b[0m\n", + "\u001b[32m2026-08-24 09:47:29.957\u001b[0m | \u001b[1mINFO \u001b[0m | \u001b[36mbw_timex.timex_lca\u001b[0m:\u001b[36mrun\u001b[0m:\u001b[36m776\u001b[0m - \u001b[1mStep 4/4: Skipping dynamic LCIA (disabled).\u001b[0m\n", + "\u001b[32m2026-08-24 09:47:29.958\u001b[0m | \u001b[1mINFO \u001b[0m | \u001b[36mbw_timex.timex_lca\u001b[0m:\u001b[36mrun\u001b[0m:\u001b[36m799\u001b[0m - \u001b[1mTimexLCA.run() completed successfully.\u001b[0m\n", + "\u001b[32m2026-08-24 09:47:29.960\u001b[0m | \u001b[1mINFO \u001b[0m | \u001b[36mbw_timex.timex_lca\u001b[0m:\u001b[36mcompare\u001b[0m:\u001b[36m948\u001b[0m - \u001b[1mComparison 3/3: bought 2075\u001b[0m\n", + "\u001b[32m2026-08-24 09:47:29.960\u001b[0m | \u001b[1mINFO \u001b[0m | \u001b[36mbw_timex.timex_lca\u001b[0m:\u001b[36mrun\u001b[0m:\u001b[36m738\u001b[0m - \u001b[1mStarting TimexLCA.run() pipeline...\u001b[0m\n", + "\u001b[32m2026-08-24 09:47:29.960\u001b[0m | \u001b[1mINFO \u001b[0m | \u001b[36mbw_timex.timex_lca\u001b[0m:\u001b[36mrun\u001b[0m:\u001b[36m745\u001b[0m - \u001b[1mStep 1/4: Building timeline...\u001b[0m\n", + "\u001b[32m2026-08-24 09:47:29.960\u001b[0m | \u001b[1mINFO \u001b[0m | \u001b[36mbw_timex.timex_lca\u001b[0m:\u001b[36mbuild_timeline\u001b[0m:\u001b[36m1177\u001b[0m - \u001b[1mNo edge filter function provided. Skipping all edges in background databases.\u001b[0m\n", + "\u001b[32m2026-08-24 09:47:29.961\u001b[0m | \u001b[1mINFO \u001b[0m | \u001b[36mbw_timex.timex_lca\u001b[0m:\u001b[36mbuild_timeline\u001b[0m:\u001b[36m1198\u001b[0m - \u001b[1mCreating activity time mapping...\u001b[0m\n", + "\u001b[32m2026-08-24 09:47:30.348\u001b[0m | \u001b[1mINFO \u001b[0m | \u001b[36mbw_timex.timeline_builder\u001b[0m:\u001b[36m__init__\u001b[0m:\u001b[36m112\u001b[0m - \u001b[1mTraversing supply chain graph...\u001b[0m\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ - "Calculation count: 103\n" + "Starting graph traversal\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ - "\u001b[32m2026-08-23 22:22:36.686\u001b[0m | \u001b[1mINFO \u001b[0m | \u001b[36mbw_timex.timex_lca\u001b[0m:\u001b[36mlci\u001b[0m:\u001b[36m1120\u001b[0m - \u001b[1mExpanding matrices...\u001b[0m\n" + "/Users/timodiepers/Documents/Coding/bw_timex/.venv/lib/python3.12/site-packages/bw_graph_tools/graph_traversal/new_node_each_visit.py:351: UserWarning: Graph traversal covered only 0.2% of the total LCA score. Consider lowering the `cutoff` (currently 1e-09) to improve coverage.\n", + " warnings.warn(\n", + "\u001b[32m2026-08-24 09:47:34.311\u001b[0m | \u001b[1mINFO \u001b[0m | \u001b[36mbw_timex.timeline_builder\u001b[0m:\u001b[36mbuild_timeline\u001b[0m:\u001b[36m183\u001b[0m - \u001b[1mBuilding timeline...\u001b[0m\n" ] }, { - "name": "stderr", + "name": "stdout", "output_type": "stream", "text": [ - "\u001b[32m2026-08-23 22:22:36.737\u001b[0m | \u001b[1mINFO \u001b[0m | \u001b[36mbw_timex.timex_lca\u001b[0m:\u001b[36mlci\u001b[0m:\u001b[36m1139\u001b[0m - \u001b[1mCalculating dynamic inventory...\u001b[0m\n" + "Calculation count: 103\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ + "\u001b[32m2026-08-24 09:47:34.506\u001b[0m | \u001b[1mINFO \u001b[0m | \u001b[36mbw_timex.timeline_builder\u001b[0m:\u001b[36mget_weights_for_interpolation_between_nearest_years\u001b[0m:\u001b[36m815\u001b[0m - \u001b[1mReference date 2073-01-01 00:00:00 is higher than all provided dates. Data will be taken from the closest lower year.\u001b[0m\n", + "\u001b[32m2026-08-24 09:47:34.568\u001b[0m | \u001b[1mINFO \u001b[0m | \u001b[36mbw_timex.timex_lca\u001b[0m:\u001b[36mrun\u001b[0m:\u001b[36m760\u001b[0m - \u001b[1mStep 2/4: Calculating LCI...\u001b[0m\n", + "\u001b[32m2026-08-24 09:47:35.189\u001b[0m | \u001b[1mINFO \u001b[0m | \u001b[36mbw_timex.timex_lca\u001b[0m:\u001b[36mlci\u001b[0m:\u001b[36m1350\u001b[0m - \u001b[1mExpanding matrices...\u001b[0m\n", + "\u001b[32m2026-08-24 09:47:35.281\u001b[0m | \u001b[1mINFO \u001b[0m | \u001b[36mbw_timex.timex_lca\u001b[0m:\u001b[36mlci\u001b[0m:\u001b[36m1369\u001b[0m - \u001b[1mCalculating dynamic inventory...\u001b[0m\n", "/Users/timodiepers/Documents/Coding/bw_timex/.venv/lib/python3.12/site-packages/scikits/umfpack/umfpack.py:737: UmfpackWarning: (almost) singular matrix! (estimated cond. number: 2.10e+13)\n", " warnings.warn(msg, UmfpackWarning)\n", - "\u001b[32m2026-08-23 22:22:39.428\u001b[0m | \u001b[1mINFO \u001b[0m | \u001b[36mbw_timex.timex_lca\u001b[0m:\u001b[36mrun\u001b[0m:\u001b[36m560\u001b[0m - \u001b[1mStep 3/4: Calculating static LCIA...\u001b[0m\n" - ] - }, - { - "name": "stderr", - "output_type": "stream", - "text": [ - "\u001b[32m2026-08-23 22:22:39.430\u001b[0m | \u001b[1mINFO \u001b[0m | \u001b[36mbw_timex.timex_lca\u001b[0m:\u001b[36mrun\u001b[0m:\u001b[36m578\u001b[0m - \u001b[1mStep 4/4: Skipping dynamic LCIA (disabled).\u001b[0m\n" - ] - }, - { - "name": "stderr", - "output_type": "stream", - "text": [ - "\u001b[32m2026-08-23 22:22:39.430\u001b[0m | \u001b[1mINFO \u001b[0m | \u001b[36mbw_timex.timex_lca\u001b[0m:\u001b[36mrun\u001b[0m:\u001b[36m580\u001b[0m - \u001b[1mTimexLCA.run() completed successfully.\u001b[0m\n" + "\u001b[32m2026-08-24 09:47:39.938\u001b[0m | \u001b[1mINFO \u001b[0m | \u001b[36mbw_timex.timex_lca\u001b[0m:\u001b[36mrun\u001b[0m:\u001b[36m769\u001b[0m - \u001b[1mStep 3/4: Calculating static LCIA...\u001b[0m\n", + "\u001b[32m2026-08-24 09:47:39.944\u001b[0m | \u001b[1mINFO \u001b[0m | \u001b[36mbw_timex.timex_lca\u001b[0m:\u001b[36mrun\u001b[0m:\u001b[36m776\u001b[0m - \u001b[1mStep 4/4: Skipping dynamic LCIA (disabled).\u001b[0m\n", + "\u001b[32m2026-08-24 09:47:39.944\u001b[0m | \u001b[1mINFO \u001b[0m | \u001b[36mbw_timex.timex_lca\u001b[0m:\u001b[36mrun\u001b[0m:\u001b[36m799\u001b[0m - \u001b[1mTimexLCA.run() completed successfully.\u001b[0m\n" ] }, { @@ -932,7 +532,7 @@ " 23247.151124\n", " 25462.997441\n", " 27\n", - " 17.390129\n", + " 33.510305\n", " \n", " \n", " 1\n", @@ -940,7 +540,7 @@ " 23247.151124\n", " 9127.954269\n", " 27\n", - " 7.374905\n", + " 10.905721\n", " \n", " \n", " 2\n", @@ -948,7 +548,7 @@ " 23247.151124\n", " 8375.067004\n", " 27\n", - " 6.790291\n", + " 9.985533\n", " \n", " \n", "\n", @@ -956,12 +556,12 @@ ], "text/plain": [ " label base_score static_score timeline_rows runtime_s\n", - "0 bought 2020 23247.151124 25462.997441 27 17.390129\n", - "1 bought 2040 23247.151124 9127.954269 27 7.374905\n", - "2 bought 2075 23247.151124 8375.067004 27 6.790291" + "0 bought 2020 23247.151124 25462.997441 27 33.510305\n", + "1 bought 2040 23247.151124 9127.954269 27 10.905721\n", + "2 bought 2075 23247.151124 8375.067004 27 9.985533" ] }, - "execution_count": 4, + "execution_count": 5, "metadata": {}, "output_type": "execute_result" } @@ -974,8 +574,85 @@ " replace(settings, starting_datetime=datetime(year, 6, 1), label=f\"bought {year}\")\n", " for year in (2020, 2040, 2075)\n", " ]\n", - ")\n", - "\n", + ")\n" + ] + }, + { + "cell_type": "code", + "execution_count": 9, + "id": "3fc163ea", + "metadata": {}, + "outputs": [ + { + "data": { + "text/html": [ + "
\n", + "\n", + "\n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + "
labelbase_scorestatic_scoretimeline_rowsruntime_s
0bought 202023247.15112425462.9974412733.510305
1bought 204023247.1511249127.9542692710.905721
2bought 207523247.1511248375.067004279.985533
\n", + "
" + ], + "text/plain": [ + " label base_score static_score timeline_rows runtime_s\n", + "0 bought 2020 23247.151124 25462.997441 27 33.510305\n", + "1 bought 2040 23247.151124 9127.954269 27 10.905721\n", + "2 bought 2075 23247.151124 8375.067004 27 9.985533" + ] + }, + "execution_count": 9, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ "comparison.summary[[\"label\", \"base_score\", \"static_score\", \"timeline_rows\", \"runtime_s\"]]" ] }, @@ -1000,7 +677,7 @@ ], "metadata": { "kernelspec": { - "display_name": "Python 3", + "display_name": ".venv (3.12.12)", "language": "python", "name": "python3" }, diff --git a/tests/test_public_api.py b/tests/test_public_api.py index e0dc95ba..22420e0f 100644 --- a/tests/test_public_api.py +++ b/tests/test_public_api.py @@ -1,11 +1,29 @@ """Tests for what `bw_timex` exposes at the top level.""" +import importlib +import pkgutil + import bw_temporalis import pytest import bw_timex from bw_timex import utils +MODULES = sorted( + module.name for module in pkgutil.iter_modules(bw_timex.__path__) +) + + +@pytest.mark.parametrize("name", MODULES) +def test_every_module_imports(name): + """Every module imports, so a rename can't leave a stale importer behind. + + Renaming something in one module and missing an importer in another makes + `import bw_timex` itself fail, which takes the whole test suite down with a + collection error rather than a test failure - so it is worth its own test. + """ + importlib.import_module(f"bw_timex.{name}") + FORWARDED_FROM_TEMPORALIS = [ "TemporalDistribution", "easy_datetime_distribution", diff --git a/tests/test_timex_lca_run.py b/tests/test_timex_lca_run.py index ff21f461..9c30b0a3 100644 --- a/tests/test_timex_lca_run.py +++ b/tests/test_timex_lca_run.py @@ -6,10 +6,77 @@ import bw2data as bd import pandas as pd import pytest +from bw2data.errors import UnknownObject from bw_timex import TimexLCA, TimexLCASettings, set_database_metadata +class TestSettingsStageGroups: + """Settings can be written grouped by the stage each knob belongs to. + + The grouping is call-site sugar: the fields stay flat, so `run()` + overrides and `dataclasses.replace` are unaffected. + """ + + demand = {("foreground", "A"): 1} + method = ("GWP", "example") + + def test_groups_set_the_same_fields_as_flat_arguments(self): + grouped = TimexLCASettings( + demand=self.demand, + method=self.method, + timeline={"temporal_grouping": "month", "cutoff": 1e-6}, + lci={"build_dynamic_biosphere": False}, + lcia={"metric": "GWP", "time_horizon": 20}, + ) + flat = TimexLCASettings( + demand=self.demand, + method=self.method, + temporal_grouping="month", + cutoff=1e-6, + build_dynamic_biosphere=False, + metric="GWP", + time_horizon=20, + ) + + assert grouped == flat + + def test_replace_still_works_flat_on_a_grouped_settings(self): + """`replace` re-runs __init__ with the groups defaulted away.""" + settings = TimexLCASettings( + demand=self.demand, method=self.method, lcia={"metric": "GWP"} + ) + + varied = replace(settings, time_horizon=20) + + assert varied.metric == "GWP" # carried over, not reset by the empty group + assert varied.time_horizon == 20 + assert settings.time_horizon == 100 # original untouched + + def test_unknown_key_in_a_group_is_rejected(self): + with pytest.raises(TypeError, match="temporal_groupng"): + TimexLCASettings( + demand=self.demand, + method=self.method, + timeline={"temporal_groupng": "month"}, + ) + + def test_key_in_the_wrong_group_names_the_right_one(self): + with pytest.raises(TypeError, match="lcia"): + TimexLCASettings( + demand=self.demand, method=self.method, timeline={"metric": "GWP"} + ) + + def test_a_group_conflicting_with_an_explicit_flat_argument_is_rejected(self): + with pytest.raises(TypeError, match="temporal_grouping"): + TimexLCASettings( + demand=self.demand, + method=self.method, + temporal_grouping="day", + timeline={"temporal_grouping": "month"}, + ) + + @pytest.mark.usefixtures("temporal_grouping_db_monthly") class TestSettingsAndRun: @@ -83,6 +150,99 @@ def test_run_accepts_a_replacement_settings_object(self): assert tlca.temporal_grouping == "month" + def test_settings_can_be_passed_straight_to_the_constructor(self): + """`TimexLCA(settings)` - no separate builder to remember.""" + settings = self.base_settings() + tlca = TimexLCA(settings) + + assert tlca.settings is settings + assert tlca.demand == settings.demand + assert tlca.method == settings.method + + tlca.run() + assert isinstance(tlca.static_score, float) + assert tlca.static_score != 0 + + def test_settings_in_the_constructor_reject_a_second_argument(self): + with pytest.raises(TypeError, match="TimexLCASettings"): + TimexLCA(self.base_settings(), self.method) + + # ─── what a notebook sees after run() ─── + + def test_repr_reports_the_scores_a_run_produced(self): + """`tlca.run()` is the last line of a cell, so its repr is what's shown.""" + tlca = TimexLCA(self.base_settings()) + assert "static_score" not in repr(tlca) + + tlca.run() + + assert "TimexLCA" in repr(tlca) + assert f"static_score={tlca.static_score:.4g}" in repr(tlca) + + def test_timeline_summary_is_what_build_timeline_returns(self): + """`run()` returns no timeline, so the readable view has to be reachable.""" + tlca = TimexLCA(self.base_settings()) + returned = tlca.build_timeline() + + pd.testing.assert_frame_equal(returned, tlca.timeline_summary) + assert list(tlca.timeline_summary.columns) == [ + "date_producer", + "producer_name", + "date_consumer", + "consumer_name", + "amount", + "temporal_market_shares", + ] + + # ─── dynamic LCIA is opportunistic by default ─── + + def test_run_skips_dynamic_lcia_when_the_flows_cannot_be_characterized(self): + """The fixture's biosphere is not `biosphere3`, so nothing maps. + + `run()` is the "just give me a result" path, so it says so and carries + on with the static score rather than raising. + """ + tlca = TimexLCA( + demand={self.fu.key: 1}, + method=self.method, + database_dates=self.database_dates, + ) + + tlca.run(starting_datetime=self.start) + + assert isinstance(tlca.static_score, float) + assert not hasattr(tlca, "characterized_inventory") + + def test_run_raises_when_dynamic_lcia_is_asked_for_explicitly(self): + tlca = TimexLCA( + demand={self.fu.key: 1}, + method=self.method, + database_dates=self.database_dates, + ) + + with pytest.raises(UnknownObject): + tlca.run(starting_datetime=self.start, dynamic_lcia_enabled=True) + + def test_run_on_a_plainly_constructed_object(self): + """`TimexLCA(...).run()` is the path the docs recommend first. + + No settings object anywhere: `run()` has to fall back to the demand, + method and background the constructor was given, and still accept + per-call overrides. + """ + tlca = TimexLCA( + demand={self.fu.key: 1}, + method=self.method, + database_dates=self.database_dates, + ) + + tlca.run(starting_datetime=self.start, dynamic_lcia_enabled=False) + + assert hasattr(tlca, "timeline") + assert isinstance(tlca.static_score, float) + assert tlca.static_score != 0 + assert tlca.settings is None # the object's own settings stay untouched + def test_run_rejects_unknown_kwarg(self): tlca = TimexLCA.from_settings(self.base_settings()) with pytest.raises(TypeError, match="not_a_setting"): diff --git a/tests/test_unused_vintage_pruning.py b/tests/test_unused_vintage_pruning.py new file mode 100644 index 00000000..3c8ba8a8 --- /dev/null +++ b/tests/test_unused_vintage_pruning.py @@ -0,0 +1,68 @@ +"""Vintages the timeline never sources from must not be loaded by `lci()`. + +`TimexLCA` maps every database that declares a point in time, but a study only +ever draws on the vintages its timeline actually reaches. The others contribute +no matrix entries - the expanded technosphere only references databases that +appear in a row's `temporal_market_shares` - so loading their datapackages only +inflates the matrix that has to be solved. +""" + +from datetime import datetime + +import bw2data as bd +import pytest + +from bw_timex import TimexLCA + +DATES_ALL = { + "background_2020": datetime(2020, 1, 1), + "modified_2020": datetime(2020, 1, 1), + "background_2030": datetime(2030, 1, 1), + "modified_2030": datetime(2030, 1, 1), + "background_2040": datetime(2040, 1, 1), + "modified_2040": datetime(2040, 1, 1), + "foreground": "dynamic", +} +DATES_USED = {k: v for k, v in DATES_ALL.items() if not k.endswith("2040")} + + +def _run(database_dates): + tlca = TimexLCA( + demand={("foreground", "fu"): 1}, + method=("GWP", "example"), + database_dates=database_dates, + ) + tlca.build_timeline(starting_datetime="2026-01-01") + tlca.lci() + tlca.static_lcia() + return tlca + + +def _node_ids(database): + return {node.id for node in bd.Database(database)} + + +@pytest.mark.usefixtures("same_date_db_three_dates") +class TestUnusedVintagePruning: + + def test_unused_vintage_is_not_in_the_expanded_matrix(self): + """The 2040 vintage gets no temporal market share for a 2026 demand.""" + tlca = _run(DATES_ALL) + used_databases = set() + for shares in tlca.timeline["temporal_market_shares"]: + if shares: + used_databases.update(shares) + assert "background_2040" not in used_databases + + in_matrix = set(tlca.lca.dicts.activity) & _node_ids("background_2040") + assert not in_matrix + + def test_pruning_does_not_change_the_score(self): + assert _run(DATES_ALL).static_score == pytest.approx( + _run(DATES_USED).static_score + ) + + def test_used_vintages_stay_in_the_matrix(self): + tlca = _run(DATES_ALL) + for database in ("background_2020", "background_2030"): + assert set(tlca.lca.dicts.activity) & _node_ids(database) diff --git a/zensical.toml b/zensical.toml index d8bc4b79..e571d865 100644 --- a/zensical.toml +++ b/zensical.toml @@ -29,6 +29,8 @@ nav = [ { "Step 3 - Calculating the time-explicit LCI" = "content/getting_started/time_explicit_lci.md" }, { "Step 4 - Impact assessment" = "content/getting_started/lcia.md" }, ]}, + { "Configured Runs & Scenario Comparison" = "content/getting_started/configured_runs.md" }, + { "Creating premise databases On-the-fly" = "content/create_premise_dbs.md" }, { "What LCA should I do?" = "content/decisiontree.md" }, ]}, { Theory = "content/theory.md" },