Skip to content
Open
Show file tree
Hide file tree
Changes from 9 commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 2 additions & 5 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -29,6 +29,8 @@ __pycache__/
*_metadata.json
*_diffs.yaml
!bedrock/utils/**/*.csv
# Published original vs EIA-anchored histogram panels (required for deck CI)
!bedrock/analysis/electricity/historical/original_vs_eia_anchored_deck/figures/*.png
# asets stands for activity sets
!bedrock/extract/**/*_asets.csv
!bedrock/extract/external_data/*.csv
Expand All @@ -52,8 +54,3 @@ scratch/

# Ignore analysis output
bedrock/analysis/**/output/
# Original electricity-disagg implementation freeze (must be committed)
!bedrock/analysis/electricity/historical/original_elec_disagg_implementation/output/
!bedrock/analysis/electricity/historical/original_elec_disagg_implementation/output/**
!bedrock/analysis/electricity/historical/original_elec_disagg_implementation/output/**/*.parquet
!bedrock/analysis/electricity/historical/original_elec_disagg_implementation/output/**/*.json
17 changes: 7 additions & 10 deletions bedrock/analysis/electricity/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,19 +5,16 @@ Two eras, one tree.
- **`current/`** — analyses of **today’s** EIA-anchored generation / transmission /
distribution production (YAML flags `implement_electricity_disaggregation` /
`implement_electricity_mixed_units`).
- **`historical/`** — immutable freeze of the **original** UGO / Table 8.3 /
Table 2.4 electricity-disagg implementation. Do not run historical Python
against current production.
- **`historical/`** — published original vs EIA-anchored (pre-MECS) comparison
tables and figures (not live production).
- **`shared/`** — helpers used by both (eGRID vs national totals; 221100
reallocation matrix export).

| Path | What it is |
|---|---|
| `current/diagnostics/` | BLy / EF / full-trace / year-alignment / F01000 BLy attribution |
| `current/eia_gtd/` | Stub for results-deck G/T/D tables (`EIAPurchaserAllocation`) |
| `current/vs_original_elec_disagg/` | Live EIA path vs the original-implementation freeze |
| `current/eia_gtd_code_impl_f8f73b01.plan.md` | Design note for current production (internal names are stale) |
| `historical/original_elec_disagg_implementation/` | Read-only freeze (`q`, `x`, E/D/N/BLy, …) |
| `current/diagnostics/` | BLy / EF / full-trace / year-alignment / F01000 BLy attribution / comparison deck |
| `current/eia_gtd/` | Results-deck G/T/D tables (`EIAPurchaserAllocation`) |
| `historical/original_vs_eia_anchored_deck/` | PPTX extract: class MWh, D/N tables, vs-footing histogram PNGs |

Do not invent a snapshotter for the freeze. If freeze files are missing, they
are missing or not tracked.
Do not regenerate freeze parquets into this tree. Archived dumps live under Dropbox
`Documentation/Archive/`.
38 changes: 36 additions & 2 deletions bedrock/analysis/electricity/current/diagnostics/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,7 @@ diagnostics/
year_alignment/ # BLy vs E under A/q year handling
hh_vs_interindustry/ # F01000 BLy attribution vs interindustry (not D0 class MWh)
probes/ # one-off sector probes
deck/ # five-slide PPTX (current vs pre-MECS vs original vs production)
output/ # reports/figures (gitignored)
__tests__/
```
Expand All @@ -30,6 +31,7 @@ Run everything from the **repo root** with the project venv
## Shared prerequisites

1. **Configs** (in repo):
- `2025_usa_cornerstone_v0_3` — production (non-disagg, margins on)
- `2025_usa_cornerstone_v0_3_electricity_footing` — footing
- `2025_usa_cornerstone_v0_3_electricity_reallocation`
- `2025_usa_cornerstone_v0_3_electricity_disaggregation`
Expand Down Expand Up @@ -154,10 +156,12 @@ python -m bedrock.analysis.electricity.current.diagnostics.ef_comparison.analyze
| `electricity_reallocation/` | Suite PNGs for that step vs v0.3.1 electricity footing |
| `electricity_disaggregation/` | Suite PNGs for 3-way vs v0.3.1 electricity footing |
| `electricity_mixed_units/` | Suite PNGs for mixed units vs v0.3.1 electricity footing |
| `panel/ef_panels_vs_v0_3_N.png` | 3-panel N % hist |
| `panel/ef_panels_vs_v0_3_D.png` | 3-panel D % hist |
| `panel/ef_panels_vs_v0_3_{N,D}.png` | Live `plot_ef` write path (absent until a new sheet run) |
| `panel/n_variance_*.csv`, `n_variance_explained.md` | From `analyze_n_variance` |

Published original / pre-MECS EIA histogram panels and D/N tables live in
`historical/original_vs_eia_anchored_deck/`, not under `output/ef/panel/`.

Dropped sectors (e.g. mixed-units `221110` kg/MWh vs kg/USD) are footnoted on figures.

---
Expand Down Expand Up @@ -235,6 +239,35 @@ Prints to stdout (no dedicated output file).

---

## 7. Comparison PPTX — `deck/`

Five-slide decks matching the original-vs-EIA-anchored template: class MWh,
electricity-sector D/N by step, and 3-panel histograms. Writes ``.pptx`` under
``output/deck/``. Existing BLy / footing-plot / full-trace scripts stay as they are.

| Pair | Histograms |
|---|---|
| `current_vs_eia_gtd` | Both rows vs v0.3.1 footing (current generated; pre-MECS published PNG). MECS vs dollar-weight D/N is in the tables (**same** when they match). |
| `current_vs_original` | Original row = published `v0.2_original_electricity_disagg_*`; bottom = current vs v0.3.1 |
| `eia_gtd_vs_original` | Published original PNGs over published `v0.3_eia_gtd_pre_mecs_*` |
| `current_vs_production` | Both rows vs Cornerstone v0.3 production (non-disagg, margins on). Top = current electricity-disagg steps; bottom = production vs itself (0% check). Production has no G/T/D or class MWh. |

Identical D/N (or class MWh) cells are labeled **same**. Sectors not in the model
at a step stay **N/A**. Missing live cache shows **—**. Original and pre-MECS
EIA G/T/D tables come from
`historical/original_vs_eia_anchored_deck/tables.yaml`.

```bash
python -m bedrock.analysis.electricity.current.diagnostics.deck --pair current_vs_eia_gtd
python -m bedrock.analysis.electricity.current.diagnostics.deck --pair current_vs_production
python -m bedrock.analysis.electricity.current.diagnostics.deck --all
python -m bedrock.analysis.electricity.current.diagnostics.deck --all --derive
```

``--derive`` live-runs missing **current** (post-MECS) electricity-disagg steps and the **production** (`2025_usa_cornerstone_v0_3`) cache.

---

## Suggested order

1. Sheet cache + BLy waterfalls (`bly_dispersion`) and EF plots (`ef_comparison.plot_ef`)
Expand All @@ -248,5 +281,6 @@ Prints to stdout (no dedicated output file).

```bash
python -m pytest bedrock/analysis/electricity/current/diagnostics/__tests__ \
bedrock/analysis/electricity/current/diagnostics/deck/__tests__ \
bedrock/analysis/electricity/current/eia_gtd/__tests__ -q
```
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,7 @@
- ``year_alignment`` — BLy vs E under A/q year handling
- ``hh_vs_interindustry`` — F01000 BLy attribution vs interindustry (not D0 class MWh)
- ``probes`` — one-off sector probes (e.g. 221200)
- ``deck`` — five-slide PPTX comparisons (current vs pre-MECS vs original)

Shared infrastructure (``paths``, ``manifest``, ``local_data``) stays at package root.
Outputs remain under ``output/`` (layout unchanged by reorganization).
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
"""Five-slide PPTX comparing electricity-disaggregation implementations.

``python -m bedrock.analysis.electricity.current.diagnostics.deck --all``
"""
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
"""``python -m bedrock.analysis.electricity.current.diagnostics.deck``."""

from bedrock.analysis.electricity.current.diagnostics.deck.build import main

if __name__ == '__main__':
main()
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
"""Deck tests."""
Loading
Loading