Run new epmc runs for NY and validate results

context/README.md

@@ -12,11 +12,12 @@ Documents that answer **"How does this work in the real world?"** — policy exp
 
 Fairness, cost allocation, and how regulators use ECOS vs MCOS. Feeds BAT/marginal-cost/residual methodology.
 
-| File                                                  | Purpose                                                                                                                                                                                                                                                                                                       |
-| ----------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
-| fairness_in_cost_allocation.md                        | Step-by-step philosophical guide to fairness in cost allocation: beneficiary-pays principle, why cost-causation works for energy and new capacity but fails for sunk costs, benefit-proportional sunk cost allocation, practical options, and how the BAT framework measures departures from these principles |
-| ny_psc_how_ecos_and_mcos_are_used.md                  | How NY allocates electric utility costs between customer classes: ECOS for inter-class revenue allocation (NARUC three-step), MCOS for rate design and DER compensation (VDER, marginal-cost floors, TOU guidance). Dual-study framework.                                                                     |
-| residual_allocation_in_solar_cross_subsidy_studies.md | Allocation mechanics in solar cross-subsidy studies: ECOS vs marginal-cost-plus-residual, NARUC three-step, demand allocators (CP, NCP, AED), residual treatment; how allocator choice affects measured cross-subsidy.                                                                                        |
+| File                                                  | Purpose                                                                                                                                                                                                                                                                                                                       |
+| ----------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
+| fairness_in_cost_allocation.md                        | Step-by-step philosophical guide to fairness in cost allocation: beneficiary-pays principle, why cost-causation works for energy and new capacity but fails for sunk costs, benefit-proportional sunk cost allocation, practical options, and how the BAT framework measures departures from these principles                 |
+| ny_psc_how_ecos_and_mcos_are_used.md                  | How NY allocates electric utility costs between customer classes: ECOS for inter-class revenue allocation (NARUC three-step), MCOS for rate design and DER compensation (VDER, marginal-cost floors, TOU guidance). Dual-study framework.                                                                                     |
+| residual_allocation_in_solar_cross_subsidy_studies.md | Allocation mechanics in solar cross-subsidy studies: ECOS vs marginal-cost-plus-residual, NARUC three-step, demand allocators (CP, NCP, AED), residual treatment; how allocator choice affects measured cross-subsidy.                                                                                                        |
+| supply_vs_delivery_cost_allocation.md                 | How cost allocation differs for supply (generation) and delivery (T+D): supply costs are observable and traceable to load (crude flat → load-weighted → TOU → RTP progression with worked examples), delivery costs are committed historically and attributed by proxy (ECOS); why the BAT matters most on the delivery side. |
 
 ### domain/charges/
 
@@ -97,19 +98,21 @@ Run ordering, Justfile dependency chain, and multi-step workflows.
 
 How CAIRO works: LMI, demand flex, tax rate, tiered rates, performance, parallelism.
 
-| File                                            | Purpose                                                                                                                                                                                                                                |
-| ----------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
-| cairo_lmi_and_bat_analysis.md                   | CAIRO LMI parameters, discount mechanisms, and how the Bill Alignment Test (BAT) works                                                                                                                                                 |
-| cairo_demand_flexibility_workflow.md            | CAIRO demand-flexibility workflow, two-pass RR recalibration, and data flow                                                                                                                                                            |
-| demand_flex_residual_treatment.md               | Frozen vs. fixed-RR residual treatment in demand flex: two approaches, economic interpretations, temporal assumptions, and when to use each                                                                                            |
-| cairo_utility_tax_rate.md                       | CAIRO `utility_tax_rate` parameter: how to apply percentage-of-bill charges (taxes, assessments, settlement surcharges) in both bill calcs and BAT; wiring through run_scenario, single-rate limitation, PSEG-LI NYSA/Shoreham example |
-| cairo_tiered_rates_support.md                   | CAIRO tiered rate support: precalc and bill calc use (period, tier); evidence from codebase                                                                                                                                            |
-| cairo_performance_analysis.md                   | CAIRO execution profile, compute bottlenecks, parallelism status, and speedup opportunities                                                                                                                                            |
-| cairo_parallelize_two_undasked_stages.md        | Handoff: parallelize process_residential_hourly_demand and BAT in CAIRO via chunk + dask.delayed                                                                                                                                       |
-| cairo_parallelism_and_workers.md                | How to think about parallelism: infra instance, worker count, series vs parallel tracks                                                                                                                                                |
-| cairo_elastic_cluster.md                        | Elastic Dask cluster: why (many runs), options (dask-cloudprovider etc.), CAIRO + platform changes                                                                                                                                     |
-| cairo_speedup_summary.md / cairo_speedup_log.md | Speedup notes and log                                                                                                                                                                                                                  |
-| er_bat_increase_under_hp_seasonal_rate.md       | Why electric resistance BAT increases under the HP seasonal rate: subclass RR split mechanics, non-HP flat rate rise, ER as collateral damage of HP-only fix                                                                           |
+| File                                               | Purpose                                                                                                                                                                                                                                |
+| -------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
+| cairo_lmi_and_bat_analysis.md                      | CAIRO LMI parameters, discount mechanisms, and how the Bill Alignment Test (BAT) works                                                                                                                                                 |
+| cairo_demand_flexibility_workflow.md               | CAIRO demand-flexibility workflow, two-pass RR recalibration, and data flow                                                                                                                                                            |
+| demand_flex_seasonal_elasticity_derivation_path.md | Seasonal ε + TOU derivation JSON (`find_tou_derivation_path`), YAML vs runtime stems, supply derivation gap; links to incident doc; hour-share / `FLEX_SHIFT_MIN_PERIOD_ABS_KWH` pointer                                               |
+| nimo_flex_demand_charge_regression.md              | NiMo run 15 incident: ε wiring (`185519c`, `fee4706`), TOU derivation alignment, PV/near-zero **`q_orig`** division bug, `utils/cairo.py` fix + tests, statewide verify scripts; **`ur_dc_enable`** note                               |
+| demand_flex_residual_treatment.md                  | Frozen vs. fixed-RR residual treatment in demand flex: two approaches, economic interpretations, temporal assumptions, and when to use each                                                                                            |
+| cairo_utility_tax_rate.md                          | CAIRO `utility_tax_rate` parameter: how to apply percentage-of-bill charges (taxes, assessments, settlement surcharges) in both bill calcs and BAT; wiring through run_scenario, single-rate limitation, PSEG-LI NYSA/Shoreham example |
+| cairo_tiered_rates_support.md                      | CAIRO tiered rate support: precalc and bill calc use (period, tier); evidence from codebase                                                                                                                                            |
+| cairo_performance_analysis.md                      | CAIRO execution profile, compute bottlenecks, parallelism status, and speedup opportunities                                                                                                                                            |
+| cairo_parallelize_two_undasked_stages.md           | Handoff: parallelize process_residential_hourly_demand and BAT in CAIRO via chunk + dask.delayed                                                                                                                                       |
+| cairo_parallelism_and_workers.md                   | How to think about parallelism: infra instance, worker count, series vs parallel tracks                                                                                                                                                |
+| cairo_elastic_cluster.md                           | Elastic Dask cluster: why (many runs), options (dask-cloudprovider etc.), CAIRO + platform changes                                                                                                                                     |
+| cairo_speedup_summary.md / cairo_speedup_log.md    | Speedup notes and log                                                                                                                                                                                                                  |
+| er_bat_increase_under_hp_seasonal_rate.md          | Why electric resistance BAT increases under the HP seasonal rate: subclass RR split mechanics, non-HP flat rate rise, ER as collateral damage of HP-only fix                                                                           |
 
 ### code/data/
 

context/code/cairo/cairo_demand_flexibility_workflow.md

@@ -12,6 +12,8 @@ The core CAIRO demand-response functions are commented out in `cairo/rates_tool/
 - `_shift_building_hourly_demand(...)`
 - runtime wrapper: `apply_runtime_tou_demand_response(...)`
 
+**Period-level shift → hourly rows:** elasticity gives a **`load_shift`** per building per TOU **period**; the code allocates it with **`hour_share`** so hours that used a larger share of the period’s baseline kWh absorb a larger share of the shift. For **PV** / netting, the period sum **`q_orig`** can be **tiny**; dividing **`electricity_net / q_orig`** without a floor **blows up** hourly values. The repo uses **`FLEX_SHIFT_MIN_PERIOD_ABS_KWH`**, **`_flex_shift_hour_share_from_groups`**, and **`_zero_unsafe_period_shifts_and_rebalance`** (see **`context/code/cairo/nimo_flex_demand_charge_regression.md`**).
+
 The runtime wrapper supports both:
 
 - full-year TOU shifting, and
@@ -25,7 +27,7 @@ The multi-phase demand-flex orchestration (phases 1a, 1.5, 1.75, 2) lives in `ut
 
 - `apply_demand_flex(...)` — entry point called by `run_scenario.py`; returns a `DemandFlexResult` dataclass
 - `is_diurnal_tou(tariff_path)` — detect TOU tariffs with intra-day rate variation
-- `find_tou_derivation_path(tariff_key, tou_derivation_dir)` — locate the TOU derivation JSON for a tariff
+- `find_tou_derivation_path(tariff_key, tou_derivation_dir)` — locate `{base}_derivation.json` from the tariff JSON **stem** (after `_parse_path_tariffs` rekeying; YAML `all` is not the runtime key). Supply stems often end in `…_supply` with no matching JSON; see **`context/code/cairo/demand_flex_seasonal_elasticity_derivation_path.md`**
 - `recompute_tou_precalc_mapping(...)` — recompute precalc rel_values from shifted-load MC weights (Phase 1.75)
 
 The scenario entrypoint is `rate_design/hp_rates/run_scenario.py`, which delegates demand-flex orchestration to `utils/demand_flex.py:apply_demand_flex()` and handles the CAIRO simulation.