Problem Scaling - #1403
Open
eranschweitzer wants to merge 9 commits into
Open
Conversation
Makes it possible for a transmission line to have no lower or upper limit. In these cases, the constraints are skipped for that line and period. This is useful for modeling lines where the flow capacity is known to not be a limiting factor. An example may be an radial connection where congestion in or out is not expected to occur. This is also useful for using the simultaneous flow constraints to model PTDFs by splitting up zones into multiple sub-zones with unlimited radial connection to the "main" zone.
Add --power_scale_factor and --dollar_scale_factor run arguments that solve the model in scaled units (e.g. GW/GWh, millions of dollars) for better solver conditioning, then map the solution back to native units. Scaling is an exact affine reformulation applied via Pyomo's core.scale_model transformation, driven by a scaling_factor Suffix that gridpath/auxiliary/scaling.py builds from GridPath's naming conventions (power/energy vars, dollar vars, integer vars left unscaled; constraint factors inferred from their variables). Because the reformulation is exact for any positive factor assignment, the classifier only affects conditioning, never the optimum. solve_problem clones the built instance, solves the scaled clone, and maps values and duals back onto the original (native-unit) instance, so all downstream export/objective/dual code is unchanged. The default (1.0, 1.0) is an exact no-op that skips the clone entirely. A custom propagate_scaled_solution tolerates constraints the solver leaves without a dual (Pyomo's own propagate_solution raises on those). Co-Authored-By: Claude <noreply@anthropic.com>
Fast, DB-free tests for gridpath/auxiliary/scaling.py: variable-unit classification, factor assignment (container granularity, integer vars left unscaled, objective/constraint factors), exact solve equivalence including duals via the real solve_problem path, the no-op default path, and the guard rejecting scaling combined with load-solution / lp-only flags. Co-Authored-By: Claude <noreply@anthropic.com>
TestScaledExamples subclasses TestExamples and re-runs every example scenario with scaling on, checking the objective still matches the stored expected value to a relative tolerance (1e-6) instead of the parent's absolute places=1 (which only holds for bit-deterministic unscaled runs). It overrides only the comparison hook, so it inherits the test database setup and all example test_* methods automatically -- any example that later breaks under scaling fails here without being added by hand. Co-Authored-By: Claude <noreply@anthropic.com>
Degenerate LPs can settle on a different equally-optimal vertex once scaling perturbs the coefficients, especially scenarios dominated by commodity chains the classifier leaves unscaled (fuel, emissions, water). The observed worst case across the example suite is ~2.85e-6 (test_new_solar_carbon_cap_2zones_tx_hydrogen_prod_new), which exceeded the initial 1e-6. 1e-5 covers it with margin while real breakage (a crash or a gross objective difference) still fails loudly. Co-Authored-By: Claude <noreply@anthropic.com>
The default out-of-place scaling clones the built model (create_using),
which for large models is a significant fraction of setup time and
doubles peak memory. Add --scale_mode {out_of_place,in_place}: in_place
scales the model itself (apply_to, rename=False), solves it, then
restores native units via invert_scaled_solution_in_place -- reversing
the objective substitution and dividing out the objective factor,
un-scaling variable values, and rescaling duals (skipping any the solver
left unset). No clone, ~half the peak memory.
Both modes produce identical native-unit results; test_scaling.py checks
the equivalence directly, and TestScaledExamples gains a SCALE_MODE
class attribute so the full example sweep can be run through either path
by subclassing. Default remains out_of_place (keeps the original model
pristine).
Co-Authored-By: Claude <noreply@anthropic.com>
# Conflicts: # gridpath/run_scenario.py
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This merge request allows to scale power/energy quantities and cost/dollar quantities by a factor.
For example, passing
--power_scale_factor 1000to the run command will convert all MW and MWh quantities to GW and GWh quantities. Passing--dollar_scale_factor 1000000will scale dollars to millions of dollars.There are two modes possible:
*
--scale_mode out_of_placecreates a scaled clone of the model.*
--scale_mode in_placemodifies the pyomo model in place (can be a bit faster).NOTE: this only impacts the problem seen by the solver (HiGHs, gurobi, etc.). The pyomo model is transformed and the re-transformed so solution values do not change.
As part of the testing, all the example tests are re-run with scaling. The tests are setup, such that any new example test will automatically be inherited by the scaling tests. This way, in the event that a new feature breaks under scaling, as long as an example test is written for it, this will be flagged.
Finally, note this does not scale other quantities like MMBTU etc. but it would be a fairly trivial extension to do so in the future if/when desired.