[v2] Implement conditional autoprompt importing#10231
Conversation
hssyoo
left a comment
There was a problem hiding this comment.
The measurements in the PR description show a ~5ms improvement, which is far from the ~66ms mean import time we measured. Based on these numbers, it seems highly likely that prompt_toolkit is still being imported elsewhere.
I was able to confirm this by checking sys.modules at the end of a aws --version call.
Great catch! I have analyzed |
e8d5797 to
e82544d
Compare
…ing in a more idiomatic way.
…gated to SingleWizardCommand instances.
Description of changes:
AutoPromptDriverobject. The mode resolution code has been moved toclidriver.pyas the standalone functionsresolve_auto_prompt_modeandvalidate_auto_prompt_args_are_mutually_exclusive.AutoPromptDriveris now only imported lazily when autoprompt mode is actually active.resolve_modeandvalidate_auto_prompt_args_are_mutually_exclusivetests fromtests/unit/autoprompt/test_core.pytotests/unit/test_clidriver.pyto reflect the new location of those functions.PrompterKeyboardInterruptinto a new minimalawscli/autoprompt/exceptions.pymodule, so thatawscli.errorhandlercan import it without pulling inawscli.autoprompt.factory(which has many top-levelprompt_toolkitimports).Displayimport inecs/monitorexpressgatewayservice.pylazy, deferring the import ofprompt_toolkit_display(and thereforeprompt_toolkit) until the interactive ECS monitoring display is actually used.PTKPromptandRequiredInputValidatorfromconfigure/sso.pyincustomizations/login/login.pylazy, deferring the import ofconfigure/sso.pyuntil the login region prompt is actually needed.StartLiveTailCommandandTailCommandincustomizations/logs/__init__.pylazy, deferring the import ofprompt_toolkituntil theaws logssubcommand is invoked (i.e. the logs subcommand table is built).ConfigureSSOCommand,ConfigureSSOSessionCommand, and their shared base classBaseSSOConfigurationCommandinto a newcustomizations/configure/sso_commands.pymodule, which has noprompt_toolkitimports. This allowsconfigure/configure.pyto keep its reference to these SSO commands and defer the import ofprompt_toolkit(viaconfigure/sso.py) untilaws configure ssois actually invoked.select_menufromwizard/ui/selectmenuinconfigure/sso.pylazy, deferring the import ofwizard/ui/__init__.py(which importsprompt_toolkit) untilConfigureSSOCommandis actually instantiated.create_wizard_appfromwizard/factoryinwizard/devcommands.pylazy, deferring the import of the wizard UI chain (which importsprompt_toolkit) until a wizard dev command is actually run.wizard/factoryinwizard/commands.pylazy by introducing a_get_runnermethod onTopLevelWizardCommand, deferring the import of the wizard factory and UI chain until a wizard command is actually executed.devcommandsinawscli/customizations/wizard/commands.pyfrom module-level to within theregister_wizard_commandsfunction. Although this currently gives no behavioral difference sinceregister_wizard_commandsis always called for every command, in the future when plugin/customization initialization is only called when a relevant command is entered (lazy plugin loading), this change will lead to lazy-importing ofdevcommandsonly when it's needed.Description of tests:
Benchmark Results: Performance Improvement Analysis
The following benchmarking analysis evaluates the impact of the Conditional Autoprompt feature across two standard CLI operations. Tests were performed with 2,000 samples per state to ensure statistical precision.
Tested on m7i.xlarge EC2 instance, x86 architecture, AL2023 os, 64 GB storage, 2,000 iterations each.
Consolidated Performance Gains
Statistical Significance & Reliability
To filter out environmental noise (CPU spikes, OS scheduling), we measured the performance shift in terms of Standard Error (SE).
aws --versionand ~379 SEs foraws sts get-caller-identity.Visual Analysis
Average execution times compared
Significant time savings are shown. We can expect similar improvements across all modeled commands, and some improvement(s) to customization commands.
Probability density functions of execution time
aws --version execution time probability.
aws sts get-caller-identity execution time probability.
The probability density curves in both graphs show no overlap between the "Before" and "After" states. The narrow, tall peaks confirm that the large sample size has provided a high-confidence estimate of the true performance improvement. For the version command, we note that the peak is taller and narrower in the "After" state, signaling that the patch not only improves execution time but also reduces noise.
Summary: The feature provides a consistent and significant performance reduction (11.70-13.35%) across different command types with statistical certainty.
By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.