Skip to content

perf: Defer black import to first use#4

Open
KRRT7 wants to merge 4 commits intomainfrom
perf/defer-black
Open

perf: Defer black import to first use#4
KRRT7 wants to merge 4 commits intomainfrom
perf/defer-black

Conversation

@KRRT7
Copy link
Copy Markdown
Owner

@KRRT7 KRRT7 commented Apr 10, 2026

  • Defer import black from module level to first use in answers.py and utils.py
  • black (code formatter + transitive deps: pathspec, black.nodes, etc.) loaded on every import typeagent but only used in two cold formatting paths

black.format_str() is called in two places:

  • create_context_prompt() in knowpro/answers.py — formats debug context for LLM prompts
  • format_code() in aitools/utils.py — developer pretty-print utility

Neither runs during normal library operation. Moving the import inside each function eliminates ~78ms of transitive module loading from the import chain.

Benchmark

Azure Standard_D2s_v5 — 2 vCPU, 8 GiB RAM, Python 3.13

Import Time (hyperfine, warmup 5, min-runs 30)

Benchmark Before After Speedup
import typeagent 791 ms ± 11 ms 713 ms ± 8 ms 1.11x

Offline E2E Test Suite (hyperfine, warmup 2, min-runs 10)

Benchmark Before After Speedup
69 offline tests 5.72 s ± 90 ms 5.60 s ± 98 ms 1.02x

Generated by codeflash optimization agent

Reproduce the benchmark locally
# Verify black is in the import chain on main:
git checkout main
uv run python -X importtime -c 'import typeagent' 2>&1 | grep black
# Expected: multiple black.* lines (black.nodes, black.mode, pathspec, etc.)

# Verify black is deferred on this branch:
git checkout perf/defer-black
uv run python -X importtime -c 'import typeagent' 2>&1 | grep black
# Expected: no output (black no longer imported)

# A/B benchmark (requires hyperfine: https://github.com/sharkdp/hyperfine):
hyperfine --warmup 5 --min-runs 30 --shell=none \
  --prepare 'git checkout main && uv sync -q' \
  --command-name main \
  'uv run python -c "import typeagent"' \
  --prepare 'git checkout perf/defer-black && uv sync -q' \
  --command-name defer-black \
  'uv run python -c "import typeagent"'

Test plan

  • 69 offline tests pass (uv run pytest)
  • A/B benchmarked on dedicated non-burstable VM
  • python -X importtime confirms black no longer in import chain

parse_azure_endpoint returned the raw URL including ?api-version=...
which AsyncAzureOpenAI then mangled into invalid paths like
...?api-version=2024-06-01/openai/. Strip the query string before
returning — api_version is already returned as a separate value and
passed to the SDK independently.
black is only used in create_context_prompt() and format_code() -- both
cold paths. Moving the import inside the functions avoids loading black
and its transitive deps (pathspec, black.nodes, etc.) on every
import typeagent.
@KRRT7 KRRT7 force-pushed the perf/defer-black branch from 36ae570 to bcf75b4 Compare April 10, 2026 10:21
KRRT7 and others added 2 commits April 10, 2026 09:23
…crosoft#231)

- `parse_azure_endpoint` returned the full URL including
`?api-version=...`
- `AsyncAzureOpenAI` appends `/openai/` to `azure_endpoint`, producing a
mangled URL with the query string in the path
- Now strips the query string with `str.split("?", 1)[0]` before
returning
- Added 6 unit tests covering: basic URL, no version, separate env var,
missing env var, empty query string

## Benchmark

No performance impact — this is a correctness fix.

---

*Generated by codeflash optimization agent*
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants