[ty] Move script environment initialization to the CLI and LSP - #28038
[ty] Move script environment initialization to the CLI and LSP#28038MichaReiser wants to merge 4 commits into
Conversation
Typing conformance resultsNo changes detected ✅Current numbersThe percentage of diagnostics emitted that were expected errors held steady at 97.69%. The percentage of expected errors that received a diagnostic held steady at 93.71%. The number of fully passing files held steady at 110/136. |
Memory usage reportMemory usage unchanged ✅ |
|
| Project | Old Time | New Time | Change |
|---|---|---|---|
dd-trace-py |
2.91s | 1.20s | -59% |
Flaky changes detected. This PR summary excludes flaky changes; see the HTML report for details.
96be0e7 to
5323898
Compare
|
ea88848 to
94b0ebf
Compare
Performance comparison unavailableThe base and head of this comparison were measured with different runner settings, so their benchmark values are not directly comparable. What changed between base and head:
Re-run the base with the same settings to get a valid performance comparison. Comparing |
e8d77af to
518956b
Compare
Classify PEP 723 scripts during file discovery and keep their membership updated when files or inline metadata change. Expose an iterable script view that also includes explicitly opened files. Restore the saved file classification when an editor overlay is closed, so discarded script metadata cannot leave an ordinary file excluded. Keep environment initialization and program selection unchanged.
518956b to
33b7a37
Compare
| let exclude_script = is_script | ||
| && project.settings(self).src().exclude_scripts | ||
| && !project.is_file_explicitly_included(self, file); |
There was a problem hiding this comment.
This is sort of an unrelated fix. Previously, we always added a script to the known project files even when exclude_scripts was on, which was wrong. We now respect the exclude_scripts setting
642d7f1 to
fd65ba2
Compare
There was a problem hiding this comment.
We now create a single progress indicator to sync scripts. This is important now, because VS Code otherwise shows one progress bar per script, which could get annoying real quick in a workspace with many scripts
61a7e15 to
09e911f
Compare
Request script synchronization from CLI and language-server startup and file lifecycle handling. Analysis uses the available environment without starting or waiting for uv, and poll_sync applies completed results. The CLI waits for requested synchronization before checking. The language server waits to report initial semantic diagnostics while other editor operations can continue. Document pulls return empty reports until the initial environment is available, then receive a diagnostic refresh. Remove blocking initialization and its separate worker mode. Submission does not wait for queue space; per-owner coalescing and the worker limit keep outstanding work bounded by environments rather than edits.
Show completed and total script synchronizations in the CLI and language server. Give CLI synchronization batches and checking passes independent progress bars, retaining a separate status line for each running uv command. Keep one LSP indicator from the first queued script until all requests finish, displaying the last started script. Count a request only after its final result is handled, and retain its reporter across replacements.
Return the project refresh path from apply_changes instead of submitting the uv request while applying filesystem changes. Let the CLI and language server schedule the refresh alongside script synchronization. Remove the progress factory from the database change API and update the watcher integration helper to submit the returned refresh requests. Preserve the existing refresh conditions, ancestor fallback, and uv coordination.
09e911f to
aed1c7f
Compare
Summary
This PR moves from a lazy PEP 723 script discovery do eagerly discover all scripts as part of the project's file discovery.
There are two motivations for doing this:
file_settingsandpython_version_with_sourcewhich are both environment dependent (the former less so, but uv can tell us the python version). Initializing all scripts before we start checking ensures that type inference sees the right environment even when a script has been imported. This is the bug that Zsolt identified in my initial PEP 723 PR.Now, there are a few downsides as well, but I think this is the right trade off (honestly, I don't know how to fix 1 easily without it, and 3 was the main reason why I considered this approach even before Zsolt found the bug).
didChangeordidOpento block, only because there are already 8 queued scripts. But more importantly, we don't want to block the LSP startup until all scripts have synchronized, but queuing them up during startup would just have done that (because the main loop blocks everytime time the queue is full). I think this is okay, and I'll add another PR on top of this one that adds cancellation support for sync requests. I also think that our previous back pressure wasn't as impactful as I thoughtI think this also gets us into a position where adding support for nested projects becomes much easier, because they'll naturally fit in. We'll discover nested projects as part of indexing
Files(we'd need to move that offProject), synchronizing their environments can happen at the exactly same points as synchronizing scripts. The biggest remaining lift is to changety_projectandProjectDatabaseto no longer assume that there's a singleProject(but there is still one main project).Test Plan
CLI
Screen.Recording.2026-08-27.at.16.29.49.mov
Screen.Recording.2026-08-27.at.16.36.21.mov