[ty] Add "Find references" support for pytest fixtures

[lerebear]

Summary

This adds "Find References" support for pytest fixtures. It connects fixture declarations to imports and reexports that expose them, fixture requests in tests and fixture functions, and the requests' lexical uses. Searches can start from any of those locations and respect fixture visibility and shadowing across class hierarchies, modules, applicable conftest.py files, and fixtures registered by pytest's installed core plugins.

For example, given:

# src/test_example.py
import pytest

@pytest.fixture(name="resource")
def implementation(): ...

copy = implementation

@pytest.fixture
def dependent(resource):
    print(resource)

def test_use(resource):
    print(resource)

Finding references on implementation reports its ordinary Python alias together with requests for its public fixture name, resource, and the requests' uses:

info[references]: Found 6 references
  --> src/test_example.py:5:5
   |
 5 | def implementation(): ...
   |     --------------
 6 |
 7 | copy = implementation
   |        --------------
 8 |
 9 | @pytest.fixture
10 | def dependent(resource):
   |               --------
11 |     print(resource)
   |           --------
12 |
13 | def test_use(resource):
   |              --------
14 |     print(resource)
   |           --------

Closes astral-sh/ty#4115.

Approach

The fixture reference search proceeds as follows:

1. We map the selected symbol to a set of fixture function definitions that form its reference identity. For instance, we map a test parameter to the definition of the fixture function that it requests.
2. We find the function declarations, imports, and fixture requests related to the identity set in each eligible file. For example, we find all other test parameters that request a particular fixture.
3. Finally, we use the existing semantic reference visitor to find identifiers that resolve to one of those declarations, imports, or fixture requests.

Fixture references can involve several identifier names for the same fixture, but walking a file once per name would be expensive. As a result, this change introduce a LocalReferenceSearches data type which groups those searches such that each file's syntax tree is still traversed only once. Its Single variant keeps ordinary one-name reference searches on the existing direct-comparison path without allocating a hash map, but fixture searches that need multiple names use Multiple for fast lookup during the shared traversal.

Test Plan

See included tests.

[astral-sh-bot]

Typing conformance results

No changes detected ✅

Current numbers

The percentage of diagnostics emitted that were expected errors held steady at 97.79%. The percentage of expected errors that received a diagnostic held steady at 94.33%. The number of fully passing files held steady at 112/136.

[astral-sh-bot]

Memory usage report

Memory usage unchanged ✅

[astral-sh-bot]

ecosystem-analyzer results

No diagnostic changes detected ✅

Full report with detailed diff (timing results)

[MichaReiser]

It seems unfortunate to iterate over all files here, only to then iterate over all files again to find ordinary references. On large projects, this requires repeating a lot of work. It can also trash Salsa caches because most eviction algorithms aren't scan-resistent.

• Clone the db for each batch: The cloning itself is cheap, but droping a clone isn't because it requires some coordination work.
• Read the source_text of each file. This is where we start trashing our cache. This isn't the case today, but we could have a compressed source text and a recent-used uncompressed cache. Iterating twice will kill any form of recency
• Run a memchr search across the source
• Run a local reference finder

Can you say more why you decided for this split?

[MichaReiser]

Sorry for being a bit late on picking up on this. I think the implementation you have here makes sense, given how the pytest go-to-definition is defined, but it's inconsistent with how find references and go-to-definition work outside pytest. Reviewing the PRs in isolation never fully consider the LSP experience.

If we use this example (https://play.ty.dev/85c8eeb6-b0b5-4f4a-8779-b1928bacda63) without pytest support and search all references of def resource, it only returns def resource and the from fixtures import resource as alias. That makes sense to me, because alias is a new and independent definition. Renaming resource shouldn't rename alias, and the opposite is also true, renaming alias shouldn't rename resource.

The same behavior applies to goto definition. Clicking on alias in print(alias) jumps to the import statement. This is where the name is defined. You can then jump to resource, which jumps to the def resource(...).

Now, I can see that pytest fixtures are a bit different, but only if we enforce that alias has to point to a proper fixture for go to definition to offer it as a target. And I wonder if this is something we should do in the first place. Let's say you import a name which isn't a fixture and you run your test. Pytest then fails because you forgot to mark the imported symbol as a fixture. It would then be rather annoying if goto definition doesn't work.

That's why I think we should actually simplify pytest go to definition by simply implementing Pytest's name lookup. For functions that look like pytest functions, use it's name lookup to find the name in the enclosing scope and jump to it regardless on whether it is a pytest fixture or not. Don't resolve through aliases, matching our normal go to behavior. Clicking on an imported symbol also implements pytest resolution in test files.

This then also allows find references to work more like normal find references. And rename should also fit more naturally.

This might be worth talking about in person.