Skip to content

Align constant resolution with CRuby - #1091

Draft
paracycle wants to merge 10 commits into
mainfrom
sf/67e17db9-96a9-47db-9815-15d6d7ae6b60
Draft

paracycle wants to merge 10 commits into
mainfrom
sf/67e17db9-96a9-47db-9815-15d6d7ae6b60

Conversation

@paracycle

@paracycle paracycle commented Oct 27, 2026 •

Copy link
Copy Markdown
Member

This aligns Rubydex's constant resolution with CRuby in nine places. Fixes #1073. Each commit fixes one behaviour and adds tests that fail without it, for both indexer backends, plus a note in docs/ruby-behaviors.md.

The gaps came from comparing Rubydex with a formal model of CRuby's constant lookup (ported from vm_get_ev_const, rb_const_search and rb_const_search_from). Every case was then checked against real Ruby (4.0.9). On that differential suite, the number of probes where Rubydex and CRuby agree goes from 78 of 103 to 96 of 103. The remaining 7 are intentional; see below.

Changes

Commit Ruby behaviour Before
Check a namespace's own constants before its prepended modules rb_const_search_from reads the starting namespace's own table first, so with class C; prepend P; X = 2; end, C::X is C::X P::X
Stop qualified constant lookups at Object Foo::TOP raises NameError: a qualified search stops when the first owner is Object itself, unless it started at Object. Kernel/BasicObject constants past Object are still found resolved to ::TOP
Report constant paths that reference private constants Only :: enforces private_constant, even inside the class (Foo::PRIV in class Foo) and for ::PRIV. New PrivateConstantReference warning; the reference still resolves, so navigation keeps working no diagnostic
Include top level mixins into Object A top level include M is main.include, which includes M into Object M's constants unresolved
Treat ||= constant writes as conditional definitions VERSION ||= "2" in a subclass of a class that defines VERSION defines nothing declared Child::VERSION
Make the last assignment the primary target of a reassigned alias X = A; X = B; X::C is B::C A::C
Resolve self::X directly in namespace bodies In a class, module or singleton class body, self::X is Foo::X dynamic reference
Don't resolve a superclass to the class being defined In module GSD; class ApplicationRecord < ApplicationRecord, the superclass is evaluated before the class exists, so it is ::ApplicationRecord (#1073) GSD::ApplicationRecord itself
Apply built-in mixins before user mixins when linearizing class Object; include M; end gives [Object, M, Kernel, BasicObject]: the VM includes Kernel before any code runs [Object, Kernel, M, BasicObject]

A follow-up commit refines the ||= change: when the existing constant is only defined later in the same file, the ||= still defines a new constant, because Ruby runs defined? at that point.

Design choices worth a look

  • Top level include is recorded as a synthetic reopening of Object: a ClassDefinition at the include call site, carrying the mixins. This reuses the existing linearization and incremental invalidation. The trade-off is that Object gains a definition at that location. Only statements directly in the program body count: inside a block (Struct.new do ... end), self is unknown. Top level prepend is a NoMethodError, and top level extend only affects main.
  • ||=: both indexers set a new DefinitionFlags::CONDITIONAL on ||= constant and alias definitions. Before declaring one, the resolver looks the name up the way defined? does: lexically for a bare name, with a qualified search for a path. If that finds a constant owned by another namespace and defined earlier (or in another file), the definition joins that declaration instead of creating a new one. If the lookup is blocked by a partial ancestor chain, it waits once and then falls back to the old behaviour.
  • PrivateConstantReference is a Warning, like the other resolution rules. Visibility is only known after the convergence loop, so the resolver remembers the constant paths it resolved in this run and checks them once visibilities are applied.
  • Namesake superclass: a new Name flag (before_definition) gives the superclass in class Foo < Foo its own name instead of sharing the class's name. Its lookup skips the declaration the class's own name resolves to. The flag only changes the NameId when it is set, so all existing ids are unchanged.
  • self::X is only resolved when no block encloses it within the namespace body. A block may be instance_eval'd or class_eval'd, and inside a method self depends on the receiver.

Known limitations and remaining differences

  • These are decided once, like other cross-file effects. If the constant an ||= deferred to is deleted, the conditional constant is re-declared in its own scope. Adding the original back doesn't undo that until the file with the ||= is re-indexed.
  • The 7 remaining differences in the differential suite:
    • 5 private constant paths. They resolve, which is intentional, but are now reported.
    • 1 load-order case: Ruby resolves against the constants that exist when the line runs, Rubydex against the final graph.
    • 1 self::X inside a block, where self cannot be known statically.
  • Mixins from different user files are still applied in URI order. That includes RBS core definitions, so when RBS core files and workspace files both mix modules into the same class, their relative order depends on the paths rather than on load order.

Testing

  • cargo test (1187 lib tests, including the new ones for both backends), cargo fmt --check and cargo clippy --all-targets are clean.
  • bundle exec rake compile and bundle exec rake ruby_test: 411 runs, 0 failures.
  • I ran each new test without its fix to confirm it fails.
  • Each Ruby behaviour above was checked with ruby 4.0.9.

Requested by Ufuk Kayserilioglu ufuk.kayserilioglu@shopify.com

AI Confidence Score: 4/5. All behaviours were verified against CRuby and covered by tests. The ||= and top level include representations are design choices the maintainers may want to change.

shopify-river and others added 9 commits October 26, 2026 23:45
CRuby's constant search (rb_const_search_from) reads the starting namespace's own constant table on its first iteration, and only skips a prepended class in favour of its origin on later iterations. So for `class C; prepend P; X = 2; end`, `C::X` is C's own X even though `C.ancestors` starts with P. The same applies to every search that starts from Object: top-level references, `::X` and the module fallback.

search_ancestors walked the linearization directly, so it found P::X first. It now checks the namespace's own members before walking its ancestors.

Co-authored-by: Ufuk Kayserilioglu <ufuk.kayserilioglu@shopify.com>
In Ruby, `Foo::TOP` raises NameError even though Object is one of Foo's ancestors. A qualified lookup (rb_public_const_get_from) passes `exclude` to rb_const_search_from: when the first owner it finds is Object itself and the search did not start at Object, the constant is not found. Owners that come after Object, like Kernel or BasicObject, are still found, so `Foo::KERNEL_CONST` resolves.

Rubydex searched the whole ancestor chain for `Foo::Bar`, so it resolved references that raise at runtime, and everything chained after them (`Foo::Bar::Baz`).

Co-authored-by: Ufuk Kayserilioglu <ufuk.kayserilioglu@shopify.com>
`private_constant` is only enforced by the `::` operator: `Foo::PRIV` raises NameError, even inside `class Foo` itself, and so does `::PRIV` for a private top-level constant. Unqualified references from the lexical scope or through inheritance are allowed.

Rubydex records constant visibility but resolution ignored it. Constant visibility is only known after the convergence loop, so the resolver now remembers the constant paths it resolves and, once visibilities are applied, reports each one that names a private constant with a new PrivateConstantReference warning. The references still resolve, so go-to-definition keeps working.

Co-authored-by: Ufuk Kayserilioglu <ufuk.kayserilioglu@shopify.com>
A top level `include M` is `main.include`, which includes M into Object, so M's constants become reachable from every class, from modules through the Object fallback, and as `Foo::MX`. Both indexers dropped mixins without an enclosing namespace, so those references stayed unresolved.

A top level `include` statement is now recorded as a reopening of Object at the call site, carrying the mixins. Only statements directly in the program body count: inside a block (`Struct.new do ... end`) `self` is unknown, top level `prepend` is a NoMethodError and top level `extend` only affects `main`.

Co-authored-by: Ufuk Kayserilioglu <ufuk.kayserilioglu@shopify.com>
`X ||= value` is `defined?(X) ? X : (X = value)`: it only defines X when the full lookup for X fails. A common case is `VERSION ||= ...` in a subclass or a reopened namespace, where the existing constant is inherited or lexically visible and nothing new is defined.

Both indexers now mark `||=` constant and alias definitions as CONDITIONAL. When the resolver reaches one, it first looks the name up the way `defined?` does (lexically for a bare name, through a qualified search for a constant path). If that finds a constant owned by another namespace, the definition joins that declaration and its name resolves there, instead of declaring a new constant in the current scope. If the lookup is blocked by a partial ancestor chain, the definition waits once, then falls back to the old behaviour.

Known limitation: like other cross-file effects, this is decided once. Deleting the existing constant later re-declares the conditional one in its own scope, but adding it back afterwards does not undo that until the file with the `||=` is re-indexed.

Co-authored-by: Ufuk Kayserilioglu <ufuk.kayserilioglu@shopify.com>
Reassigning a constant replaces its value, so after `X = A; X = B`, `X::C` is `B::C` and `module X` reopens B. A `||=` assignment only takes effect when nothing was assigned before it, so `Z = A; Z ||= B` leaves Z as A.

alias_targets returned targets in definition order and resolution treats the first one as primary, so Rubydex used the first assignment. Targets are now ordered with the last plain assignment first, then earlier plain assignments, then `||=` assignments in order. All targets are still returned, so constant paths through an alias with several possible values keep trying each of them.

Co-authored-by: Ufuk Kayserilioglu <ufuk.kayserilioglu@shopify.com>
Directly in a class, module or singleton class body, `self` is that namespace, so `self::X` is the same qualified lookup as `Foo::X`. Both indexers reported every `self::X` as a dynamic constant reference.

`self::X` is now indexed with the enclosing namespace as its parent scope when `self` is known statically: the innermost nesting is a class, module or singleton class body, and no block encloses the reference within it. Inside a method `self` depends on the receiver, and a block can run with another `self` (instance_eval, class_eval), so those stay dynamic.

Co-authored-by: Ufuk Kayserilioglu <ufuk.kayserilioglu@shopify.com>
Ruby runs the `defined?` check of `X ||= value` when the assignment executes. In

    module Early
      LATER ||= 1
    end
    LATER = 2

the top level LATER doesn't exist yet when the `||=` runs, so Early::LATER is defined. A conditional definition now only joins an existing constant if that constant has a definition in another file, or earlier in the same file. Definitions in other files may have been loaded first, so they still count.

Co-authored-by: Ufuk Kayserilioglu <ufuk.kayserilioglu@shopify.com>
The VM includes Kernel into Object before any code runs, so a module included into Object later lands in front of Kernel: `class Object; include M; end` gives `[Object, M, Kernel, BasicObject]`.

Definitions are processed in URI order, and `rubydex:built-in` sorts after `file://` URIs, so the built-in `include Kernel` was applied last and Kernel ended up in front of every user module included into Object (including top level includes). Linearization now applies the mixins of built-in definitions first, without changing the order definitions are listed in.

Co-authored-by: Ufuk Kayserilioglu <ufuk.kayserilioglu@shopify.com>
@paracycle paracycle self-assigned this Oct 27, 2026
In `class ApplicationRecord < ApplicationRecord` inside `module GSD`, Ruby evaluates the superclass before `class` creates GSD::ApplicationRecord, so the lookup continues past it and finds ::ApplicationRecord. A class can never be its own superclass: if GSD::ApplicationRecord already existed, the definition would raise a superclass mismatch.

The superclass name was the same name as the class's own name (same string, scope and nesting), so it resolved to the class being defined. Both indexers now mark a bare superclass with the class's own bare name as evaluated before the definition. That gives it a separate name, and its lookup skips the declaration the class's own name resolves to. The rest of the lookup is unchanged, so a namesake inherited by the enclosing namespace is still found.

Fixes #1073

Co-authored-by: Ufuk Kayserilioglu <ufuk.kayserilioglu@shopify.com>

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Incorrect resolved superclass name

2 participants