Skip to content

New Typedef library#1204

Draft
castwide wants to merge 114 commits into
masterfrom
typedefs
Draft

New Typedef library#1204
castwide wants to merge 114 commits into
masterfrom
typedefs

Conversation

@castwide
Copy link
Copy Markdown
Owner

@castwide castwide commented May 28, 2026 •
edited
Loading

Certain features that involve resolution and inference have their responsibilities spread across multiple components. ComplexType, for example, has methods for resolving namespaces, generics, and self. The processes for type inference are scattered between ApiMap, Pin::*, and Source::Chain.

The Typedef library provides a single authoritative and testable source for the following:

  • Qualifying namespaces
  • Inferring types from code
  • Expanding generics
  • Processing macros

This PR will add Typedef to the codebase but will not modify existing functionality. In cases where I needed to make a change elsewhere in the codebase, I avoided conflicts by implementing a new method with a typedef_ prefix (e.g., Pin::Base#typedef_generics). This should allow us to migrate features iteratively.

Typedef::Typeset will eventually replace ComplexType. To allow for iterative migration, there are methods for converting between Typeset and ComplexType.

In addition to ComplexType, Typedef will preclude the need for the following methods:

  • ApiMap#get_complex_type_methods
  • ApiMap#can_resolve_generics?
  • ApiMap#erase_generics
  • ApiMap#has_generics?
  • ApiMap#process_macros
  • ApiMap#should_erase_generics_when_done?
  • ApiMap#var_at_location
  • Pin::Base#infer
  • Pin::Base#typify
  • Pin::Base#probe
  • Pin::Base#binder
  • Pin::Block#rebind
  • Pin::Closure#rebind
  • Source::Chain#define
  • Source::Chain#infer
  • Source::Chain::Link#resolve

Pins and chains will still exist, but they will largely be reduced to sources of data that Typedef consumes. Typedef::Dictionary will provide methods for defining and inferring types.

lekemula and others added 30 commits May 28, 2026 03:08
@lekemula @claude @castwide
Add YARD macro support for DSL methods (#1187)
ff6c1b2 
* Port macros from lekemula/solargraph@lm-named-macros (4572e07..389def6)

Original 11-commit diff:
lekemula/solargraph@524c94e...389def6

Squashes the original branch and ports it onto current upstream/master,
where the YardMap class was gutted and replaced with DocMap/GemPins
(upstream 94006fb).

Differences from the original implementation:

- Parser layer: original work added `simple_convert` and `process_dsl_method`
  to `parser/rubyvm/{node_methods,node_processors/send_node}`. Upstream
  removed the rubyvm parser entirely. Rewrote both for the parser_gem AST
  shape: lowercase node types (`:send`, `:hash`, `:const`, `:array`),
  `:send` children indexed as `[receiver, method_name, *args]`, literals
  split into `:int`/`:float`/`:sym`/`:str` instead of `:LIT`.

- ApiMap integration: original `process_macros(pins)` hooked into a `pins`
  parameter that no longer exists. Adapted to the new `catalog(bench)`
  flow — consumes `iced_pins + live_pins + doc_map.pins`, filters
  `Pin::Ephemeral::ClassMethodSend` from iced and live separately before
  the store update. Kept the original logging.

- MethodDirective: original `Parser.process_node(...).first.last`
  regressed `spec/source_map/mapper_spec.rb:89`. Upstream had since added
  a `Pin::Method` filter inline; backported that into the extracted
  directive module.

- Spec relocation: `spec/yard_map_spec.rb` was deleted upstream. The
  `loads macros from gems` test moved to `spec/yard_map/mapper_spec.rb`
  and uses the new `pins_with(name)` (DocMap-based) helper. Assertion
  tightened from `macros.count > 0` to checking that the
  `MyStruct.my_attribute` method pin exists and exposes the macro by
  name.

- All other new files (Macro, Directives::*, Pin::Ephemeral::*,
  gem-with-yard-macros fixture, api_map_spec/clip_spec additions) landed
  unchanged from the squashed branch.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

* Fix invalid gemspec for gem-with-yard-macros fixture

The skeleton gemspec from `bundle gem` left TODO placeholders in
summary, description, homepage, and metadata fields, which Bundler
rejects in CI. Replaced with real values describing the fixture's
purpose and trimmed the file list to `lib/**/*.rb` so it doesn't depend
on `git ls-files` working in the CI checkout.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

* Fix rubocop offenses

- Autocorrected style issues across the new/ported files (string quoting,
  empty-method one-liners, redundant cop disables, def-without-parens, etc).
- Excluded the gem-with-yard-macros fixture from rubocop entirely; it's a
  `bundle gem` skeleton that exists to be loaded as a gem, not as project
  source.
- Bumped Metrics/ModuleLength.Max in the todo file from 167 to 195 to
  accommodate the simple_convert helpers added to ParserGem::NodeMethods.
- Cleaned up YARD `@param` mismatches in Macro and ClassMethodSend, and
  rewrote one multi-line block chain in Macro#generate_yardoc_from.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

* Trim gem-with-yard-macros fixture to essentials

Removed the `bundle gem` skeleton boilerplate (LICENSE, README, CHANGELOG,
CODE_OF_CONDUCT, Rakefile, bin/, the gem's own Gemfile/Gemfile.lock, RBS
sig, .gitignore). None are needed: the fixture exists only to be resolved
as a path gem and have its YARD macro loaded. What remains is the
gemspec, the macro definition, and version.rb.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

* Set source: :yard_map on directive-generated pins

`Pin::Base#assert_source_provided` raises (under SOLARGRAPH_ASSERTS=on,
as the overcommit CI job runs) when a pin is created without a `source:`.
The extracted attribute/override directive modules built `Pin::Method`,
`Pin::Parameter`, and `Pin::Reference::Override` pins without one.
Tagged them `:yard_map` since they originate from YARD `@!` directives.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

* Fix pre-existing rubocop offenses in untouched files

The `.rubocop_todo.yml` CI job runs `rubocop -c .rubocop.yml` across the
whole repo and was failing on 8 offenses unrelated to this PR. Fixed them
in place rather than suppressing:

- Style/ArgumentsForwarding: anonymous block forwarding (`&`) in
  Solargraph.with_clean_env, UniqueType#each, Host#show_message_request.
- Style/ArrayIntersect: `(a & b).any?` -> `a.intersect?(b)` in
  TypeChecker#parameterized_arity_problems_for.
- Lint/UnreachableCode: the body of Pin::Method#combine_same_type_arity_
  signatures is intentionally preserved behind a debug stub `return`
  (upstream 6d8ce95); wrapped it in a scoped rubocop:disable with a
  comment explaining why, instead of deleting the kept code.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

* Fix ArgumentValue struct init on Ruby < 3.2

`ArgumentValue = Struct.new(:value)` was constructed with a keyword
argument (`ArgumentValue.new(value: ...)`). On Ruby 3.1 a plain Struct
treats that as a positional Hash, so `#value` returned `{ value: x }`
instead of `x`. That garbled `ClassMethodSend#argument_values`, which
shifted every macro placeholder (`$1`, `$2`, ...) — producing method
pins like `value` and dropping real ones. Added `keyword_init: true`.

Fixes the 6 macro specs failing on the Ruby 3.1 CI matrix job.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

* Drop 'head' from RSpec matrix temporarily

ruby/setup-ruby@v1 currently 404s on `head` for ubuntu-24.04
("Unavailable version head for ruby"). Removed it from the matrix so CI
isn't blocked; left a @todo to restore once setup-ruby publishes it.

See: https://github.com/castwide/solargraph/actions/runs/25863741955/job/76000137015?pr=1187

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

* Fix strong typechecking

---------

Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
@lekemula @castwide
YARD macros - More typecheck fixes (#1188)
d282a3b 
* More typecheck fixes

* Even more typecheck fixes - is this a dead code?

* ApiMap fix

* Mapper fix

/home/runner/work/solargraph/solargraph/lib/solargraph/yard_map/mapper.rb:28
- Unresolved call to filename on Solargraph::Location, nil

* NodeMethods fix

/home/runner/work/solargraph/solargraph/lib/solargraph/parser/parser_gem/node_methods.rb:107
- Declared return type ::String, ::Integer, ::Float, ::Symbol, ::Array,
::Hash, ::Solargraph::Source::Chain, nil does not match inferred type
::String, ::Parser::AST::Node, ::Array, ::Hash,
::Solargraph::Source::Chain, nil for
Solargraph::Parser::ParserGem::NodeMethods.simple_convert
/home/runner/work/solargraph/solargraph/lib/solargraph/parser/parser_gem/node_methods.rb:107
- Declared return type ::String, ::Integer, ::Float, ::Symbol, ::Array,
::Hash, ::Solargraph::Source::Chain, nil does not match inferred type
::String, ::Parser::AST::Node, ::Array, ::Hash,
::Solargraph::Source::Chain, nil for
Solargraph::Parser::ParserGem::NodeMethods#simple_convert
@castwide @lekemula @claude
Macro fixes (#1189)
44917c8 
* Port macros from lekemula/solargraph@lm-named-macros (4572e07..389def6)

Original 11-commit diff:
lekemula/solargraph@524c94e...389def6

Squashes the original branch and ports it onto current upstream/master,
where the YardMap class was gutted and replaced with DocMap/GemPins
(upstream 94006fb).

Differences from the original implementation:

- Parser layer: original work added `simple_convert` and `process_dsl_method`
  to `parser/rubyvm/{node_methods,node_processors/send_node}`. Upstream
  removed the rubyvm parser entirely. Rewrote both for the parser_gem AST
  shape: lowercase node types (`:send`, `:hash`, `:const`, `:array`),
  `:send` children indexed as `[receiver, method_name, *args]`, literals
  split into `:int`/`:float`/`:sym`/`:str` instead of `:LIT`.

- ApiMap integration: original `process_macros(pins)` hooked into a `pins`
  parameter that no longer exists. Adapted to the new `catalog(bench)`
  flow — consumes `iced_pins + live_pins + doc_map.pins`, filters
  `Pin::Ephemeral::ClassMethodSend` from iced and live separately before
  the store update. Kept the original logging.

- MethodDirective: original `Parser.process_node(...).first.last`
  regressed `spec/source_map/mapper_spec.rb:89`. Upstream had since added
  a `Pin::Method` filter inline; backported that into the extracted
  directive module.

- Spec relocation: `spec/yard_map_spec.rb` was deleted upstream. The
  `loads macros from gems` test moved to `spec/yard_map/mapper_spec.rb`
  and uses the new `pins_with(name)` (DocMap-based) helper. Assertion
  tightened from `macros.count > 0` to checking that the
  `MyStruct.my_attribute` method pin exists and exposes the macro by
  name.

- All other new files (Macro, Directives::*, Pin::Ephemeral::*,
  gem-with-yard-macros fixture, api_map_spec/clip_spec additions) landed
  unchanged from the squashed branch.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

* Fix invalid gemspec for gem-with-yard-macros fixture

The skeleton gemspec from `bundle gem` left TODO placeholders in
summary, description, homepage, and metadata fields, which Bundler
rejects in CI. Replaced with real values describing the fixture's
purpose and trimmed the file list to `lib/**/*.rb` so it doesn't depend
on `git ls-files` working in the CI checkout.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

* Fix rubocop offenses

- Autocorrected style issues across the new/ported files (string quoting,
  empty-method one-liners, redundant cop disables, def-without-parens, etc).
- Excluded the gem-with-yard-macros fixture from rubocop entirely; it's a
  `bundle gem` skeleton that exists to be loaded as a gem, not as project
  source.
- Bumped Metrics/ModuleLength.Max in the todo file from 167 to 195 to
  accommodate the simple_convert helpers added to ParserGem::NodeMethods.
- Cleaned up YARD `@param` mismatches in Macro and ClassMethodSend, and
  rewrote one multi-line block chain in Macro#generate_yardoc_from.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

* Trim gem-with-yard-macros fixture to essentials

Removed the `bundle gem` skeleton boilerplate (LICENSE, README, CHANGELOG,
CODE_OF_CONDUCT, Rakefile, bin/, the gem's own Gemfile/Gemfile.lock, RBS
sig, .gitignore). None are needed: the fixture exists only to be resolved
as a path gem and have its YARD macro loaded. What remains is the
gemspec, the macro definition, and version.rb.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

* Set source: :yard_map on directive-generated pins

`Pin::Base#assert_source_provided` raises (under SOLARGRAPH_ASSERTS=on,
as the overcommit CI job runs) when a pin is created without a `source:`.
The extracted attribute/override directive modules built `Pin::Method`,
`Pin::Parameter`, and `Pin::Reference::Override` pins without one.
Tagged them `:yard_map` since they originate from YARD `@!` directives.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

* Fix pre-existing rubocop offenses in untouched files

The `.rubocop_todo.yml` CI job runs `rubocop -c .rubocop.yml` across the
whole repo and was failing on 8 offenses unrelated to this PR. Fixed them
in place rather than suppressing:

- Style/ArgumentsForwarding: anonymous block forwarding (`&`) in
  Solargraph.with_clean_env, UniqueType#each, Host#show_message_request.
- Style/ArrayIntersect: `(a & b).any?` -> `a.intersect?(b)` in
  TypeChecker#parameterized_arity_problems_for.
- Lint/UnreachableCode: the body of Pin::Method#combine_same_type_arity_
  signatures is intentionally preserved behind a debug stub `return`
  (upstream 6d8ce95); wrapped it in a scoped rubocop:disable with a
  comment explaining why, instead of deleting the kept code.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

* Fix ArgumentValue struct init on Ruby < 3.2

`ArgumentValue = Struct.new(:value)` was constructed with a keyword
argument (`ArgumentValue.new(value: ...)`). On Ruby 3.1 a plain Struct
treats that as a positional Hash, so `#value` returned `{ value: x }`
instead of `x`. That garbled `ClassMethodSend#argument_values`, which
shifted every macro placeholder (`$1`, `$2`, ...) — producing method
pins like `value` and dropping real ones. Added `keyword_init: true`.

Fixes the 6 macro specs failing on the Ruby 3.1 CI matrix job.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

* Drop 'head' from RSpec matrix temporarily

ruby/setup-ruby@v1 currently 404s on `head` for ubuntu-24.04
("Unavailable version head for ruby"). Removed it from the matrix so CI
isn't blocked; left a @todo to restore once setup-ruby publishes it.

See: https://github.com/castwide/solargraph/actions/runs/25863741955/job/76000137015?pr=1187

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

* Fix strong typechecking

* Resolve paths for macro methods

* Handle macros dynamically

* Reinstate optional cache clearing

* Deprecate Ephemeral::ClassMethodSend

* Avoid chains for macro resolution when possible

* Minor refactor

* Pending specs

* Clarify macro spec for keyword arguments

* Linting

* Erroneous reversions

* Typechecking

* Minor spec tweak for Ruby 3.x

* More erroneous reversions

---------

Co-authored-by: Lekë Mula <l.mula@finlink.de>
Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
@castwide
Unused macro methods (#1191)
3cc4c9b 
* Unused macro methods

* Update Ruby to 4.0 in typecheck workflow

* Revert unneeded sg-ignore

* Update to Ruby 4.0 in plugins workflow
@castwide
Infer method calls with parameter names in return types
6ffe726
@castwide
Retain existing tags
5dbd500
@castwide
JIT macro expansion
a90492d
@castwide
Expand types from chain calls
f2b4c7e
@castwide
Typedef classes
d9c8522
@castwide
ComplexType to Typedef::Type
11bc2b8
@castwide
Resolve named tokens
b7c02db
@castwide
Resolve rooted paths
4bc3614
@castwide
Type resolves rooted params
abac126
@castwide
First iteration of Typedef-based inference
6161eab
@castwide
Typedef resolves generics
7656f90
@castwide
Generic syntax
68e3199
@castwide
Typedef chain inference
74f91b7
@castwide
Typedef::Dictionary
7d42b49
@castwide
Dictionary#closure
b0f78df
@castwide
Replace Inference with Dictionary
e971284
@castwide
DRY Dictionary#define
b8ce2fe
@castwide
Typedef::Linker
356bb2e
@castwide
Dictionary#initialize parameters
6a48f8a
@castwide
Param documentation
3ec3101
@castwide
Probe from Dictionary#infer
2389353
@castwide
Unneeded path exception
dedc649
@castwide
Typedef inspects invalid token values
1c1fb45
@castwide
Redundant code
8afe0ae
@castwide
Infer from new
b040722
@castwide
Token resolution and probes in Dictionary#infer_from
9821d07
castwide added 26 commits May 28, 2026 03:09
@castwide
Enable nilable types
ef7dbb9
@castwide
Redundant line
0fc8094
@castwide
Dictionary handles local variable inference
e41ff02
@castwide
Minor refactoring
aa195e4
@castwide
Expand on Dictionary#infer
5bd452d
@castwide
Move local variable inference back to call linker
cb60eef
@castwide
First iteration of generics
f81dbdc
@castwide
Tweaking infer_proxies
e8dbb68
@castwide
Comment
5ba33d1
@castwide
Expansion and inference issues
bcea361
@castwide
Fix variable inference from dictionary
16ef90b
@castwide
Expand generics from same pin
190e915
@castwide
Minor refactoring
e7d3a84
@castwide
Generics class
37238f7
@castwide
Refactor Generics
aeeaa18
@castwide
WIP specs
b431bad
@castwide
Organize specs
1b631e8
@castwide
Move more generics specs
a3dfbe5
@castwide
Generics#names
63518dd
@castwide
Type/Pin conversions
2c46b27
@castwide
Pin::Base#typedef_generics
a7f479e
@castwide
Concrete Generics#names specs
af0d1bd
@castwide
Enable stubbed specs
215dd0e
@castwide
Typedef::Typeset
40efc04
@castwide
Typeset specs
b03e141
@castwide
Typeset#expand
475af41
@castwide castwide mentioned this pull request May 30, 2026
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@castwide @lekemula