Merge YARD macro support#1194
Merged
Merged
Conversation
lekemula
reviewed
May 18, 2026
lekemula
reviewed
May 20, 2026
* Port macros from lekemula/solargraph@lm-named-macros (4572e07..389def6) Original 11-commit diff: lekemula/solargraph@524c94e...389def6 Squashes the original branch and ports it onto current upstream/master, where the YardMap class was gutted and replaced with DocMap/GemPins (upstream 94006fb). Differences from the original implementation: - Parser layer: original work added `simple_convert` and `process_dsl_method` to `parser/rubyvm/{node_methods,node_processors/send_node}`. Upstream removed the rubyvm parser entirely. Rewrote both for the parser_gem AST shape: lowercase node types (`:send`, `:hash`, `:const`, `:array`), `:send` children indexed as `[receiver, method_name, *args]`, literals split into `:int`/`:float`/`:sym`/`:str` instead of `:LIT`. - ApiMap integration: original `process_macros(pins)` hooked into a `pins` parameter that no longer exists. Adapted to the new `catalog(bench)` flow — consumes `iced_pins + live_pins + doc_map.pins`, filters `Pin::Ephemeral::ClassMethodSend` from iced and live separately before the store update. Kept the original logging. - MethodDirective: original `Parser.process_node(...).first.last` regressed `spec/source_map/mapper_spec.rb:89`. Upstream had since added a `Pin::Method` filter inline; backported that into the extracted directive module. - Spec relocation: `spec/yard_map_spec.rb` was deleted upstream. The `loads macros from gems` test moved to `spec/yard_map/mapper_spec.rb` and uses the new `pins_with(name)` (DocMap-based) helper. Assertion tightened from `macros.count > 0` to checking that the `MyStruct.my_attribute` method pin exists and exposes the macro by name. - All other new files (Macro, Directives::*, Pin::Ephemeral::*, gem-with-yard-macros fixture, api_map_spec/clip_spec additions) landed unchanged from the squashed branch. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * Fix invalid gemspec for gem-with-yard-macros fixture The skeleton gemspec from `bundle gem` left TODO placeholders in summary, description, homepage, and metadata fields, which Bundler rejects in CI. Replaced with real values describing the fixture's purpose and trimmed the file list to `lib/**/*.rb` so it doesn't depend on `git ls-files` working in the CI checkout. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * Fix rubocop offenses - Autocorrected style issues across the new/ported files (string quoting, empty-method one-liners, redundant cop disables, def-without-parens, etc). - Excluded the gem-with-yard-macros fixture from rubocop entirely; it's a `bundle gem` skeleton that exists to be loaded as a gem, not as project source. - Bumped Metrics/ModuleLength.Max in the todo file from 167 to 195 to accommodate the simple_convert helpers added to ParserGem::NodeMethods. - Cleaned up YARD `@param` mismatches in Macro and ClassMethodSend, and rewrote one multi-line block chain in Macro#generate_yardoc_from. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * Trim gem-with-yard-macros fixture to essentials Removed the `bundle gem` skeleton boilerplate (LICENSE, README, CHANGELOG, CODE_OF_CONDUCT, Rakefile, bin/, the gem's own Gemfile/Gemfile.lock, RBS sig, .gitignore). None are needed: the fixture exists only to be resolved as a path gem and have its YARD macro loaded. What remains is the gemspec, the macro definition, and version.rb. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * Set source: :yard_map on directive-generated pins `Pin::Base#assert_source_provided` raises (under SOLARGRAPH_ASSERTS=on, as the overcommit CI job runs) when a pin is created without a `source:`. The extracted attribute/override directive modules built `Pin::Method`, `Pin::Parameter`, and `Pin::Reference::Override` pins without one. Tagged them `:yard_map` since they originate from YARD `@!` directives. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * Fix pre-existing rubocop offenses in untouched files The `.rubocop_todo.yml` CI job runs `rubocop -c .rubocop.yml` across the whole repo and was failing on 8 offenses unrelated to this PR. Fixed them in place rather than suppressing: - Style/ArgumentsForwarding: anonymous block forwarding (`&`) in Solargraph.with_clean_env, UniqueType#each, Host#show_message_request. - Style/ArrayIntersect: `(a & b).any?` -> `a.intersect?(b)` in TypeChecker#parameterized_arity_problems_for. - Lint/UnreachableCode: the body of Pin::Method#combine_same_type_arity_ signatures is intentionally preserved behind a debug stub `return` (upstream 6d8ce95); wrapped it in a scoped rubocop:disable with a comment explaining why, instead of deleting the kept code. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * Fix ArgumentValue struct init on Ruby < 3.2 `ArgumentValue = Struct.new(:value)` was constructed with a keyword argument (`ArgumentValue.new(value: ...)`). On Ruby 3.1 a plain Struct treats that as a positional Hash, so `#value` returned `{ value: x }` instead of `x`. That garbled `ClassMethodSend#argument_values`, which shifted every macro placeholder (`$1`, `$2`, ...) — producing method pins like `value` and dropping real ones. Added `keyword_init: true`. Fixes the 6 macro specs failing on the Ruby 3.1 CI matrix job. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * Drop 'head' from RSpec matrix temporarily ruby/setup-ruby@v1 currently 404s on `head` for ubuntu-24.04 ("Unavailable version head for ruby"). Removed it from the matrix so CI isn't blocked; left a @todo to restore once setup-ruby publishes it. See: https://github.com/castwide/solargraph/actions/runs/25863741955/job/76000137015?pr=1187 Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * Fix strong typechecking --------- Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
* More typecheck fixes * Even more typecheck fixes - is this a dead code? * ApiMap fix * Mapper fix /home/runner/work/solargraph/solargraph/lib/solargraph/yard_map/mapper.rb:28 - Unresolved call to filename on Solargraph::Location, nil * NodeMethods fix /home/runner/work/solargraph/solargraph/lib/solargraph/parser/parser_gem/node_methods.rb:107 - Declared return type ::String, ::Integer, ::Float, ::Symbol, ::Array, ::Hash, ::Solargraph::Source::Chain, nil does not match inferred type ::String, ::Parser::AST::Node, ::Array, ::Hash, ::Solargraph::Source::Chain, nil for Solargraph::Parser::ParserGem::NodeMethods.simple_convert /home/runner/work/solargraph/solargraph/lib/solargraph/parser/parser_gem/node_methods.rb:107 - Declared return type ::String, ::Integer, ::Float, ::Symbol, ::Array, ::Hash, ::Solargraph::Source::Chain, nil does not match inferred type ::String, ::Parser::AST::Node, ::Array, ::Hash, ::Solargraph::Source::Chain, nil for Solargraph::Parser::ParserGem::NodeMethods#simple_convert
* Port macros from lekemula/solargraph@lm-named-macros (4572e07..389def6) Original 11-commit diff: lekemula/solargraph@524c94e...389def6 Squashes the original branch and ports it onto current upstream/master, where the YardMap class was gutted and replaced with DocMap/GemPins (upstream 94006fb). Differences from the original implementation: - Parser layer: original work added `simple_convert` and `process_dsl_method` to `parser/rubyvm/{node_methods,node_processors/send_node}`. Upstream removed the rubyvm parser entirely. Rewrote both for the parser_gem AST shape: lowercase node types (`:send`, `:hash`, `:const`, `:array`), `:send` children indexed as `[receiver, method_name, *args]`, literals split into `:int`/`:float`/`:sym`/`:str` instead of `:LIT`. - ApiMap integration: original `process_macros(pins)` hooked into a `pins` parameter that no longer exists. Adapted to the new `catalog(bench)` flow — consumes `iced_pins + live_pins + doc_map.pins`, filters `Pin::Ephemeral::ClassMethodSend` from iced and live separately before the store update. Kept the original logging. - MethodDirective: original `Parser.process_node(...).first.last` regressed `spec/source_map/mapper_spec.rb:89`. Upstream had since added a `Pin::Method` filter inline; backported that into the extracted directive module. - Spec relocation: `spec/yard_map_spec.rb` was deleted upstream. The `loads macros from gems` test moved to `spec/yard_map/mapper_spec.rb` and uses the new `pins_with(name)` (DocMap-based) helper. Assertion tightened from `macros.count > 0` to checking that the `MyStruct.my_attribute` method pin exists and exposes the macro by name. - All other new files (Macro, Directives::*, Pin::Ephemeral::*, gem-with-yard-macros fixture, api_map_spec/clip_spec additions) landed unchanged from the squashed branch. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * Fix invalid gemspec for gem-with-yard-macros fixture The skeleton gemspec from `bundle gem` left TODO placeholders in summary, description, homepage, and metadata fields, which Bundler rejects in CI. Replaced with real values describing the fixture's purpose and trimmed the file list to `lib/**/*.rb` so it doesn't depend on `git ls-files` working in the CI checkout. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * Fix rubocop offenses - Autocorrected style issues across the new/ported files (string quoting, empty-method one-liners, redundant cop disables, def-without-parens, etc). - Excluded the gem-with-yard-macros fixture from rubocop entirely; it's a `bundle gem` skeleton that exists to be loaded as a gem, not as project source. - Bumped Metrics/ModuleLength.Max in the todo file from 167 to 195 to accommodate the simple_convert helpers added to ParserGem::NodeMethods. - Cleaned up YARD `@param` mismatches in Macro and ClassMethodSend, and rewrote one multi-line block chain in Macro#generate_yardoc_from. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * Trim gem-with-yard-macros fixture to essentials Removed the `bundle gem` skeleton boilerplate (LICENSE, README, CHANGELOG, CODE_OF_CONDUCT, Rakefile, bin/, the gem's own Gemfile/Gemfile.lock, RBS sig, .gitignore). None are needed: the fixture exists only to be resolved as a path gem and have its YARD macro loaded. What remains is the gemspec, the macro definition, and version.rb. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * Set source: :yard_map on directive-generated pins `Pin::Base#assert_source_provided` raises (under SOLARGRAPH_ASSERTS=on, as the overcommit CI job runs) when a pin is created without a `source:`. The extracted attribute/override directive modules built `Pin::Method`, `Pin::Parameter`, and `Pin::Reference::Override` pins without one. Tagged them `:yard_map` since they originate from YARD `@!` directives. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * Fix pre-existing rubocop offenses in untouched files The `.rubocop_todo.yml` CI job runs `rubocop -c .rubocop.yml` across the whole repo and was failing on 8 offenses unrelated to this PR. Fixed them in place rather than suppressing: - Style/ArgumentsForwarding: anonymous block forwarding (`&`) in Solargraph.with_clean_env, UniqueType#each, Host#show_message_request. - Style/ArrayIntersect: `(a & b).any?` -> `a.intersect?(b)` in TypeChecker#parameterized_arity_problems_for. - Lint/UnreachableCode: the body of Pin::Method#combine_same_type_arity_ signatures is intentionally preserved behind a debug stub `return` (upstream 6d8ce95); wrapped it in a scoped rubocop:disable with a comment explaining why, instead of deleting the kept code. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * Fix ArgumentValue struct init on Ruby < 3.2 `ArgumentValue = Struct.new(:value)` was constructed with a keyword argument (`ArgumentValue.new(value: ...)`). On Ruby 3.1 a plain Struct treats that as a positional Hash, so `#value` returned `{ value: x }` instead of `x`. That garbled `ClassMethodSend#argument_values`, which shifted every macro placeholder (`$1`, `$2`, ...) — producing method pins like `value` and dropping real ones. Added `keyword_init: true`. Fixes the 6 macro specs failing on the Ruby 3.1 CI matrix job. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * Drop 'head' from RSpec matrix temporarily ruby/setup-ruby@v1 currently 404s on `head` for ubuntu-24.04 ("Unavailable version head for ruby"). Removed it from the matrix so CI isn't blocked; left a @todo to restore once setup-ruby publishes it. See: https://github.com/castwide/solargraph/actions/runs/25863741955/job/76000137015?pr=1187 Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * Fix strong typechecking * Resolve paths for macro methods * Handle macros dynamically * Reinstate optional cache clearing * Deprecate Ephemeral::ClassMethodSend * Avoid chains for macro resolution when possible * Minor refactor * Pending specs * Clarify macro spec for keyword arguments * Linting * Erroneous reversions * Typechecking * Minor spec tweak for Ruby 3.x * More erroneous reversions --------- Co-authored-by: Lekë Mula <l.mula@finlink.de> Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
* Unused macro methods * Update Ruby to 4.0 in typecheck workflow * Revert unneeded sg-ignore * Update to Ruby 4.0 in plugins workflow
* Infer method calls with parameter names in return types * Retain existing tags * JIT macro expansion * Expand types from chain calls
Contributor
|
Hey @castwide, I tried to test this today on my project and I found something strange. The LSP was stuck in a seemingly endless loop, and it was never finishing cataloging. These are the last 500lines of the logs, which seem to be kept looping over and over again: One thing that caught my eye, in the whole river of the logs, was the This issue does not seem to appear in the I hope this helps somehow. 🤞 Let me know if I can provide you with further details. |
Owner
Author
|
@lekemula Did you encounter this with the
Is there a particular operation you attempted that triggered the endless loop? If you encountered the problem on a different project, can you provide a reproducible example? |
Contributor
|
@castwide, sorry for the ambiguity, but I meant the closed-source project at my current job. So I ran the 
v0.59.2_definition_benchmark.json.gz main_definition_benchmark.json.gz Note: The process on Here are some stats regarding our project: I fear that the exact DSL methods inference might be a little too expensive an operation. Maybe we should go with a more "naive approach" for the sake of performance? SIDE NOTE: regarding |
4 tasks
Owner
Author
|
@lekemula Thanks for clarifying. It looks like my attempt to minimize the amount of macro processing performed during catalog operations is insufficient for very large codebases.
I tend to agree. Would you like to profile your solution in #1202? If its performance is acceptable, we can merge that instead. AFAICT, the worst case scenario is that the language server might surface some incompletely inferred macros as |
Contributor
@castwide Unfortunately, the #1202 showed no difference. However, while removing the lines that seem to be causing the performance bottleneck based on vernier traces, the performance improved, and there were no failing specs (at least locally). I'm wondering what that call was added for? |
castwide
pushed a commit
that referenced
this pull request
Jun 1, 2026
This seem to be causing the performance issues reported #1194 (comment)
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.