Skip to content

Support numpydoc parameter documentation - #5421

Closed
Hirujan-R wants to merge 2 commits into
meta:mainfrom
Hirujan-R:support-numpydoc
Closed

Hirujan-R wants to merge 2 commits into
meta:mainfrom
Hirujan-R:support-numpydoc

Conversation

@Hirujan-R

@Hirujan-R Hirujan-R commented Oct 27, 2026 •

Copy link
Copy Markdown
Contributor

Summary

Adds numpydoc parameter documentation to hover and signature help.

Pyrefly already extracts per-parameter documentation from Sphinx and
Google-style docstrings, but numpydoc Parameters sections were ignored. As a
result, hovering a keyword argument or parameter documented in numpydoc showed
only its type. This adds a numpydoc parser alongside the existing style
parsers.

  • Parse numpydoc Parameters and Other Parameters sections using the dashed
    underline form.
  • Support multiple names per entry (x, y : int), *args/**kwargs,
    defaults/colons in type annotations, and indented multi-line descriptions.
  • Stop parsing at the next numpydoc section, such as Returns.
  • Reuse the existing parse_parameter_documentation path, so both hover and
    signature help gain support without LSP-specific changes.
  • Generalize the parameter commit helper to attach one description to multiple
    names. Existing Sphinx and Google behavior is unchanged.

This is limited to parameter documentation for hover/signature help; it does
not add special rendering for other numpydoc sections such as Returns or
Raises.

Fixes #5313

  • I am a human, and I fully understand the code I am submitting.

Test Plan

  • cargo test -p pyrefly_python docstring — 50 passed.
  • cargo test -p pyrefly parameter_doc — 10 passed, including the new
    numpydoc hover test.
  • python3 test.py --no-test --no-tensor-shapes --no-conformance --no-jsonschema --no-extension — formatting and clippy passed.

Added unit coverage for numpydoc parsing (multiple names, untyped declarations,
varargs/kwargs, Other Parameters, section boundaries, indented docstrings,
defaults, multi-paragraph descriptions, and empty descriptions) plus an LSP
hover test verifying that keyword arguments show numpydoc parameter
documentation.

@meta-cla

meta-cla Bot commented Oct 27, 2026

Copy link
Copy Markdown

Hi @Hirujan-R!

Thank you for your pull request and welcome to our community.

Action Required

In order to merge any pull request (code, docs, etc.), we require contributors to sign our Contributor License Agreement, and we don't seem to have one on file for you.

Process

In order for us to review and merge your suggested changes, please sign at https://code.facebook.com/cla. If you are contributing on behalf of someone else (eg your employer), the individual CLA may not be sufficient and your employer may need to sign the corporate CLA.

Once the CLA is signed, our tooling will perform checks and validations. Afterwards, the pull request will be tagged with CLA signed. The tagging process may take up to 1 hour after signing. Please give it that time before contacting us about it.

If you have received this in error or have any questions, please contact us at cla@meta.com. Thanks!

@meta-cla

meta-cla Bot commented Oct 27, 2026

Copy link
Copy Markdown

Thank you for signing our Contributor License Agreement. We can now accept your code for this (and any) Meta Open Source project. Thanks!

@meta-cla meta-cla Bot added the cla signed label Oct 27, 2026
@meta-cla

meta-cla Bot commented Oct 27, 2026

Copy link
Copy Markdown

Thank you for signing our Contributor License Agreement. We can now accept your code for this (and any) Meta Open Source project. Thanks!

@stroxler stroxler left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks pretty good to me, thanks!

It could use a rebase, because for technical reasons auto-import (which makes merging much easier) doesn't happen on a first PR before the CLA is signed; usually rebasing and pushing will trigger auto-import.

While we're at it, two suggestions for improvement to make this even better:

  • Support declarations without a type. extract_numpydoc_param_names requires a colon, so it ignores x, *args, and **kwargs followed by indented descriptions. The numpydoc guide explicitly permits omitted types—and recommends that form for varargs/kwargs. The new tests only cover typed declarations.

  • Preserve multi-paragraph descriptions. The blank-line branch commits and clears the current parameter. Consequently, a description containing first paragraph, a blank line, then an indented second paragraph silently loses the second paragraph. Keep the parameter active across blank lines and add a regression test.

@github-actions

Copy link
Copy Markdown

According to mypy_primer, this change doesn't affect type check results on a corpus of open source code. ✅

Hover and signature help extract parameter documentation from docstrings,
but only for Sphinx and Google styles. Parse numpydoc Parameters and Other
Parameters sections so these docstrings also provide parameter
documentation.

The parser matches dashed section headers, handles multiple names per
declaration, and stops when another section begins.
Numpydoc permits omitting parameter types, especially for *args and
**kwargs, and descriptions may contain multiple paragraphs. Treat lines
without a type as declarations and keep the current parameter active
across blank lines so later paragraphs are preserved.

Add regression tests for both cases.
@meta-codesync

meta-codesync Bot commented Oct 29, 2026

Copy link
Copy Markdown
Contributor

This pull request has been imported. If you are a Meta employee, you can view this in D128370293. (Because this pull request was imported automatically, there will not be any future comments.)

@Hirujan-R

Copy link
Copy Markdown
Contributor Author

@stroxler Thanks for the review! I addressed the feedback in abd6ab98e and rebased the branch onto main.

  • Untyped declarations are now supported: extract_numpydoc_param_names accepts lines without a :, so x, *args, and **kwargs followed by indented descriptions are parsed. Added test_parse_numpydoc_untyped_params.
  • Multi-paragraph descriptions are preserved: blank lines no longer commit and clear the active parameter. Added test_parse_numpydoc_preserves_multiple_paragraphs.
  • The branch is rebased onto the latest main, which should trigger auto-import now that the CLA is signed.

Tests:

  • cargo test -p pyrefly_python docstring — 50 passed
  • cargo test -p pyrefly parameter_doc — 10 passed
  • Formatting and clippy via python3 test.py --no-test --no-tensor-shapes --no-conformance --no-jsonschema --no-extension — passed

Ready for another review.

@stroxler stroxler left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review automatically exported from Phabricator review in Meta.

@meta-codesync

meta-codesync Bot commented Oct 31, 2026

Copy link
Copy Markdown
Contributor

This pull request has been merged in 4e8e31a.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Support numpydoc

3 participants