`tapioca gem`: duplicate file comments (license headers repeated 145x for selenium-webdriver)

[dduugg]

Description

When tapioca gem builds documentation comments for a constant, it concatenates the comments from every definition of that constant. For gems where each source file starts with a license/copyright preamble, that preamble ends up attached once per file that reopens the constant, producing thousands of lines of byte-identical comments in the RBI.

selenium-webdriver v4.47.0 is a clear repro: the Apache/SFC license header is emitted 145 times in a row as the doc comment for module Selenium, accounting for ~2,320 of the file's 24,508 lines (~10% of the RBI).

Steps to reproduce

# Gemfile
source "https://rubygems.org"

gem "tapioca"
gem "selenium-webdriver", "4.47.0"

$ bundle exec tapioca gem selenium-webdriver
$ wc -l sorbet/rbi/gems/selenium-webdriver@4.47.0.rbi
24508
$ grep -c "Licensed to the Software Freedom Conservancy" sorbet/rbi/gems/selenium-webdriver@4.47.0.rbi
145
$ awk 'NR < 2368 && /Licensed to the Software Freedom/ { c++ } END { print c }' \
    sorbet/rbi/gems/selenium-webdriver@4.47.0.rbi
145   # all 145 copies precede the single `module Selenium; end` on line 2368

Reproduced with tapioca 0.19.2 (rubygems) and with main (56c8d6c), Ruby 4.0.2.

Actual output

Lines 8–2367 of sorbet/rbi/gems/selenium-webdriver@4.47.0.rbi are this 16-line block, repeated 145 times, followed by the one-line constant it documents:

# Licensed to the Software Freedom Conservancy (SFC) under one
# or more contributor license agreements.  See the NOTICE file
# distributed with this work for additional information
# regarding copyright ownership.  The SFC licenses this file
# to you under the Apache License, Version 2.0 (the
# "License"); you may not use this file except in compliance
# with the License.  You may obtain a copy of the License at
#
#   http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing,
# software distributed under the License is distributed on an
# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
# KIND, either express or implied.  See the License for the
# specific language governing permissions and limitations
# under the License.
# Licensed to the Software Freedom Conservancy (SFC) under one
# ... (x145)
#
# pkg:gem/selenium-webdriver#lib/selenium/webdriver/atoms.rb:20
module Selenium; end

Expected output

The identical comment block should appear at most once:

# Licensed to the Software Freedom Conservancy (SFC) under one
# ... (license header, once)
# under the License.
#
# pkg:gem/selenium-webdriver#lib/selenium/webdriver/atoms.rb:20
module Selenium; end

Why the existing uniq! doesn't catch it

Documentation#documentation_comments already tries to de-dupe:

https://github.com/Shopify/tapioca/blob/main/lib/tapioca/gem/listeners/documentation.rb#L72-L74

comments = declaration.definitions.flat_map(&:comments)
comments.uniq!

But Rubydex::Comment has no value equality, so uniq! falls back to identity and never collapses anything:

$ bundle exec ruby -e 'require "rubydex"
> p Rubydex::Comment.instance_methods(false).sort  # => [:location, :string]
> p Rubydex::Comment.instance_method(:==).owner    # => BasicObject
> p Rubydex::Comment.instance_method(:hash).owner  # => Kernel'

Each of the 145 definitions of module Selenium contributes its own comment objects, so all 145 copies survive.

Suggested fix

De-dupe on comment content rather than object identity — and ideally per definition block rather than per line, so that a repeated line inside one block (e.g. the bare # separators in the license header) isn't collapsed and distinct docs on different definitions are preserved. Roughly:

comments = declaration.definitions
  .map(&:comments)
  .uniq { |block| block.map(&:string) }
  .flatten

Locally, that change takes selenium-webdriver@4.47.0.rbi from 24,508 to 22,172 lines (145 copies of the header → 1) with no changes to any non-comment line of the RBI. Happy to open a PR if this shape looks right.

(For context, this was found in a large private monorepo where the duplicated headers add several thousand lines of churn to checked-in gem RBIs; selenium-webdriver is just the most extreme public example we found.)