Skip to content

fix(validator): 0.0 veto labels are silently overridden by co-applied reward labels - #1662

Open
RealDiligent wants to merge 1 commit into
entrius:testfrom
RealDiligent:fix/label-veto-multiplier-override
Open

fix(validator): 0.0 veto labels are silently overridden by co-applied reward labels#1662
RealDiligent wants to merge 1 commit into
entrius:testfrom
RealDiligent:fix/label-veto-multiplier-override

Conversation

@RealDiligent

Copy link
Copy Markdown

Summary

Repositories use a 0.0 entry in label_multipliers as a hard veto — a rejection / void marker that should zero a PR's score. resolve_highest_label_multiplier took max() across a PR's matching labels, so a veto label applied alongside a reward label was silently ignored. A maintainer could never revoke a PR that had already earned a reward label.

This is live in the committed master_repositories.json on seven repos (eval:REJECT, dataset:REJECT, invalid-pr, duplicate-pr, slop, kata:invalid, kata:defeat:*, eval:none, type:*, …).

Root cause

gittensor/validator/oss_contributions/label_resolution.py:

return max(matches) if matches else None
...
label, multiplier = max(candidates, key=lambda c: (c[1], c[0]))

max() treats a 0.0 veto as just "the lowest value", so any positive label out-ranks it. A 0.0 label is categorically different from a low reward — it means disqualified, not low-value.

Fix

A 0.0-configured label now vetoes: if any matched label resolves to 0.0, the label multiplier is 0.0 regardless of a co-applied higher label. Among purely positive labels the highest still wins, so normal scoring is unchanged. The veto is also detected within a single label that matches both a 0.0 pattern and a positive pattern (e.g. an eval:* catch-all plus an explicit eval:REJECT). A low positive multiplier (e.g. mult:contribution: 0.05) is a reward, not a veto, and is unaffected.

Both the OSS PR path (resolve_trusted_label_multiplier → this helper) and the issue-discovery solving-PR path (_mirror_issue_for_scoring → same helper) resolve labels here, so both are fixed by the one change.

Evidence

Resolved multiplier for a reward + veto label pair, against the committed master_repositories.json (each row is a maintainer applying a documented rejection marker to a PR that already carries a reward label):

Repo Labels applied Before After
gittensor-agent-forge/gt-imagent round-winner + invalid-pr 10.00 0.00
gittensor-agent-forge/gt-imagent round-winner + duplicate-pr 10.00 0.00
gittensor-ai-lab/sparkinfer eval:L + eval:REJECT 2.50 0.00
gittensor-model-hub/SparkDistill dataset:xl + dataset:REJECT 4.00 0.00
JSONbored/loopover gittensor:priority + slop 1.50 0.00
Autovara/kata kata:winner:s1 + kata:invalid 0.70 0.00

Related Issues

None open.

Type of Change

  • Bug fix
  • New feature
  • Refactor
  • Documentation
  • Other (describe below)

Testing

  • Tests added/updated
  • Manually tested

New tests/validator/test_label_veto.py (9 cases): veto-beats-reward for the real registry shapes, order-independence, the single-label-matches-both case, and three guards that non-veto behaviour is unchanged (highest positive wins, a low positive is not a veto, default multiplier still applies). The six veto assertions were verified to fail against the current helper and pass with the fix.

Full CI on this branch: 969 tests passed; ruff lint, ruff format, pyright, vulture all pass with no auto-format changes.

No CLI output changes, so the CONTRIBUTING before/after-screenshot requirement does not apply.

Risk / tradeoffs

Behaviour change, and that is the intent: PRs that carry both a reward and a 0.0 veto label now score 0.0 instead of the reward. This only affects PRs where a veto label is actually present — purely positive label sets are unchanged (verified by the "highest positive still wins" test). A repo that genuinely wanted an overridable low weight would configure a small positive value, not 0.0; that path is preserved.

Checklist

  • Code follows project style guidelines
  • Self-review completed
  • Changes are documented (if applicable)

🤖 Generated with Claude Code

@xiao-xiao-mao xiao-xiao-mao Bot added the bug Something isn't working label Jul 28, 2026
…el override them

Repositories use a `0.0` entry in `label_multipliers` as a hard veto — a
rejection / void marker. The live registry has these on seven repos:
`eval:REJECT`, `dataset:REJECT`, `invalid-pr`, `duplicate-pr`, `slop`,
`kata:invalid`, `kata:defeat:*`, `eval:none`, ...

`resolve_highest_label_multiplier` took `max()` across a PR's matching labels,
so a veto label applied alongside a reward label was silently ignored. A
maintainer could never revoke a PR that already carried a reward label:

  gt-imagent   round-winner + invalid-pr     -> 10.00   (should be 0)
  sparkinfer   eval:L + eval:REJECT          ->  2.50   (should be 0)
  sparkdistill dataset:xl + dataset:REJECT   ->  4.00   (should be 0)
  loopover     gittensor:priority + slop     ->  1.50   (should be 0)
  kata         kata:winner:s1 + kata:invalid ->  0.70   (should be 0)

A `0.0`-configured label now vetoes: if any matched label resolves to `0.0`
the label multiplier is `0.0`, regardless of a co-applied higher label. Among
purely positive labels the highest still wins, so normal scoring is unchanged.
The veto is also detected within a single label that matches both a `0.0`
pattern and a positive pattern (e.g. an `eval:*` catch-all plus `eval:REJECT`).

Both the OSS PR path and the issue-discovery solving-PR path resolve labels
through this helper, so both are fixed. A low positive multiplier (e.g. 0.05)
is a reward, not a veto, and is unaffected.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant