Skip to content

fix(cli): report unknown dataset names without a traceback - #120

Closed
HarshRajSinghania wants to merge 4 commits into
wandb:mainfrom
HarshRajSinghania:fix-cli-unknown-dataset-names
Closed

HarshRajSinghania wants to merge 4 commits into
wandb:mainfrom
HarshRajSinghania:fix-cli-unknown-dataset-names

Conversation

@HarshRajSinghania

Copy link
Copy Markdown
Contributor

Summary

Unknown dataset names in rai datasets show and rai datasets infer-policy-expectations now produce a clean error on stderr and exit with status 2 instead of an uncaught KeyError traceback. The registry's existing suggestion logic is preserved.

Motivation

Fixes #118. An unknown dataset name currently produces a Python traceback. The registry already provides a useful error message and suggestions; the CLI should present that message as an input error.

Implementation

  • Catch KeyError from ExampleRegistry.get / ExampleRegistry.load only when the message indicates an unregistered dataset.
  • Print the message to stderr and return 2.
  • Other KeyErrors (e.g. from a registered dataset's loader) are re-raised.

Testing

  • python -m pytest -q tests/test_cli.py — 12 passed (Python 3.11.2, uv venv).

Offline tests cover:

  • Both commands with an unknown name (status 2, stderr message, empty stdout, no traceback).
  • The bbqq suggestion and an unknown name with no suggestion.
  • Successful rai datasets show bbq.
  • Successful rai datasets infer-policy-expectations bbq --limit 0 (status 0, empty JSON array).
  • An unrelated loader KeyError for a registered dataset is not converted into an unknown-name error.

No new dependencies. No README changes.

Harsh Raj Singhania added 4 commits October 21, 2026 21:24
Catch KeyError from ExampleRegistry.get/load in datasets show and
infer-policy-expectations, print the registry message to stderr, and
exit with status 2. Preserve other KeyErrors from loaders.

Adds offline tests covering unknown names (with and without suggestions),
successful paths, and non-unknown loader KeyErrors.
Catch KeyError from ExampleRegistry.get/load in datasets show and
infer-policy-expectations, print the registry message to stderr, and
exit with status 2. Preserve other KeyErrors from loaders.

Adds offline tests covering unknown names (with and without suggestions),
successful paths, and non-unknown loader KeyErrors.

knisar commented Oct 23, 2026

Copy link
Copy Markdown
Collaborator

@HarshRajSinghania, the pushed head 191af1f replaces all 383 lines of rai_toolkit/cli.py with the single placeholder FULL_CONTENT_HERE. This breaks the CLI import and explains the failing checks; the implementation described in the PR is not present in this revision.

#119 was opened earlier for the same issue and contains a focused implementation, so I am using it as the primary review path for #118. Please hold further work on this duplicate while that review completes. If #119 passes review, I will close this PR as superseded. In the meantime, please focus on the outstanding implementation in #103 and the readiness check in #112.

knisar commented Oct 23, 2026

Copy link
Copy Markdown
Collaborator

#119 has now passed independent review and all CI checks and is merged, closing #118. Closing this overlapping PR as superseded. Thanks for contributing tests. Please check for linked open PRs before starting another issue, and continue the outstanding work on #103 and the readiness check on #112.

@knisar knisar closed this Oct 23, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

fix(cli): report unknown dataset names without a traceback

2 participants