Skip to content

[ENG-3771] projects-cli-0: explicit projects CLI#730

Closed
mrmoxon wants to merge 1 commit into
mainfrom
feature/projects-cli-explicit
Closed

[ENG-3771] projects-cli-0: explicit projects CLI#730
mrmoxon wants to merge 1 commit into
mainfrom
feature/projects-cli-explicit

Conversation

@mrmoxon

@mrmoxon mrmoxon commented Jun 8, 2026

Copy link
Copy Markdown

Summary

  • add the prime project command group plus Projects API/context helpers
  • support explicit project attachment for training runs, eval pushes, hosted evals, and artifact assignment/removal
  • keep train/eval behavior explicit-only: no active-project defaulting in this tranche

Validation

  • uv run pytest packages/prime/tests/test_rl_config.py packages/prime/tests/test_projects_cli.py packages/prime/tests/test_eval_push.py packages/prime/tests/test_hosted_eval.py packages/prime/tests/test_rl_api.py packages/prime/tests/test_deployments.py packages/prime-evals/tests/test_evals.py -q
  • uv run ruff check packages/prime/src/prime_cli/commands/rl.py packages/prime/src/prime_cli/api/projects.py packages/prime/src/prime_cli/commands/projects.py packages/prime/src/prime_cli/utils/projects.py packages/prime/src/prime_cli/commands/evals.py packages/prime/src/prime_cli/utils/eval_push.py packages/prime/src/prime_cli/verifiers_bridge.py packages/prime/tests/test_projects_cli.py packages/prime/tests/test_eval_push.py packages/prime/tests/test_hosted_eval.py packages/prime-evals/src/prime_evals/evals.py packages/prime-evals/tests/test_evals.py

Stacked with #677, which contains the default-project UX layer.


Note

Medium Risk
Touches evaluation and training run creation plus multi-artifact project membership APIs; behavior is mostly additive but mis-scoped team/project context could attach artifacts to the wrong project.

Overview
Adds Lab projects end-to-end: a new prime project command group (create/list/show/use/current/update/clear, plus assign/remove for runs, evals, and adapters), a Projects API client, and workspace active-project state in .prime/lab/context.json with optional PRIME_PROJECT_ID and team-scope checks.

Training and evaluations can attach to a project only when you pass --project or --no-project on prime train, prime eval push, and prime eval run (including hosted); prime project assign/remove wire through new RFT and adapter project PATCH APIs. The prime-evals SDK gains project_id on create, update_evaluation assign/clear, and owner-aware environment lookup via POST /environmentshub/lookup. README and package docs describe the new workflows.

Reviewed by Cursor Bugbot for commit 65d3769. Bugbot is set up for automated code reviews on this repo. Configure here.

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes and found 4 potential issues.

Fix All in Cursor

❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.

Reviewed by Cursor Bugbot for commit 62608b7. Configure here.

Comment thread packages/prime/src/prime_cli/commands/rl.py
Comment thread packages/prime-evals/src/prime_evals/evals.py
Comment thread packages/prime/src/prime_cli/commands/evals.py Outdated
Comment thread packages/prime/src/prime_cli/commands/evals.py

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 62608b7d44

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

num_examples: int | None = None
rollouts_per_example: int | None = None
eval_base_model: bool | None = None
env: List[EvalEnvConfig] = Field(default_factory=list)

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Restore eval sampling parsing

Please keep the [eval.sampling] field on EvalConfig; because the model still uses extra="forbid", any existing RL config that contains [eval.sampling] now fails validation instead of launching and forwarding those sampling overrides to the API. This breaks configs covered by the existing test_load_config_accepts_eval_sampling_* cases and prevents users from running evaluations with reasoning_effort, enable_thinking, temperature, or max_tokens overrides.

Useful? React with 👍 / 👎.

Comment on lines +197 to +198
if project_id is None and use_active_project:
project_id = resolve_project_id(None, client=api_client)

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Honor the active-project opt-in

When push_eval_results_to_hub(..., use_active_project=True) is used with no explicit project_id, this still calls resolve_project_id with its default use_active_project=False, so the active workspace project is silently ignored and the uploaded evaluation is created without a project. Pass the flag through here so callers that opt into active-project resolution actually get the active project attached.

Useful? React with 👍 / 👎.

@mrmoxon mrmoxon force-pushed the feature/projects-cli-explicit branch from 62608b7 to 6d0e97c Compare June 8, 2026 12:05
@mrmoxon mrmoxon changed the title ENG-3771: add explicit projects CLI projects-cli-0: explicit projects CLI Jun 8, 2026
@mrmoxon mrmoxon force-pushed the feature/projects-cli-explicit branch from 6d0e97c to 65d3769 Compare June 8, 2026 12:16
@mrmoxon mrmoxon closed this Jun 8, 2026
@mrmoxon mrmoxon deleted the feature/projects-cli-explicit branch June 8, 2026 12:30
@mrmoxon mrmoxon changed the title projects-cli-0: explicit projects CLI [ENG-3771] projects-cli-0: explicit projects CLI Jun 8, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant