[ENG-3771] projects-cli-0: explicit projects CLI#730
Conversation
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes and found 4 potential issues.
❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.
Reviewed by Cursor Bugbot for commit 62608b7. Configure here.
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 62608b7d44
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| num_examples: int | None = None | ||
| rollouts_per_example: int | None = None | ||
| eval_base_model: bool | None = None | ||
| env: List[EvalEnvConfig] = Field(default_factory=list) |
There was a problem hiding this comment.
Please keep the [eval.sampling] field on EvalConfig; because the model still uses extra="forbid", any existing RL config that contains [eval.sampling] now fails validation instead of launching and forwarding those sampling overrides to the API. This breaks configs covered by the existing test_load_config_accepts_eval_sampling_* cases and prevents users from running evaluations with reasoning_effort, enable_thinking, temperature, or max_tokens overrides.
Useful? React with 👍 / 👎.
| if project_id is None and use_active_project: | ||
| project_id = resolve_project_id(None, client=api_client) |
There was a problem hiding this comment.
Honor the active-project opt-in
When push_eval_results_to_hub(..., use_active_project=True) is used with no explicit project_id, this still calls resolve_project_id with its default use_active_project=False, so the active workspace project is silently ignored and the uploaded evaluation is created without a project. Pass the flag through here so callers that opt into active-project resolution actually get the active project attached.
Useful? React with 👍 / 👎.
62608b7 to
6d0e97c
Compare
6d0e97c to
65d3769
Compare

Summary
prime projectcommand group plus Projects API/context helpersValidation
uv run pytest packages/prime/tests/test_rl_config.py packages/prime/tests/test_projects_cli.py packages/prime/tests/test_eval_push.py packages/prime/tests/test_hosted_eval.py packages/prime/tests/test_rl_api.py packages/prime/tests/test_deployments.py packages/prime-evals/tests/test_evals.py -quv run ruff check packages/prime/src/prime_cli/commands/rl.py packages/prime/src/prime_cli/api/projects.py packages/prime/src/prime_cli/commands/projects.py packages/prime/src/prime_cli/utils/projects.py packages/prime/src/prime_cli/commands/evals.py packages/prime/src/prime_cli/utils/eval_push.py packages/prime/src/prime_cli/verifiers_bridge.py packages/prime/tests/test_projects_cli.py packages/prime/tests/test_eval_push.py packages/prime/tests/test_hosted_eval.py packages/prime-evals/src/prime_evals/evals.py packages/prime-evals/tests/test_evals.pyStacked with #677, which contains the default-project UX layer.
Note
Medium Risk
Touches evaluation and training run creation plus multi-artifact project membership APIs; behavior is mostly additive but mis-scoped team/project context could attach artifacts to the wrong project.
Overview
Adds Lab projects end-to-end: a new
prime projectcommand group (create/list/show/use/current/update/clear, plus assign/remove for runs, evals, and adapters), a Projects API client, and workspace active-project state in.prime/lab/context.jsonwith optionalPRIME_PROJECT_IDand team-scope checks.Training and evaluations can attach to a project only when you pass
--projector--no-projectonprime train,prime eval push, andprime eval run(including hosted);prime project assign/removewire through new RFT and adapter project PATCH APIs. The prime-evals SDK gainsproject_idon create,update_evaluationassign/clear, and owner-aware environment lookup viaPOST /environmentshub/lookup. README and package docs describe the new workflows.Reviewed by Cursor Bugbot for commit 65d3769. Bugbot is set up for automated code reviews on this repo. Configure here.