Skip to content

fix(cli): record a submission when no autograder is configured - #535

Merged
rongxin-liu merged 1 commit into
mainfrom
fix/record-no-autograder-submissions
Aug 7, 2026
Merged

fix(cli): record a submission when no autograder is configured#535
rongxin-liu merged 1 commit into
mainfrom
fix/record-no-autograder-submissions

Conversation

@rongxin-liu

Copy link
Copy Markdown
Contributor

Summary

PR #458 made the autograde workflow's grade and set-latest jobs skip entirely when an assignment has no autograder configured (no declarative tests, no per-assignment bundle, no classroom-default autograder.py). That correctly stopped spending Actions minutes, but it also stopped the side effect that records a submission: the submit/* GitHub Release. The submit/* tag was still minted in the setup job, but the teacher submissions dashboard reads Releases (via latestSubmitReleaseAndCount), not tags — so a student who pushed to a no-autograder assignment silently vanished from the teacher's view and it read like a bug.

This restores the "records" half while keeping #458's cost win. The grade job now runs for the no-autograder case so runner.py's existing vacuous-pass path (Finalizer.no_autograder()) synthesizes a result.json scoring 0/0 with status=success, which the existing "Publish submission release" step publishes as the submit/* release. Language-toolchain setup (Python/Node/Java/Go/Rust + apt) is gated off no-autograder, so the recording path provisions no toolchain it won't use — it needs only system python3.

This is forward-compatible with the "milestone tags trigger; submit/* records" invariant reaffirmed in the draft PR #531: submit/* stays the canonical record channel.

Changes

  • grade.if now gates only on the acceptance commit; the no-autograder clause is dropped so the job runs and records the submission.
  • The five toolchain setup steps and the apt step now also carry && needs.setup.outputs.no-autograder != 'true', preserving the feat(cli): skip autograde grade job when no autograder is configured #458 minute savings.
  • Removed the setup-job "Autograde-skipped commit status" step — the grade job's always() "Post commit status" step already posts classroom50/autograde=success with the runner's "no autograder configured" summary, so the status-polling channel is at parity with no double-post.
  • set-latest drops its no-autograder guard so the recorded submission advances the latest pointer.
  • Updated the init_skeleton_test.go parity assertions (the pinned grade.if string and the six toolchain if: strings) and a stale comment in test_inline_validator.py. No runner.py logic change — its no-autograder synthesis was already correct and covered.

Verification

  • go test ./... in cli/gh-teacher passes, including TestSkeletonFiles_AutogradeRunner; go vet and gofmt clean.
  • python3 -m pytest cli/gh-teacher/autograders_tests — 358 pass, including test_no_bundle_and_no_default_synthesizes_vacuous_pass, which pins the end-to-end 0/0-success recording (result.json, release body, status=success).
  • Workflow YAML parses; jobs remain setup/grade/set-latest (no job added).
  • No web changes needed: the dashboard surfaces the release automatically once it exists again.

PR #458 made the grade + set-latest jobs skip entirely when no autograder
is configured, which also stopped publishing the submit/* release that
records a submission. The submit/* tag was still minted, but the teacher
dashboard reads releases, so submissions silently vanished and looked like
a bug.

Run the grade job for the no-autograder case so runner.py's existing
vacuous-pass (0/0 success) result is published as the submit/* release and
the submission stays visible. Keep #458's cost win by gating the
language-toolchain setup off no-autograder (the recording path needs only
system python3). Advance set-latest for the recorded submission too.
@rongxin-liu rongxin-liu self-assigned this Aug 7, 2026
@rongxin-liu
rongxin-liu merged commit f3dd96c into main Aug 7, 2026
14 checks passed
@rongxin-liu
rongxin-liu deleted the fix/record-no-autograder-submissions branch August 7, 2026 16:01
@github-actions github-actions Bot mentioned this pull request Aug 7, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant