Issues

• Built-in HF Space templates generate invalid colorTo: cyan metadata

  needs-triage
  MakazhanAlpamys/Soup#2639 · Filter by author harshitthek opened on Nov 8, 2026

• task: classifier reports no accuracy or F1 at evaluation: eval shows only eval_loss

  needs-triage
  MakazhanAlpamys/Soup#2638 · Filter by author AlibekBerik opened on Nov 8, 2026

• loop gate v3: the held-out guard is lexical and can hardly fail at loop sizes

  difficulty:7enhancement
  MakazhanAlpamys/Soup#2619 · Filter by author MakazhanAlpamys opened on Nov 8, 2026

• task: reranker / cross_encoder report no ranking metrics: eval shows only eval_loss, so there is no way to tell whether a fine-tune ranks the right document first

  claimeddifficulty:5enhancement
  MakazhanAlpamys/Soup#2618 · Filter by author AlibekBerik opened on Nov 8, 2026
  3

• SOUP_LAYER_STREAM_CACHE_DIR and SOUP_SPECTRUM_CACHE_DIR outside $HOME, the working directory and the temp directory are dropped silently: the cache lands in ~/.soup on the system disk

  area:trainingbugdifficulty:4
  MakazhanAlpamys/Soup#2610 · Filter by author MakazhanAlpamys opened on Nov 7, 2026

• Resident 4-bit SFT: 837 of 916 ms of elementwise GPU time runs on fp32 inputs, and three fp32 logits tensors set the memory peak (Qwen2.5-7B NF4, RTX 4090 profile)

  area:trainingdifficulty:7enhancement
  MakazhanAlpamys/Soup#2609 · Filter by author MakazhanAlpamys opened on Nov 7, 2026

• GRPO gradient watchdog reads gradients after the clip, so one NaN marks every parameter

  area:trainingbugclaimeddifficulty:3
  MakazhanAlpamys/Soup#2601 · Filter by author MakazhanAlpamys opened on Nov 7, 2026
  5

• [QA] Add an opt-in one-step CPU fixture for Hugging Face Hub round-trip testing

  claimeddifficulty:4enhancementqa
  MakazhanAlpamys/Soup#2599 · Filter by author harshitthek opened on Nov 7, 2026
  3

• Layer streaming under torchrun/accelerate (WORLD_SIZE>1) is let through but not supported

  area:trainingbugdifficulty:4
  MakazhanAlpamys/Soup#2585 · Filter by author MakazhanAlpamys opened on Nov 6, 2026

• No way to turn off intermediate checkpoints: every run leaves a checkpoint folder

  area:trainingdifficulty:5enhancement
  MakazhanAlpamys/Soup#2575 · Filter by author MakazhanAlpamys opened on Nov 6, 2026

• ship: gated (loop-v2) verdicts need replayable evidence before Soup Zero's engine bump

  difficulty:4enhancement
  MakazhanAlpamys/Soup#2571 · Filter by author MakazhanAlpamys opened on Nov 6, 2026
  2

• advise history: a verdict can be lost on Windows when the history lock is not taken in time

  area:clibugdifficulty:3
  MakazhanAlpamys/Soup#2569 · Filter by author MakazhanAlpamys opened on Nov 6, 2026
  4

• test_the_self_test_passes of the Fast-LoRA Q/K/V harness takes down a pytest-xdist worker on Windows CI

  area:trainingbugdifficulty:5
  MakazhanAlpamys/Soup#2562 · Filter by author MakazhanAlpamys opened on Nov 6, 2026
  3

• soup serve (transformers) silently ignores the OpenAI stop parameter and /v1/messages stop_sequences

  area:clibugdifficulty:4
  MakazhanAlpamys/Soup#2549 · Filter by author MakazhanAlpamys opened on Nov 6, 2026

• soup diff: --prompts silently drops missing files, crashes on directories, and fails on UTF-8 BOM

  area:clibugdifficulty:3
  MakazhanAlpamys/Soup#2547 · Filter by author Shubham-dev-stack opened on Nov 6, 2026
  2

• soup draft distill --plan-only: the YAML preview wraps at the console width, so a long path is cut inside a quoted value

  area:clibugclaimeddifficulty:2good first issue
  MakazhanAlpamys/Soup#2536 · Filter by author MakazhanAlpamys opened on Nov 5, 2026
  12

• eval gate: a judge task validates its JSONL per row, so a bad row at line 500 is found after 499 generations

  area:evaldifficulty:3enhancement
  MakazhanAlpamys/Soup#2535 · Filter by author MakazhanAlpamys opened on Nov 5, 2026
  6

• soup ci init --python accepts 03.11 and 3.011 and writes them into the workflow unchanged

  area:clibugclaimeddifficulty:2
  MakazhanAlpamys/Soup#2534 · Filter by author MakazhanAlpamys opened on Nov 5, 2026
  6

• Paths and names printed through for_terminal inside Rich markup lose a backslash before a "[" that is not a complete tag

  area:clibugdifficulty:4
  MakazhanAlpamys/Soup#2533 · Filter by author MakazhanAlpamys opened on Nov 5, 2026
  6

• soup ship --adapter holds two copies of the base model; score the base side with the adapter disabled

  area:evaldifficulty:5enhancement
  MakazhanAlpamys/Soup#2524 · Filter by author MakazhanAlpamys opened on Nov 5, 2026
  2

• training.eval_gate.baseline: 'previous' is documented but not implemented, and the gate then runs with no baseline

  area:evalbugclaimeddifficulty:3
  MakazhanAlpamys/Soup#2523 · Filter by author MakazhanAlpamys opened on Nov 5, 2026
  4

• ship: an opt-in reference-aware judge (--judge-reference) that shows the task's expected answer to judge_score / pairwise

  claimeddifficulty:5
  MakazhanAlpamys/Soup#2520 · Filter by author MakazhanAlpamys opened on Nov 5, 2026
  5

• serve: a streamed request queued behind the first runtime adapter load runs with the new adapter active

  claimeddifficulty:3
  MakazhanAlpamys/Soup#2517 · Filter by author MakazhanAlpamys opened on Nov 5, 2026
  6

• soup serve: a "stream": true request to /v1/messages sends nothing until the whole reply is generated

  area:clibugdifficulty:3
  MakazhanAlpamys/Soup#2515 · Filter by author MakazhanAlpamys opened on Nov 5, 2026

• Layer streaming: a forward-only inference mode (no LoRA, no training config) for prefill and scoring of models larger than VRAM

  area:trainingdifficulty:7enhancement
  MakazhanAlpamys/Soup#2509 · Filter by author MakazhanAlpamys opened on Nov 5, 2026
  1