Skip to content

Drop session from the fmap BIDS filter so session-level *prep runs can start - #395

Draft
asmacdo wants to merge 1 commit into
PennLINC:mainfrom
asmacdo:fmap-filter-session
Draft

Drop session from the fmap BIDS filter so session-level *prep runs can start#395
asmacdo wants to merge 1 commit into
PennLINC:mainfrom
asmacdo:fmap-filter-session

Conversation

@asmacdo

@asmacdo asmacdo commented Jul 19, 2026

Copy link
Copy Markdown
Collaborator

What & why

Session-level fmriprep aborts during workflow construction with:

ValueError: Filters include session, but session is already defined.

The fmap entry of the generated --bids-filter-file pins a session, and fmriprep independently passes the session to sdcflows' find_estimators(). sdcflows treats receiving the session from both sources as a conflict and raises, so no session-level job can start. Dropping session from that one entry unblocks the run. Every other entry keeps its session.

History — this restores the pre-#337 fmap entry

The fmap entry carried no session from #231 until #337 (@tien-tong's sparse-checkout PR) added 'session': '$sesid' to it. This PR puts it back to {'datatype': 'fmap'}. So the question isn't "is it safe to remove a key" but "was #337's addition of the key intentional, or a blanket session-scoping of every filter entry that didn't notice fmap conflicts with sdcflows?"

Why it should be safe — reasoning, please poke holes

The argument: since #337, each job's input is git sparse-checkout'd down to a single sub-XX/ses-YY (participant_job.sh.jinja2), so the working tree contains only the target session's files. If that holds, a {'datatype': 'fmap'} filter with no session can't match a fieldmap from another session — no other session's files are on disk — so dropping the key shouldn't broaden what any app sees, whatever app reads the filter. (Note the same PR that made the checkout single-session is the one that added the key — so on this reading the key was never needed.)

I have not proven this beyond the one fmriprep run, so I'd rather the meeting kick the tires than take it on faith. The place I'd look first: that per-dataset narrowing is best-effort — participant_job.sh.jinja2 wraps it in || true. If it ever fails, the full dataset tree is present and a session-less fmap filter could match other sessions. The mitigation is that datalad get -n only fetched the target session's content, so other-session files are annex pointers without content — the app would likely error rather than silently use the wrong fieldmap — but I haven't confirmed that path.

Scope — verified vs. expected

The filter file is generated for any *prep app at session level (gate: 'prep' in container_name.lower()), so the same fmap entry reaches fmriprep / qsiprep / aslprep alike.

  • Verified: fmriprep 25.2.5, session level, OpenNeuro ds004044 — crashes without this change, runs with it.
  • Expected but not run e2e: qsiprep / aslprep share the sdcflows find_estimators() path, so they should hit the same conflict and the same fix — but I have not run them end-to-end.

Open questions for review

Changes

  • bidsapp_run.sh.jinja2 / bidsapp_pipeline_run.sh.jinja2 — fmap entry is now {'datatype': 'fmap'}.
  • Deleted filter_file.sh.jinja2 — unreferenced.
  • Added test_session_filter_file_fmap_carries_no_session (fmriprep / qsiprep / aslprep). It executes the rendered filter-file block under bash and asserts on the parsed JSON, so it covers what the app receives, not template text. Fails without the fix on all three (asserted diff is exactly the extra {'session': '1'}).
Call path
fmriprep/workflows/base.py:637   map_fieldmap_estimation(...)
fmriprep/workflows/base.py:969   find_estimators(...)
sdcflows/utils/wrangler.py:336   raise ValueError('Filters include session, but session is already defined.')

Three conditions must coincide to hit this, which is why it went unnoticed: processing_level == 'session', an app with prep in its name (the flag_filterfile gate), and not --anat-only (which skips fieldmap estimation entirely). Subject-level runs never render the block.

🤖 Generated with Claude Code

Session-level runs of any `*prep` app abort during workflow construction:

    sdcflows/utils/wrangler.py, in find_estimators
    ValueError: Filters include session, but session is already defined.

fmriprep passes the fmap entry of `--bids-filter-file` to sdcflows'
`find_estimators()` alongside the session it resolved itself, and sdcflows
refuses to receive the session from both sources. The session is already
enforced by the sparse checkout, so the key can simply go.

Also removes `filter_file.sh.jinja2`, which nothing references.

The new test executes the generated filter-file block and asserts on the
parsed JSON, so it covers what the BIDS app actually receives rather than
the rendered text.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@codecov-commenter

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 79.19%. Comparing base (2cc536a) to head (c09ac62).

Additional details and impacted files
@@           Coverage Diff           @@
##             main     #395   +/-   ##
=======================================
  Coverage   79.19%   79.19%           
=======================================
  Files          17       17           
  Lines        2168     2168           
  Branches      385      385           
=======================================
  Hits         1717     1717           
  Misses        308      308           
  Partials      143      143           

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants