Skip to content

Evaluate CWL jobs that should be skipped on the leader#5507

Open
lonbar wants to merge 1 commit into
DataBiosphere:masterfrom
lonbar:master
Open

Evaluate CWL jobs that should be skipped on the leader#5507
lonbar wants to merge 1 commit into
DataBiosphere:masterfrom
lonbar:master

Conversation

@lonbar
Copy link
Copy Markdown

@lonbar lonbar commented Apr 30, 2026

CWL jobs that have a when condition that evaluates to False should not be executed. Currently anything that is not a Workflow or ExpressionTool will always run on a worker node, which means that the check on the Conditional is only done when the worker node is already allocated. If the job is instead run on the leader (local), the step won't be executed.

Checking at the instantiation level makes it possible to determine dynamically if the step should be run on the leader or the worker. This prevents unnecessary overhead in scheduling systems.

Resolves #3990.

Changelog Entry

To be copied to the draft changelog by merger:

  • Toil checks the CWL when conditional on the leader.

Reviewer Checklist

  • Make sure it is coming from issues/XXXX-fix-the-thing in the Toil repo, or from an external repo.
    • If it is coming from an external repo, make sure to pull it in for CI with:
      contrib/admin/test-pr otheruser theirbranchname issues/XXXX-fix-the-thing
      
    • If there is no associated issue, create one.
  • Read through the code changes. Make sure that it doesn't have:
    • Addition of trailing whitespace.
    • New variable or member names in camelCase that want to be in snake_case.
    • New functions without type hints.
    • New functions or classes without informative docstrings.
    • Changes to semantics not reflected in the relevant docstrings.
    • New or changed command line options for Toil workflows that are not reflected in docs/running/{cliOptions,cwl,wdl}.rst
    • New features without tests.
  • Comment on the lines of code where problems exist with a review comment. You can shift-click the line numbers in the diff to select multiple lines.
  • Finish the review with an overall description of your opinion.

Merger Checklist

  • Make sure the PR passed tests, including the Gitlab tests, for the most recent commit in its branch.
  • Make sure the PR has been reviewed. If not, review it. If it has been reviewed and any requested changes seem to have been addressed, proceed.
  • Merge with the Github "Squash and merge" feature.
    • If there are multiple authors' commits, add Co-authored-by to give credit to all contributing authors.
  • Copy its recommended changelog entry to the Draft Changelog.
  • Append the issue number in parentheses to the changelog entry.

@adamnovak adamnovak requested a review from annagiroti May 7, 2026 15:36
CWL jobs that have a when condition that evaluates to False should
not be executed. Currently anything that is not a Workflow or
ExpressionTool will always run on a worker node, which means that
the check on the Conditional is only done when the worker node is
already allocated. If the job is instead run on the leader (local),
the step won't be executed.

Checking at the instantiation level makes it possible to determine
dynamically if the step should be run on the leader or the worker.
This prevents unnecessary overhead in scheduling systems.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

CWL Conditional steps that aren't running still get scheduled to a job scheduler.

1 participant