Summary
Currently, tasks can be run without using a queue, but pipelines cannot.
It would be useful to have the ability to run critical pipeline tasks directly via a command, without requiring a queue.
Use case
In some cases, especially for critical operations, it’s convenient to run them from the command line and wait until all jobs complete successfully. This is acceptable in CLI workflows, where synchronous execution is often preferred to ensure reliability.
Technical notes
Most of the pipeline task code is already decoupled from middleware and requires minimal changes. Implementing this feature would only require cloning a small part of the middleware logic to execute the next task in sequence.
Benefits
- Direct execution of critical pipelines without a queue
- Simplified debugging and testing
- More control over execution flow in CLI environments
Summary
Currently, tasks can be run without using a queue, but pipelines cannot.
It would be useful to have the ability to run critical pipeline tasks directly via a command, without requiring a queue.
Use case
In some cases, especially for critical operations, it’s convenient to run them from the command line and wait until all jobs complete successfully. This is acceptable in CLI workflows, where synchronous execution is often preferred to ensure reliability.
Technical notes
Most of the pipeline task code is already decoupled from middleware and requires minimal changes. Implementing this feature would only require cloning a small part of the middleware logic to execute the next task in sequence.
Benefits