refactor[next]: replace factory-boy factories with plain builders, fixing run_gtfn_imperative

docs/development/ADRs/next/0028-Plain-Builders-Instead-of-Factories.md

@@ -0,0 +1,123 @@
+---
+tags: [backend, otf, toolchain, workflows, dependencies]
+---
+
+# Plain Builders Instead of factory-boy Factories
+
+- **Status**: valid
+- **Authors**: Enrique González Paredes (@egparedes)
+- **Created**: 2026-08-20
+- **Updated**: 2026-08-20
+
+In the context of composing the GTFN and DaCe backends and their OTF compile
+workflows, facing a production dependency on `factory-boy` — a test-data
+library — whose `Trait` / `SubFactory` / `SelfAttribute` / `LazyAttribute`
+machinery and stringly-typed `__`-path overrides are invisible to the type
+checker, we decided to replace the factory classes with plain builder
+functions over the existing frozen dataclasses, and to validate injected
+sub-components explicitly, to achieve statically checked composition, one
+fewer runtime dependency, and loud failures where the factories failed
+silently.
+
+## Context
+
+Every object these factories build — `Backend`, `OTFCompileWorkflow`,
+`GTFNTranslationStep`, `DaCeTranslator`, the compilers — is already a frozen
+dataclass. `factory-boy` added a second, parallel construction language on top:
+
+- **Untyped.** The declarations are class attributes of a `Params` block, so
+  `mypy` cannot check them. `src/` carried **8** \`# type: ignore[assignment]
+
+  # factory-boy typing not precise enough\` suppressions solely to keep the
+
+  factories quiet.
+
+- **Silently wrong.** Overrides are `__`-delimited strings resolved at
+  runtime. When a path does not resolve, nothing happens. This was not
+  hypothetical: `run_gtfn_imperative` was declared as
+
+  ```python
+  run_gtfn_imperative = GTFNBackendFactory(
+      name_postfix="_imperative",
+      otf_workflow__translation__use_imperative_backend=True,
+  )
+  ```
+
+  but the `cached_translation` trait replaces `translation` with a
+  `CachedStep`, so the path never reached the wrapped `GTFNTranslationStep`.
+  The backend had `use_imperative_backend=False` — it was the declarative
+  backend under another name, and the `GTFN_CPU_IMPERATIVE` entry of the test
+  matrix had therefore never exercised imperative code generation.
+  `run_gtfn_no_transforms` was likewise named `run_gtfn_cpu`, colliding with
+  `run_gtfn`.
+
+- **A runtime dependency for a test-time concern.** `factory-boy` sat in
+  `[project] dependencies`, shipped to every user, to compose four backends.
+
+## Decision
+
+Factory classes are replaced by **plain builder functions**; `factory-boy`
+moves to the `test` dependency group, where the `cartesian` and `eve` IR
+test-data factories keep using it for what it is designed for.
+
+Builders follow two rules:
+
+1. **A builder takes cross-cutting configuration only** — device, caching,
+   build type, auto-optimize — and uses it to configure the steps it creates.
+2. **An injected sub-component is used verbatim.** A caller that wants a
+   different translation step builds one and passes it; the builder never
+   reaches into it to stamp fields onto it.
+
+```python
+run_gtfn_imperative = make_gtfn_backend(
+    name_postfix="_imperative",
+    translation=gtfn_module.GTFNTranslationStep(use_imperative_backend=True),
+)
+```
+
+Rule 2 creates one hazard: an injected step could disagree with the
+cross-cutting configuration — a CPU translation step in a GPU toolchain.
+`workflow.check_device_agreement(step, device_type, what)` turns that into a
+`ValueError` at construction time. It inspects only steps that structurally
+declare a device (the `workflow.DeviceConfigurable` protocol) and is used to
+**check**, never to mutate.
+
+We considered a `with_changes(step, **changes)` helper that stamps
+cross-cutting fields onto whichever component is present, applying only the
+fields the target declares. We rejected it: silently ignoring the fields a
+target does not declare reproduces exactly the failure mode that motivated
+this ADR — the `run_gtfn_imperative` bug is what a silent no-op looks like
+after a year. Checking is the same amount of introspection with the opposite
+failure mode.
+
+Builder defaults preserve the previous factory semantics: a standalone
+compile-workflow builder leaves translation caching **off** (the
+`cached_translation` trait was opt-in), while the backend builders turn it on.
+
+## Consequences
+
+- Composition is ordinary, statically checked Python. The 8 factory-related
+  `type: ignore` suppressions are gone, and a misspelled parameter is now a
+  `TypeError` at import rather than a silently ignored override.
+- One fewer runtime dependency.
+- **`run_gtfn_imperative` now actually uses the imperative backend.** This is
+  a behaviour change: the `GTFN_CPU_IMPERATIVE` test-matrix entry begins
+  exercising imperative code generation for the first time, and it
+  immediately fails on the pre-existing IR defect tracked in issue #2810.
+  Two call sites are xfailed against that issue (`test_hdiff` and
+  `test_concat_where::test_lap_like[static_domains]`); fixing the defect is
+  out of scope for a construction refactor.
+- **`run_gtfn_no_transforms` is renamed** from `run_gtfn_cpu` to
+  `run_gtfn_cpu_no_transforms`, removing the collision with `run_gtfn`. No
+  cache is affected: the build cache keys on the entry-point name plus a
+  fingerprint of the `ExtensionSource`, and the translation-cache directory
+  is keyed on the literal backend family (`gtfn` / `dace`). `Backend.name`
+  reaches only the metrics source key and one error message, so what the
+  collision actually cost was two distinct backends sharing one metrics
+  identity.
+- All other pre-built backends are unchanged, verified field-by-field against
+  the previous construction.
+- Customizing a single knob of a sub-component now means building that
+  component, rather than passing a `__`-path string. This is more explicit and
+  slightly more verbose; `make_dace_backend` keeps its translator-local
+  keyword arguments so existing external callers are unaffected.

docs/development/ADRs/next/README.md

@@ -52,6 +52,7 @@ Writing a new ADR is simple:
 - [0016 - Multiple Backends and Build Systems](0016-Multiple-Backends-and-Build-Systems.md)
 - [0017 - Toolchain Configuration](0017-Toolchain-Configuration.md)
 - [0027 - External Workspace Memory for DaCe Transients](0027-External_Workspace_Memory.md)
+- [0028 - Plain Builders Instead of factory-boy Factories](0028-Plain-Builders-Instead-of-Factories.md)
 
 ### Python Integration
 

docs/user/next/advanced/HackTheToolchain.md

@@ -46,25 +46,28 @@ skip_linting_transforms = SkipLinting(**same_steps)
 skip_linting_transforms.step_order(DUMMY_FOP)
 ```
 
-## Alternative Factory
+## Alternative Workflow
+
+Compile workflows are plain frozen dataclasses, so a variant is built by
+replacing the steps you want to change on one the builders produced.
 
 ```python
-class MyCodeGen: ...
+import dataclasses
 
 
-class Cpp2BindingsGen: ...
+class MyCodeGen: ...
 
 
-class PureCpp2WorkflowFactory(gtx.program_processors.runners.gtfn.GTFNCompileWorkflowFactory):
-    translation: workflow.Workflow[
-        gtx.otf.stages.CompilableProgramDef, gtx.otf.artifacts.ProgramSource
-    ] = MyCodeGen()
-    bindings: workflow.Workflow[
-        gtx.otf.artifacts.ProgramSource, gtx.otf.artifacts.ExtensionSource
-    ] = Cpp2BindingsGen()
+class Cpp2BindingsGen: ...
 
 
-PureCpp2WorkflowFactory(cmake_build_type=gtx.config.CMAKE_BUILD_TYPE.DEBUG)
+pure_cpp2_workflow = dataclasses.replace(
+    gtx.program_processors.runners.gtfn.make_gtfn_compile_workflow(
+        cmake_build_type=gtx.config.CMakeBuildType.DEBUG
+    ),
+    translation=MyCodeGen(),
+    bindings=Cpp2BindingsGen(),
+)
 ```
 
 ## Invent new Workflow Types

docs/user/next/advanced/WorkflowPatterns.md

@@ -17,7 +17,6 @@ jupyter:
 import dataclasses
 import re
 
-import factory
 
 import gt4py.next as gtx
 
@@ -199,7 +198,7 @@ Let's say we want to make our calculation workflow compatible with string input.
 
 ```python editable=true slideshow={"slide_type": ""}
 # A plain conversion step turning a string into an int, chained into the
-# workflow below and reused by `StrToIntFactory(cached=True)`.
+# workflow below and reused by `make_str_to_int(cached=True)`.
 def to_int(inp: str) -> int:
     assert isinstance(inp, str), "Can not work with 'int'!"  # yes, this is horribly contrived
     return int(inp)
@@ -214,9 +213,9 @@ str_calc("1")
 
 <!-- #region editable=true slideshow={"slide_type": ""} -->
 
-### Step with factory (builder)
+### Step with a builder
 
-If a step can be useful with different combinations of parameters and wrappers, it should have a factory. In this case we will add a neutral wrapper around it, so we can put any combination of wrappers into that:
+If a step is useful with different combinations of parameters and wrappers, give it a **builder function**: a plain function taking the cross-cutting options and returning the assembled step. Steps are frozen dataclasses, so the builder is ordinary code — no factory framework involved, and the result is fully type-checked.
 
 <!-- #endregion -->
 
@@ -229,32 +228,23 @@ class AnyStrToInt(gtx.otf.workflow.ChainableWorkflowMixin[str | int, int]):
         return self.inner_step(inp)
 
 
-class StrToIntFactory(factory.Factory):
-    class Meta:
-        model = AnyStrToInt
+def make_str_to_int(
+    *, cached: bool = False, step: gtx.otf.workflow.Workflow[str, int] = to_int
+) -> AnyStrToInt:
+    if cached:
+        step = gtx.otf.workflow.CachedStep.in_memory(step=step, input_fingerprinter=str)
+    return AnyStrToInt(inner_step=step)
 
-    class Params:
-        default_step = to_int
-        cached = factory.Trait(
-            inner_step=factory.LazyAttribute(
-                lambda o: gtx.otf.workflow.CachedStep.in_memory(
-                    step=o.default_step, input_fingerprinter=str
-                )
-            )
-        )
 
-    inner_step = factory.LazyAttribute(lambda o: o.default_step)
-
-
-cached = StrToIntFactory(cached=True)
-uncached = StrToIntFactory()
+cached = make_str_to_int(cached=True)
+uncached = make_str_to_int()
 uncached.inner_step
 ```
 
 ### Example in the Wild
 
 ```python
-gtx.ffront.past_passes.linters.LinterFactory??
+gtx.ffront.past_passes.linters.linter_factory??
 ```
 
 <!-- #region editable=true slideshow={"slide_type": ""} tags=["skip-execution"] -->
@@ -413,5 +403,5 @@ gtx.program_processors.runners.gtfn.run_gtfn_gpu.executor.otf_workflow??
 ```
 
 ```python
-gtx.program_processors.runners.gtfn.GTFNBackendFactory??
+gtx.program_processors.runners.gtfn.make_gtfn_backend??
 ```

pyproject.toml

@@ -44,6 +44,7 @@ profiling = [
 scripts = ["pyyaml>=6.0.1", "typer>=0.16.0", "packaging"]
 test = [
   'coverage[toml]>=7.6.1',
+  'factory-boy>=3.3.3',
   'hypothesis>=6.0.0',
   'nbmake>=1.4.6',
   'nox>=2025.02.09',
@@ -101,7 +102,6 @@ dependencies = [
   'dace>=2.0.0a7,<2.0.0a8',
   'deepdiff>=8.1.0',
   'devtools>=0.6',
-  'factory-boy>=3.3.3',
   "filelock>=3.18.0",
   'frozendict>=2.3',
   'gridtools-cpp>=2.3.9,==2.*',
@@ -260,12 +260,6 @@ module = 'gt4py.next.iterator.*'
 ignore_errors = true
 module = 'gt4py.next.iterator.runtime'
 
-[[tool.mypy.overrides]]
-ignore_missing_imports = true
-implicit_reexport = true
-# factory-boy is broken, see https://github.com/FactoryBoy/factory_boy/pull/1114
-module = "factory.*"
-
 # -- pytest --
 [tool.pytest]
 

src/gt4py/next/otf/workflow.py

@@ -17,7 +17,7 @@
 
 from typing_extensions import Self
 
-from gt4py._core import filecache
+from gt4py._core import definitions as core_defs, filecache
 from gt4py.eve.extended_typing import OpaqueMutableMapping
 from gt4py.next import config, fingerprinting, utils
 
@@ -361,3 +361,37 @@ def __call__(self, inp: StartT) -> EndT:
 
     def cache_key(self, inp: StartT) -> str:
         return self.step_fingerprinter((self._step_fingerprint, self.input_fingerprinter(inp)))
+
+
+@typing.runtime_checkable
+class DeviceConfigurable(Protocol):
+    """A step that records the device it was configured for."""
+
+    device_type: core_defs.DeviceType
+
+
+def check_device_agreement(step: Any, device_type: core_defs.DeviceType, what: str) -> None:
+    """
+    Raise if an injected step is configured for a different device.
+
+    Builders configure the steps they create themselves from the requested
+    device, but an injected step is used verbatim. Without this check a
+    mismatch would silently produce a pipeline whose steps disagree about the
+    target device, which surfaces much later as a confusing compilation or
+    runtime failure.
+
+    Args:
+        step: The step to check. Steps that do not record a device are accepted.
+        device_type: The device the surrounding pipeline is built for.
+        what: Name of the step, used in the error message.
+
+    Raises:
+        ValueError: If `step` records a device other than `device_type`.
+    """
+    if isinstance(step, DeviceConfigurable) and step.device_type is not device_type:
+        raise ValueError(
+            f"The injected {what} is configured for device '{step.device_type.name}',"
+            f" but the workflow is being built for '{device_type.name}'. Build the step"
+            f" with 'device_type=DeviceType.{device_type.name}' or leave it out to get"
+            " the default."
+        )

src/gt4py/next/program_processors/codegens/gtfn/gtfn_module.py

@@ -12,7 +12,6 @@
 import functools
 from typing import Any, Final, Optional
 
-import factory
 import numpy as np
 
 from gt4py._core import definitions as core_defs
@@ -286,13 +285,8 @@ def _not_implemented_for_device_type(self) -> NotImplementedError:
         )
 
 
-class GTFNTranslationStepFactory(factory.Factory[GTFNTranslationStep]):
-    class Meta:
-        model = GTFNTranslationStep
+translate_program_cpu: Final[stages.TranslationStep] = GTFNTranslationStep()
 
-
-translate_program_cpu: Final[stages.TranslationStep] = GTFNTranslationStepFactory()  # type: ignore[assignment] # factory-boy typing not precise enough
-
-translate_program_gpu: Final[stages.TranslationStep] = GTFNTranslationStepFactory(  # type: ignore[assignment] # factory-boy typing not precise enough
+translate_program_gpu: Final[stages.TranslationStep] = GTFNTranslationStep(
     device_type=core_defs.DeviceType.CUDA
 )

src/gt4py/next/program_processors/formatters/gtfn.py

@@ -16,7 +16,7 @@
 
 @program_formatter.program_formatter
 def format_cpp(program: itir.Program, *args: Any, **kwargs: Any) -> str:
-    gtfn_translation = gtfn.GTFNCompileWorkflowFactory(cached_translation=False).translation
+    gtfn_translation = gtfn.make_gtfn_compile_workflow().translation
     assert isinstance(gtfn_translation, GTFNTranslationStep)
     return gtfn_translation.generate_stencil_source(
         program,