Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
15 changes: 8 additions & 7 deletions docs/api/core.md
Original file line number Diff line number Diff line change
Expand Up @@ -178,11 +178,10 @@ at registration, all executors are assumed to be supported.

{func}`~pyflowreg.core.backend_registry.register_backend`,
{func}`~pyflowreg.core.backend_registry.get_backend`,
{func}`~pyflowreg.core.backend_registry.list_backends`, and
{func}`~pyflowreg.core.backend_registry.is_backend_available` are re-exported
from `pyflowreg.core`;
{func}`~pyflowreg.core.backend_registry.get_backend_executors` is available
from `pyflowreg.core.backend_registry`.
{func}`~pyflowreg.core.backend_registry.list_backends`,
{func}`~pyflowreg.core.backend_registry.is_backend_available`, and
{func}`~pyflowreg.core.backend_registry.get_backend_executors` are
re-exported from `pyflowreg.core`.

```{eval-rst}
.. automodule:: pyflowreg.core.backend_registry
Expand Down Expand Up @@ -216,8 +215,10 @@ flow (`cv2.DISOpticalFlow`) behind the same callable interface as
{func}`~pyflowreg.core.optical_flow.get_displacement`. It is registered only
when OpenCV (`cv2`) is importable. {class}`~pyflowreg.core.diso_optical_flow.DisoOF`
reduces multi-channel input to grayscale using the channel weights, accepts
an optional initial flow field for warm starts, and initializes the OpenCV
DIS object lazily so instances remain picklable.
an optional initial flow field for warm starts (both as its native `w`
keyword and as the `uv` keyword the batch pipelines and `FlowRegLive` pass,
with `w` taking precedence), and initializes the OpenCV DIS object lazily so
instances remain picklable.

Compared to the variational `flowreg` backend, `diso` has the following
restrictions (enforced when the backend is resolved through `OFOptions`):
Expand Down
6 changes: 3 additions & 3 deletions docs/snippets/user_guide/online_processing/streaming_loop.py
Original file line number Diff line number Diff line change
Expand Up @@ -13,10 +13,10 @@
video = reader[:]
reader.close()

# Scale to [0, 1]: incoming frames are normalized against the stored
# reference range, so they should arrive in a comparable intensity range
# Frames are normalized against the raw reference's intensity range
# internally, so they can stay at the recording's native scale (uint16
# here); cast to float32 only to keep the demo arithmetic simple
video = video.astype(np.float32)
video = (video - video.min()) / (video.max() - video.min())

# Configure optical flow; quality_setting is forced to "fast" internally
options = OFOptions(
Expand Down
4 changes: 4 additions & 0 deletions docs/user_guide/backends.md
Original file line number Diff line number Diff line change
Expand Up @@ -28,6 +28,10 @@ Notes:
resolved from `OFOptions`.
- `diso` excludes multiprocessing because the multiprocessing workers import
the variational solver directly and do not reconstruct registry backends.
- `diso` honors the flow initialization (`uv`) the pipelines pass between
batches and frames, forwarding it to OpenCV DIS as the initial flow; the
other variational solver keywords (`alpha`, `iterations`, ...) are accepted
but ignored.

You can inspect what is registered in the current environment:

Expand Down
2 changes: 1 addition & 1 deletion docs/user_guide/configuration.md
Original file line number Diff line number Diff line change
Expand Up @@ -194,7 +194,7 @@ The `flowreg_torch` backend additionally accepts `dtype` (`"float32"` or `"float
:end-before: "[docs:end]"
```

When a list of indices is given, the corresponding frames are read from the input, motion-compensated against their mean using increased regularization, and averaged to form the reference.
When a list of indices is given, the corresponding frames are read from the input, motion-compensated against their mean using increased regularization, and averaged to form the reference. Indices beyond the recording length are clipped to the last frame and the resulting duplicate indices are removed with a printed warning, so a short recording preregisters each available frame exactly once. When [cross-correlation pre-alignment](prealignment.md) is enabled, it also applies during this preregistration.

### Updating the Reference Frame

Expand Down
2 changes: 1 addition & 1 deletion docs/user_guide/file_formats.md
Original file line number Diff line number Diff line change
Expand Up @@ -28,7 +28,7 @@ Readers are selected by file extension: `.tif`/`.tiff` (TIFF), `.h5`/`.hdf5`/`.h
- Dataset names: `ch1`, `ch2`, ..., `chN` (pattern configurable via `dataset_names`)
- File-level attributes: `frame_count`, `height`, `width`, `n_channels`, `dimension_ordering`, `format`, `dataset_names`

The reader interprets 3D datasets as `(T, H, W)` with one dataset per channel, and a 4D dataset as `(T, H, W, C)`.
The reader interprets 3D datasets as `(T, H, W)` with one dataset per channel by default, and a 4D dataset as `(T, H, W, C)`. For files stored in a different axis order, pass the same `dimension_ordering` option the writer uses (axis positions of height, width, time — e.g. `dimension_ordering=(0, 1, 2)` for `(H, W, T)` datasets) to `get_video_file_reader`; reads are permuted back to `(T, H, W, C)`.

### TIFF (.tif, .tiff)

Expand Down
14 changes: 6 additions & 8 deletions docs/user_guide/online_processing.md
Original file line number Diff line number Diff line change
Expand Up @@ -41,11 +41,11 @@ flow_reg.set_reference()

Behavior by input shape:

- **4D stack `(T, H, W, C)`** -- the frames are preregistered with `compensate_arr` against their temporal mean, using a copy of the options with `quality_setting="balanced"`. The mean of the registered frames becomes the reference.
- **3D `(H, W, C)` or 2D `(H, W)`** -- used directly as a single-frame reference, without preregistration. A 3D input is always interpreted as one multi-channel frame, so a grayscale stack `(T, H, W)` must be given an explicit channel axis first (e.g. `frames[..., None]`).
- **Stack `(T, H, W, C)` or `(T, H, W)`** -- the frames are preregistered with `compensate_arr` against their temporal mean, using a copy of the options with `quality_setting="balanced"`. The mean of the registered frames becomes the reference.
- **Single frame `(H, W, C)` or `(H, W)`** -- used directly as the reference, without preregistration. A 3D input is interpreted as one multi-channel frame when its last dimension is at most 4 (the `ArrayReader` convention), and as a grayscale `(T, H, W)` stack otherwise.
- **No argument** -- the frames accumulated in the internal reference buffer are stacked and processed as above. Raises `ValueError` if the buffer is empty.

After the reference is established, it is normalized and spatially Gaussian-filtered; the min/max of this filtered reference is stored and used to normalize every incoming frame (per channel when `channel_normalization="separate"`, jointly otherwise). Setting a reference also resets the stored flow initialization and clears the temporal filter buffer.
After the reference is established, it is normalized and spatially Gaussian-filtered; the min/max of the raw reference is stored and used to normalize every incoming frame (per channel when `channel_normalization="separate"`, jointly otherwise) -- the same convention as the batch pipeline, so frames can arrive at the recording's native scale (e.g. uint16). Setting a reference also resets the stored flow initialization and clears the temporal filter buffer.

`reset_reference(new_reference)` is an alias that calls `set_reference()` with the given array.

Expand All @@ -65,14 +65,12 @@ If no reference has been set yet, the call instead appends the frame to the refe

Once a reference exists, each call:

1. Normalizes the frame using the stored reference min/max values.
1. Normalizes the frame using the raw reference's min/max values.
2. Applies the 2D spatial Gaussian filter and pushes the result into the temporal buffer.
3. Applies the causal temporal half-kernel filter (see below).
4. Computes the displacement field against the filtered reference, initialized with the previous frame's flow.
5. Warps the original (unfiltered) input frame with the resulting field, using `options.interpolation_method` (default cubic).
6. Every `reference_update_interval`-th frame, blends the warped frame into the reference with weight `reference_update_weight` and refreshes the normalization min/max from the updated reference.

The `normalize` keyword of `__call__` is currently unused; normalization is always applied.
6. Every `reference_update_interval`-th frame, blends the warped frame into the reference with weight `reference_update_weight` and refreshes the normalization min/max from the updated raw reference.

Convenience methods:

Expand All @@ -88,7 +86,7 @@ Batch processing filters each batch with a 3D Gaussian over `(y, x, t)`. In a st
2. The 2D-filtered frames are kept in a circular buffer of size `max(1, int(truncate * sigma_t + 0.5) + 1)`, where `sigma_t` is the temporal sigma.
3. The frame used for flow estimation is a weighted average of the current and past buffered frames with normalized half-Gaussian weights (`pyflowreg.util.image_processing.gaussian_filter_1d_half_kernel`). Only the current and past frames contribute, so the filter is causal and adds no latency, but it is not identical to the symmetric temporal filtering used in batch mode.

With the `OFOptions` default `sigma` of `[[1.0, 1.0, 0.1], [1.0, 1.0, 0.1]]`, the temporal sigma of 0.1 yields a buffer of size 1 and no effective temporal filtering; increase the third sigma component to enable it. When per-channel sigmas are configured, `FlowRegLive` currently uses the first channel's spatial sigmas for all channels and the maximum temporal sigma across channels.
With the `OFOptions` default `sigma` of `[[1.0, 1.0, 0.1], [1.0, 1.0, 0.1]]`, the temporal sigma of 0.1 yields a buffer of size 1 and no effective temporal filtering; increase the third sigma component to enable it. When per-channel sigmas are configured, each channel is spatially filtered with its own sigmas; the temporal half-kernel uses the maximum temporal sigma across channels.

## Complete Example

Expand Down
6 changes: 4 additions & 2 deletions docs/user_guide/prealignment.md
Original file line number Diff line number Diff line change
Expand Up @@ -71,12 +71,14 @@ the result.
`compensate_recording` and `compensate_arr`. All three executors
(sequential, threading, multiprocessing) implement it; see
[Parallelization](parallelization.md) for executor selection.
- Reference preregistration in `OFOptions.get_reference_frame` forwards the
`cc_*` settings, so an enabled pre-alignment also applies while building
the reference. (The MATLAB reference does not pre-align its
preregistration; this is a deliberate improvement.)
- It is **not** applied by:
- Direct `get_displacement` calls — the flow backends do not accept the
`cc_*` parameters; the executors consume them before invoking the solver.
- `FlowRegLive` online processing, which has no pre-alignment handling.
- Reference preregistration in `OFOptions.get_reference_frame`, which builds
its own internal options without `cc_initialization`.
- Each pre-aligned frame requires two additional warps and one FFT-based
correlation, so leave it disabled when frame-to-frame motion is small.

Expand Down
2 changes: 1 addition & 1 deletion docs/user_guide/workflows.md
Original file line number Diff line number Diff line change
Expand Up @@ -162,7 +162,7 @@ processor.set_reference(corrected_frames)
processor.reset_reference(new_reference_frame)
```

Note that `set_reference()` interprets a 3D array as a single multi-channel frame `(H, W, C)`, not as a grayscale stack. Calling `set_reference()` without arguments uses the frames buffered internally before a reference existed.
Note that `set_reference()` interprets a 3D array as a single multi-channel frame `(H, W, C)` when its last dimension is at most 4 (the `ArrayReader` convention), and as a grayscale `(T, H, W)` stack otherwise. Calling `set_reference()` without arguments uses the frames buffered internally before a reference existed.

### When to Use

Expand Down
3 changes: 3 additions & 0 deletions requirements_win.txt
Original file line number Diff line number Diff line change
Expand Up @@ -8,3 +8,6 @@ numba
pywin32
pydantic
hdf5storage
pyyaml
Pillow
tomli
4 changes: 4 additions & 0 deletions src/pyflowreg/core/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -35,6 +35,8 @@
List all available backends
is_backend_available
Check if a specific backend is available
get_backend_executors
Get the parallelization executors supported by a backend

See Also
--------
Expand All @@ -54,6 +56,7 @@
get_backend,
list_backends,
is_backend_available,
get_backend_executors,
)

__all__ = [
Expand All @@ -62,6 +65,7 @@
"get_backend",
"list_backends",
"is_backend_available",
"get_backend_executors",
]

# Register built-in backends
Expand Down
44 changes: 29 additions & 15 deletions src/pyflowreg/core/diso_optical_flow.py
Original file line number Diff line number Diff line change
Expand Up @@ -30,9 +30,11 @@
``gradient_descent_iterations``, ``patch_size``, ``patch_stride``,
``use_mean_normalization``, ``use_spatial_propagation``).
- Variational solver keywords forwarded by the pipelines (``alpha``,
``iterations``, ``uv``, ``const_assumption``, ``gnc_schedule``, ...) are
``iterations``, ``const_assumption``, ``gnc_schedule``, ...) are
accepted by :meth:`DisoOF.__call__` for signature compatibility but
ignored.
ignored. The flow initialization keyword ``uv`` is the exception: it is
honored as the DIS warm start (``w`` takes precedence when both are
given).

References
----------
Expand Down Expand Up @@ -339,16 +341,21 @@ def __call__(
conventions as ``fixed``.
w : ndarray, optional
Initial displacement field of shape (H, W, 2) used as warm
start. Used only if it is an ndarray with exactly this layout;
it is cast to float32.
start. Takes precedence over ``uv`` when both are given. Used
only if it is an ndarray with exactly this layout and the same
spatial size as the images; it is converted to contiguous
float32.
weight : ndarray, optional
Channel weights for multi-channel inputs; see ``_to_gray`` for
the accepted formats.
**kwargs : dict
Accepted for signature compatibility with the variational
:func:`pyflowreg.core.optical_flow.get_displacement` (e.g.
``alpha``, ``iterations``, ``uv``, ``const_assumption``,
``gnc_schedule``); all extra keyword arguments are ignored.
:func:`pyflowreg.core.optical_flow.get_displacement`. The
``uv`` keyword (the flow initialization passed by the batch
pipelines) is honored as warm start when ``w`` is not given;
all other extra keyword arguments (e.g. ``alpha``,
``iterations``, ``const_assumption``, ``gnc_schedule``) are
ignored.

Returns
-------
Expand All @@ -362,8 +369,9 @@ def __call__(
Notes
-----
The batch pipelines pass the flow initialization as the keyword
``uv``, which this wrapper does not read; a warm start is only used
when supplied via ``w``.
``uv``; it is forwarded to ``cv2.DISOpticalFlow.calc`` as the
initial flow. A warm start whose spatial size does not match the
images is silently skipped (mirroring OpenCV's own size check).
"""
self._ensure()

Expand All @@ -374,15 +382,21 @@ def __call__(
# Normalize to [0,255] uint8
A, B = self._normalize(a, b)

# Prepare initial flow if provided
# Prepare initial flow if provided. Precedence: explicit ``w`` wins,
# otherwise fall back to the pipeline keyword ``uv`` (used by the
# executors, FlowRegLive and compensate_recording).
init = None
init_src = w if w is not None else kwargs.get("uv")
if (
w is not None
and isinstance(w, np.ndarray)
and w.ndim == 3
and w.shape[2] == 2
isinstance(init_src, np.ndarray)
and init_src.ndim == 3
and init_src.shape[2] == 2
and init_src.shape[:2] == A.shape[:2]
):
init = w.astype(np.float32, copy=False)
# Copy: cv2.DISOpticalFlow.calc treats the init flow as an
# InputOutputArray and writes the result into it in place; cv2
# also requires contiguous float32 (CV_32FC2) of image size.
init = np.array(init_src, dtype=np.float32, order="C", copy=True)

# Compute optical flow
flow = self._dis.calc(A, B, init)
Expand Down
Loading
Loading