Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion docs/how-to/write-a-converter.md
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@ Raw `sensor_msgs/msg/Image` and `foxglove.RawImage` channels are not supported.

Those are the HFlow-specific acceptance rules. Other malformed records or payloads can still be rejected by the MCAP decoder, the selected message decoder, or ffmpeg. HFlow does not require source provenance metadata, canonical chunk layout, globally sorted messages, or canonical identifiers before ingest.

When ingest cannot create an episode, `failure_kind` in the [`ingest_failures` table](../CATALOG.md) now tells you which side of the boundary it was: `source-unsupported` means the transform read the file fine but refused on its contents (the cases named above -- unsupported image format, a passthrough video violation, a raw image channel, a derived-topic collision), `source-unreadable` means the file was not a readable MCAP at all, and `infrastructure` is everything else the engine did not recognize. Still inspect `error_type` and `message` alongside `failure_kind` for the specific reason -- the kind tells you where to look, not what went wrong. The classification logic is in [`classify_ingest_failure`](../../src/hflow/ingest_ledger.py).
When ingest cannot create an episode, `failure_kind` in the [`ingest_failures` table](../CATALOG.md) now tells you which side of the boundary it was: `source-unsupported` means the transform read the file fine but refused on its contents (the cases named above -- an unsupported image format, mixed image formats on one channel, a passthrough video violation, a raw image channel, a derived-topic collision), `source-unreadable` means the file was not a readable MCAP at all, and `infrastructure` is everything else the engine did not recognize. Still inspect `error_type` and `message` alongside `failure_kind` for the specific reason -- the kind tells you where to look, not what went wrong. The classification logic is in [`classify_ingest_failure`](../../src/hflow/ingest_ledger.py).

## What HFlow adds

Expand Down
9 changes: 5 additions & 4 deletions src/hflow/transform.py
Original file line number Diff line number Diff line change
Expand Up @@ -116,9 +116,10 @@ class SourceNotConforming(ValueError):

Raised where the transform has looked at what a channel actually carries
and found it outside what v1 supports or bridges -- an unsupported image
format, a passthrough video channel violating the canonical convention, an
unsupported raw image schema, a derived-topic collision. A subclass of
``ValueError`` so existing callers matching on that type keep working.
format, a compressed-image channel mixing formats, a passthrough video
channel violating the canonical convention, an unsupported raw image
schema, a derived-topic collision. A subclass of ``ValueError`` so existing
callers matching on that type keep working.

This is the ONLY path into
:attr:`~hflow.ingest_ledger.IngestFailureKind.SOURCE_UNSUPPORTED`:
Expand Down Expand Up @@ -526,7 +527,7 @@ def _decode_compressed_images(
header = getattr(decoded, "header", None)
frame_ids.append(str(header.frame_id) if header is not None else "")
if len(formats) != 1:
raise ValueError(f"camera topic {topic!r} mixes image formats {sorted(formats)}")
raise SourceNotConforming(f"camera topic {topic!r} mixes image formats {sorted(formats)}")
return images, frame_ids, _input_codec_for_image_format(formats.pop(), topic)


Expand Down
36 changes: 36 additions & 0 deletions tests/test_ingest_ledger.py
Original file line number Diff line number Diff line change
Expand Up @@ -42,6 +42,42 @@ def test_unsupported_compressed_image_format_classifies_as_source_unsupported()
assert classify_ingest_failure(raised.value) == IngestFailureKind.SOURCE_UNSUPPORTED


def test_mixed_compressed_image_formats_classify_as_source_unsupported(tmp_path: Path) -> None:
from foxglove_schemas_protobuf.CompressedImage_pb2 import CompressedImage
from mcap_protobuf.schema import build_file_descriptor_set

source = tmp_path / "mixed-image-formats.mcap"
with source.open("wb") as stream:
writer = StockWriter(stream)
writer.start(profile="", library="test")
schema_id = writer.register_schema(
name="foxglove.CompressedImage",
encoding="protobuf",
data=build_file_descriptor_set(CompressedImage).SerializeToString(),
)
channel_id = writer.register_channel(
topic="/cam", message_encoding="protobuf", schema_id=schema_id
)
for message_index, image_format in enumerate(("jpeg", "png"), start=1):
log_time = message_index * 10**9
message = CompressedImage()
message.timestamp.FromNanoseconds(log_time)
message.frame_id = "cam"
message.data = b"image bytes are not decoded before the format consistency check"
message.format = image_format
writer.add_message(
channel_id,
log_time=log_time,
data=message.SerializeToString(),
publish_time=log_time,
)
writer.finish()

with pytest.raises(SourceNotConforming, match="mixes image formats") as raised:
write_canonical_episode(source, tmp_path / "out.mcap")
assert classify_ingest_failure(raised.value) == IngestFailureKind.SOURCE_UNSUPPORTED


def test_nonconforming_passthrough_video_classifies_as_source_unsupported(tmp_path: Path) -> None:
from foxglove_schemas_protobuf.CompressedVideo_pb2 import CompressedVideo
from mcap_protobuf.schema import build_file_descriptor_set
Expand Down
Loading