Add support for MMT/TLV container

[saindriches]

Adds a parser for MMT (ISO/IEC 23008-1) over TLV packets (ARIB STD-B32), the container used by Japanese ISDB-S3 4K/8K broadcasts. This covers what is currently needed to identify and inspect these files; further fields can follow as use cases arise.

• Maps the MPT asset list to HEVC, AAC, and TTML streams, and probes each for codec detail.
• Surfaces the present MH-EIT event as a Menu chapter, with the stream start time and a duration derived from the MPU presentation timestamps, in UTC.
• Adds the MH-SDT service name, and from the MH-audio-component descriptor the audio language, a default-track flag, and a Visually Impaired service kind for audio-description tracks.
• Reports scrambled streams with Encryption set to Encrypted, and still reads timing and service metadata from the clear signaling.

Samples

• descrambled, no program: 20260705_0400_0xB111_de.mmts.zip
• scrambled: 20260222_1903_0xB0E0.mmts.zip

This is my first contribution here, so please point out anything I have done wrong or should change.

[saindriches]

A follow-up round since the first review, still all from the clear signaling so it works on scrambled files:

• Fills the video description (resolution, frame rate, scan, display aspect, BT.2020 colour) from the MH-video-component descriptor, preferring the decoded HEVC ES when present.
• Takes the audio channel count, positions, and layout from the shared AAC channel tables.
• Groups a stream's main and robust-backup versions (hierarchical transmission, used for rain fade) from the Asset_Group_Descriptor into an AlternateGroup, defaulting the selection-level-0 member.
• Binds the streams to the current MPT version so a shorter later program leaves no stale tracks, and settles the A/V config across a program boundary.
• Fixes Duration when there is no MH-TOT (the NTP clock froze after one read), and reads MH-EIT descriptor lengths per the tag's field size so the program name survives a leading 8-bit descriptor.

[cjee21]

More files have to be updated. You may refer to https://github.com/MediaArea/MediaInfoLib/pull/2458/changes. The MediaInfo_Internal::LibraryIsModified part of MediaInfo_File.cpphave to be updated too. You can also activate GitHub Actions on your fork to ensure the CI (which ensures it builds on all tested build systems, compilers and platforms) passes without waiting for a maintainer to approve the workflow run here.

Generally only General_Format needs to be filled for format. The other details come automatically from the database (Format.csv) that is used to generate MediaInfo_Config_Automatic.cpp.

I also see that this is mostly implemented differently from a typical MediaInfo parser in that is does not use MediaInfo's Get/Skip and bitstream parsing functions beyond the header part so it will not produce a trace (--trace_level=1) other than the header, bypasses internal integrity checks and offset tracking as well as may be difficult for others to maintain or read the bitstream syntax. Not sure if that is acceptable since I am just a contributor.

For example, this:

void File_MmtTlv::Parse_Mmtp(const int8u* Data, size_t Size)
{
    if (Size < 12)
        return;
    int8u b0               = Data[0];
    bool  packet_counter   = (b0 >> 5) & 1;
    bool  extension_header = (b0 >> 1) & 1;
    int8u payload_type     = Data[1] & 0x3F;
    const int8u* p = Data + 2;
    size_t       n = Size - 2;
    // packet_id(16) + distribute_timestamp(32) + packet_sequence_number(32)
    if (n < 10)
        return;
    int16u packet_id  = RB16(p);

should be something (may not be exactly like this, just example) like:

void File_MmtTlv::Parse_Mmtp()
{
    bool  packet_counter, extension_header;
    int8u payload_type;
    int16u packet_id;
    BS_Begin();
    Skip_S1(2,                                                  "version");
    Get_SB (   packet_counter,                                  "packet_counter_flag");
    Skip_S1(2,                                                  "FEC_type");
    Skip_S1(1,                                                  "reserved");
    Get_SB (   extension_header,                                "extension_flag");
    Skip_SB(                                                    "RAP_flag");
    Skip_S1(2,                                                  "reserved");
    Get_S1 (6, payload_type,                                    "payload_type");
    BS_End();
    Get_B2 (packet_id,                                          "packet_id");

I did not test but reading the codes, looks like parsing with HEVC and AAC parser may not work properly as well.

[saindriches]

Thank you for the important notes, I will do a review and maybe force-push a rewrite then.

[JeromeMartinez]

@saindriches thank you for your PR. Updated @cjee21 comments are good, please update the PR with that.
It permits to debug the files with MediaTrace.
I may be slow to merge (@cjee21 knows that :( ), but I value your work much.

[saindriches]

I rewrote this to match the style, now squashed into three commits. Structurally it follows the MPEG-TS side: File_MmtTlv does the TLV/MMTP framing and reassembly, then hands tables to File_Mmt and descriptors to File_Mmt_Descriptors, like File_MpegTs / File_Mpeg_Psi / File_Mpeg_Descriptors. Everything is parsed with the usual helpers, so the trace is complete. All STD-B60 tables and descriptors are named even when not parsed further.

Tested on a lot of real captures, and CI is green including the minimal build. Happy to make any changes needed.

[cjee21]

I still see custom byte reading functions and some manual pointer (assigned from MediaInfo's Buffer) and size operations. Is this really needed and cannot be done with MediaInfo's built-in features?

I am not sure about this but this seems incorrect. mpt_desc_len is already parsed by sub parser and after that skip mpt_desc_len again? There's also probably no need to manually pass Buffer to sub parser. Same with desc_loop_len and maybe others.

        Open_Buffer_Init(&Desc);
        Open_Buffer_Continue(&Desc, Buffer + Buffer_Offset + (size_t)Element_Offset, mpt_desc_len);
        Open_Buffer_Finalize(&Desc);
        Skip_XX(mpt_desc_len,                                   "MPT_descriptors");

And what is this Data/Size parameters and void casting that look useless?

void File_MmtTlv::Parse_Mpu(int16u packet_id, int32u seq_num, const int8u* Data, size_t Size)
{
    (void)Data; (void)Size; // parse at the element cursor (== Data/Size handed by Parse_Mmtp)

Can consider using smart pointer for the audio/video parsers so that no need to manually delete and set to null. Since new and delete is at different places far apart, will also prevent mistakes in the future if that part is ever modified.

This is from my quick look-through. I did not test or analyze in-depth.

[saindriches]

Thanks a lot for the review. I don't have deep experience with C++, this is helpful.
I will deal with the descriptor sub-parsing, the a/v parsers, and the avoidable manual byte reads.

In MMT the fragment boundaries are signaled in the MMTP header (the fragmentation_indicator), not in the payload, so the container has to decode those per-packet flags and join the fragments in packet_sequence_number order before a complete signaling message or MFU exists, and the remaining reads work on that assembled buffer.

[cjee21]

Another thing is init of variables can be done directly at the declaration in the header file instead of assigning in the constructor.

[cjee21]

Just some minor things found by static analysis:

Warning	C26819	Unannotated fallthrough between switch labels (es.78).	MediaInfoLib	MediaInfoLib\Source\MediaInfo\Multiple\File_MmtTlv.cpp	1500		

MediaInfoLib/Source/MediaInfo/Multiple/File_MmtTlv.cpp:1323:33:style: Variable 'Found' can be declared as reference to const [constVariableReference]

This part does not look right to me from the offsets but since I don't really know about this format and what's happening there, let @JeromeMartinez comment when he reviews. I am not that good at handling fragments and conversions like handling removing HEVC AnnexB emulation prevention bytes or passing fragmented XMP from JPEG to XMP parser.

002DD62   data_unit_length:                     21 (0x0015)
002DD64   access_unit_delimiter (6 bytes)
002DD64    Header (5 bytes)
002DD64     start_code_prefix_one_3bytes:       1 (0x000001)
002DD67     nal_unit_type:                      35 (0x23) - (6 bits)
002DD67     nuh_layer_id:                       0 (0x00) - (6 bits)
002DD68     nuh_temporal_id_plus1:              1 (0x1) - (3 bits)
002DD69    pic_type:                            0 (0x0) - (3 bits) - I
002DD64   data_unit:                            (21 bytes)
002DD79   data_unit_length:                     45 (0x002D)
002DD7B   video_parameter_set (30 bytes)

Also Trace for AAC and some Data looks like they are outside MPU.