Add common media file definitions for BEP044/BEP047

mkdocs.yml

@@ -47,6 +47,7 @@ nav:
           - Coordinate systems: appendices/coordinate-systems.md
           - Quantitative MRI: appendices/qmri.md
           - Arterial Spin Labeling: appendices/arterial-spin-labeling.md
+          - Media files: appendices/media-files.md
           - Cross modality correspondence: appendices/cross-modality-correspondence.md
       - Changelog: CHANGES.md
   - The BIDS Website:

src/appendices/media-files.md

@@ -0,0 +1,164 @@
+# Media Files
+
+## Introduction
+
+Several BIDS datatypes make use of media files — audio recordings, video recordings,
+combined audio-video recordings, and still images.
+This appendix defines the common file formats, metadata conventions,
+and codec identification schemes shared across all datatypes that use media files.
+
+The following media suffixes are defined:
+
+{{ MACROS___make_suffix_table(["audio", "video", "audiovideo", "image"]) }}
+
+Datatypes that incorporate media files (for example, behavioral recordings or stimuli)
+define their own file-naming rules, directory placement, and datatype-specific metadata.
+The conventions described here apply uniformly to all such datatypes.
+
+### Relationship to the `photo` suffix
+
+The media file definitions introduced here generalize the concept of all media in BIDS.
+The existing `photo` suffix (used for photographs of anatomical landmarks,
+head localization coils, and tissue samples) predates this framework and covers
+a narrower use case — still images in specific electrophysiology and microscopy datatypes.
+
+The media suffixes (`audio`, `video`, `audiovideo`, `image`) are intended as the
+general-purpose mechanism for all media content in BIDS.
+The media file framework should be generally adopted for new datatypes,
+and a future proposal may deprecate the `photo` suffix in favor of the broader `image`
+suffix with appropriate migration tooling
+(see [bids-utils](https://github.com/bids-standard/bids-utils)).
+
+## Supported Formats
+
+### Audio formats
+
+{{ MACROS___make_extension_table(["wav", "mp3", "aac", "ogg"]) }}
+
+### Video container formats
+
+{{ MACROS___make_extension_table(["mp4", "avi", "mkv", "webm"]) }}
+
+### Image formats
+
+{{ MACROS___make_extension_table(["jpg", "png", "svg", "webp", "tif", "tiff"]) }}
+
+When choosing a format, consider the trade-off between file size and data fidelity.
+Uncompressed or lossless formats (WAV, PNG, TIFF) preserve full quality
+but produce larger files.
+Lossy formats (MP3, AAC, JPEG) significantly reduce file size
+at the cost of some data loss.
+
+## Media Stream Metadata
+
+Media files SHOULD be accompanied by a JSON sidecar file
+containing technical metadata about the media streams.
+The following metadata fields are defined for media files.
+
+### Duration
+
+Applies to suffixes: `audio`, `video`, `audiovideo`.
+
+{{ MACROS___make_sidecar_table("media.MediaDuration") }}
+
+`RecordingDuration` reuses the existing BIDS metadata field already defined for
+electrophysiology recordings (EEG, iEEG, MEG, and others).
+
+### Audio stream properties
+
+Applies to suffixes: `audio`, `audiovideo`.
+
+{{ MACROS___make_sidecar_table("media.MediaAudioProperties") }}
+
+Note: `AudioSampleRate` is used instead of the existing `SamplingFrequency` field
+because audio-video files require distinguishing the audio sampling rate from the
+video frame rate. The `Audio` prefix makes this unambiguous in multi-stream containers.
+
+### Image properties
+
+Applies to suffixes: `video`, `audiovideo`, `image`.
+
+{{ MACROS___make_sidecar_table("media.MediaImageProperties") }}
+
+### Video stream properties
+
+Applies to suffixes: `video`, `audiovideo`.
+
+{{ MACROS___make_sidecar_table("media.MediaVideoProperties") }}
+
+## Codec Identification
+
+Codec identification uses two complementary naming systems:
+
+### FFmpeg codec names (RECOMMENDED)
+
+The `AudioCodec` and `VideoCodec` fields use
+[FFmpeg codec names](https://www.ffmpeg.org/ffmpeg-codecs.html) as the RECOMMENDED
+convention. These names are the de facto standard in scientific computing and can be
+auto-extracted from media files using:
+
+```bash
+ffprobe -v quiet -print_format json -show_streams <file>
+```
+
+### RFC 6381 codec strings (OPTIONAL)
+
+The `AudioCodecRFC6381` and `VideoCodecRFC6381` fields use
+[RFC 6381](https://datatracker.ietf.org/doc/html/rfc6381) codec strings.
+These provide precise codec profile and level information useful for
+web and broadcast interoperability.
+
+### Common codec reference
+
+| Codec          | FFmpeg Name | RFC 6381 String    | Notes                   |
+| -------------- | ----------- | ------------------ | ----------------------- |
+| H.264 / AVC    | `h264`      | `avc1.640028`      | Most widely supported   |
+| H.265 / HEVC   | `hevc`      | `hev1.1.6.L93.B0`  | High efficiency         |
+| VP9            | `vp9`       | `vp09.00.10.08`    | Open, royalty-free      |
+| AV1            | `av1`       | `av01.0.01M.08`    | Next-gen open codec     |
+| AAC-LC         | `aac`       | `mp4a.40.2`        | Default audio for MP4   |
+| MP3            | `mp3`       | `mp4a.6B`          | Legacy lossy audio      |
+| Opus           | `opus`      | `Opus`             | Open, low-latency audio |
+| FLAC           | `flac`      | `fLaC`             | Open lossless audio     |
+| PCM 16-bit LE  | `pcm_s16le` | —                  | Uncompressed (WAV)      |
+
+The FFmpeg name column shows the value to use for `VideoCodec` or `AudioCodec`.
+The RFC 6381 column shows the value for `VideoCodecRFC6381` or `AudioCodecRFC6381`.
+RFC 6381 strings vary by profile and level;
+the values shown are representative examples.
+
+## Privacy Considerations
+
+Media files — particularly audio and video recordings — may contain
+personally identifiable information (PII), including but not limited to:
+
+-   Voices and speech content
+-   Facial features and other physical characteristics
+-   Background environments that could identify locations
+-   Metadata embedded in file headers (for example, GPS coordinates, device identifiers)
+
+Researchers MUST ensure that sharing of media files complies with the
+informed consent obtained from participants and with applicable privacy regulations.
+De-identification techniques (for example, voice distortion, face blurring,
+metadata stripping) SHOULD be applied where appropriate before data sharing.
+
+## Example
+
+A complete sidecar JSON file for an audio-video recording:
+
+```json
+{
+    "RecordingDuration": 312.5,
+    "VideoCodec": "h264",
+    "VideoCodecRFC6381": "avc1.640028",
+    "VideoFrameRate": 30,
+    "VideoFrameCount": 9375,
+    "ImageWidth": 1920,
+    "ImageHeight": 1080,
+    "ImagePixelFormat": "yuv420p",
+    "AudioCodec": "aac",
+    "AudioCodecRFC6381": "mp4a.40.2",
+    "AudioSampleRate": 48000,
+    "AudioChannelCount": 2
+}
+```

src/schema/objects/extensions.yaml

@@ -1,12 +1,24 @@
 ---
 # This file describes valid file extensions in the specification.
+aac:
+  value: .aac
+  display_name: Advanced Audio Coding
+  description: |
+    An [Advanced Audio Coding](https://en.wikipedia.org/wiki/Advanced_Audio_Coding)
+    audio file.
 ave:
   value: .ave
   display_name: AVE # not sure what ave stands for
   description: |
     File containing data averaged by segments of interest.
 
     Used by KIT, Yokogawa, and Ricoh MEG systems.
+avi:
+  value: .avi
+  display_name: Audio Video Interleave
+  description: |
+    An [Audio Video Interleave](https://en.wikipedia.org/wiki/Audio_Video_Interleave)
+    media container file.
 bdf:
   value: .bdf
   display_name: Biosemi Data Format
@@ -153,6 +165,22 @@ md:
   display_name: Markdown
   description: |
     A Markdown file.
+mkv:
+  value: .mkv
+  display_name: Matroska Video
+  description: |
+    A [Matroska](https://www.matroska.org/) media container file.
+mp3:
+  value: .mp3
+  display_name: MP3 Audio
+  description: |
+    An [MP3](https://en.wikipedia.org/wiki/MP3) audio file.
+mp4:
+  value: .mp4
+  display_name: MPEG-4 Part 14
+  description: |
+    An [MPEG-4 Part 14](https://en.wikipedia.org/wiki/MP4_file_format)
+    media container file.
 mefd:
   value: .mefd/
   display_name: Multiscale Electrophysiology File Format Version 3.0
@@ -201,6 +229,12 @@ nwb:
     A [Neurodata Without Borders](https://nwb-schema.readthedocs.io/en/latest/) file.
 
     Each recording consists of a single `.nwb` file.
+ogg:
+  value: .ogg
+  display_name: Ogg Vorbis
+  description: |
+    An [Ogg](https://en.wikipedia.org/wiki/Ogg) audio file,
+    typically containing Vorbis-encoded audio.
 OMEBigTiff:
   value: .ome.btf
   display_name: Open Microscopy Environment BigTIFF
@@ -249,6 +283,11 @@ snirf:
   display_name: Shared Near Infrared Spectroscopy Format
   description: |
     HDF5 file organized according to the [SNIRF specification](https://github.com/fNIRS/snirf)
+svg:
+  value: .svg
+  display_name: Scalable Vector Graphics
+  description: |
+    A [Scalable Vector Graphics](https://en.wikipedia.org/wiki/SVG) image file.
 sqd:
   value: .sqd
   display_name: SQD
@@ -263,6 +302,12 @@ tif:
   display_name: Tag Image File Format
   description: |
     A [Tag Image File Format](https://en.wikipedia.org/wiki/TIFF) file.
+tiff:
+  value: .tiff
+  display_name: Tag Image File Format
+  description: |
+    A [Tag Image File Format](https://en.wikipedia.org/wiki/TIFF) image file.
+    The `.tiff` extension is the long form of `.tif`.
 trg:
   value: .trg
   display_name: KRISS TRG
@@ -307,6 +352,23 @@ vmrk:
     A text marker file in the
     [BrainVision Core Data Format](https://www.brainproducts.com/support-resources/brainvision-core-data-format-1-0/).
     These files come in three-file sets, including a `.vhdr`, a `.vmrk`, and a `.eeg` file.
+wav:
+  value: .wav
+  display_name: Waveform Audio
+  description: |
+    A [Waveform Audio File Format](https://en.wikipedia.org/wiki/WAV)
+    audio file, typically containing uncompressed PCM audio.
+webm:
+  value: .webm
+  display_name: WebM
+  description: |
+    A [WebM](https://www.webmproject.org/) media container file,
+    typically containing VP8/VP9 video and Vorbis/Opus audio.
+webp:
+  value: .webp
+  display_name: WebP Image
+  description: |
+    A [WebP](https://en.wikipedia.org/wiki/WebP) image file.
 Any:
   value: .*
   display_name: Any Extension