From ee69b4cabdb1794fbbcd4d3f553f508d5ac91b9c Mon Sep 17 00:00:00 2001 From: Ben Dichter Date: Sat, 25 Oct 2025 11:41:03 -0400 Subject: [PATCH 01/44] feat(beh): add audio and video recording support Add comprehensive support for audio and video recordings in behavioral experiments: - Add audio file extensions (mp3, wav) and video file extensions (mp4, mkv, avi) with corresponding _audio and _video suffixes - Document usage of audio/video recordings in beh directory for capturing vocalizations, speech, facial expressions, and body movements - Add metadata schema for audio/video device information and stream properties - Include privacy warnings about personally identifiable information in human subject recordings - Update behavioral experiments title to remove "with no neural recordings" restriction, clarifying data can be stored with or without neural recordings - Add examples for file organization including multi-angle recordings and split files - Define optional entities: task, acquisition, run, recording, split --- .../behavioral-experiments.md | 153 +++++- src/schema/objects/extensions.yaml | 30 + src/schema/objects/metadata.yaml | 516 ++++++++++++++---- src/schema/objects/suffixes.yaml | 15 + src/schema/rules/files/raw/beh.yaml | 39 ++ src/schema/rules/sidecars/beh.yaml | 21 + 6 files changed, 650 insertions(+), 124 deletions(-) diff --git a/src/modality-specific-files/behavioral-experiments.md b/src/modality-specific-files/behavioral-experiments.md index cfb8264ce3..32bc07a7cd 100644 --- a/src/modality-specific-files/behavioral-experiments.md +++ b/src/modality-specific-files/behavioral-experiments.md @@ -1,4 +1,4 @@ -# Behavioral experiments (with no neural recordings) +# Behavioral experiments !!! example "Example datasets" @@ -15,16 +15,12 @@ and a guide for using macros can be found at --> {{ MACROS___make_filename_template("raw", datatypes=["beh"]) }} -In addition to logs from behavioral experiments -performed alongside imaging data acquisitions, -one MAY also include data from experiments -performed with no neural recordings. -The results of those experiments MAY be stored in the `beh` directory -using the same formats for event timing (`_events.tsv`), -metadata (`_events.json`), -physiological (`_physio.tsv.gz`, `_physio.json`) -and other continuous recordings (`_stim.tsv.gz`, `_stim.json`) -as for tasks performed during MRI, electrophysiological or other neural recordings. +Behavioral data MAY be stored in the `beh` directory, +The `beh` directory uses standard formats for event timing (`_events.tsv`), +metadata (`_events.json`), video recordings (`_video.*`), audio recordings (`_audio.*`), +physiological recordings (`_physio.tsv.gz`, `_physio.json`), +and other continuous recordings (`_stim.tsv.gz`, `_stim.json`). + Additionally, events files that do not include the mandatory `onset` and `duration` columns MAY be included, @@ -76,6 +72,141 @@ A guide for using macros can be found at --> {{ MACROS___make_sidecar_table("beh.BEHInstitutionInformation") }} +## Audio and video recordings + +Audio and video recordings of behaving subjects MAY be stored in the `beh` directory +using the `_audio` and `_video` suffixes respectively. +These recordings are typically used to capture vocalizations, speech, facial expressions, +body movements, or other behavioral aspects during experimental tasks or rest periods. + +!!! warning "Privacy and personally identifiable information" + + Audio and video recordings of human subjects often contain personally identifiable + information (PII) such as faces, voices, and other identifying features. + Data curators MUST take special care to ensure compliance with applicable privacy + regulations (such as HIPAA in the United States, GDPR in the European Union, or other + local data protection laws) when handling these recordings. + + These recordings are generally more suitable for internal use or for sharing + non-human subject data, unless appropriate privacy protections are implemented. + +### File formats + +Audio recordings MUST use one of the following extensions: + +- `.mp3` - MPEG Audio Layer III +- `.wav` - Waveform Audio File Format + +Video recordings MUST use one of the following extensions: + +- `.mp4` - MPEG-4 Part 14 +- `.mkv` - Matroska video container +- `.avi` - Audio Video Interleave + +### Entities + +Audio and video files MAY use the following entities: + +- `task` - OPTIONAL for audio and video recordings +- `acquisition` - OPTIONAL, can distinguish different recording setups +- `run` - OPTIONAL, for multiple recordings with identical parameters +- `recording` - OPTIONAL, to differentiate simultaneous recordings from different angles, locations, or devices +- `split` - OPTIONAL, for continuous recordings split into multiple files + +### Examples + +```Text +sub-01/ + beh/ + sub-01_task-rest_video.mp4 + sub-01_task-rest_video.json + sub-01_task-stroop_recording-face_video.mp4 + sub-01_task-stroop_recording-face_video.json + sub-01_task-stroop_recording-room_video.mp4 + sub-01_task-stroop_recording-room_video.json + sub-01_task-vocalization_audio.wav + sub-01_task-vocalization_audio.json +``` + +For continuous recordings split into multiple files: + +```Text +sub-01/ + ses-01/ + beh/ + sub-01_ses-01_task-freeplay_run-01_video/ + split-001.mp4 + split-002.mp4 + split-003.mp4 + sub-01_ses-01_task-freeplay_run-01_video.json +``` + +### Sidecar JSON for audio and video recordings + +The following metadata fields are available for audio and video recordings: + + +{{ MACROS___make_sidecar_table("beh.AudioVideoDevice") }} + +{{ MACROS___make_sidecar_table("beh.AudioVideoStreams") }} + +### Example video sidecar JSON + +For a video file containing both video and audio streams: + +```JSON +{ + "TaskName": "RestingState", + "Device": "Sony FDR-AX53", + "AudioChannelCount": 2, + "AudioSampleRate": 48000, + "FrameRate": 30.0, + "Height": 1920, + "Width": 1080, + "Duration": 600.5 +} +``` + +### Example audio sidecar JSON + +For an audio-only recording: + +```JSON +{ + "TaskName": "Vocalization", + "Device": "Zoom H6 Handy Recorder", + "AudioChannelCount": 2, + "AudioSampleRate": 44100, + "Duration": 300.2 +} +``` + +### Annotations and events + +Behavioral annotations or event markers for audio and video recordings +SHOULD be stored in accompanying `_events.tsv` files following the standard +[events file format](../modality-agnostic-files/events.md). +These events files use the same filename entities as the audio/video file they describe, +but with the `_events` suffix. + +For example: + +```Text +sub-01/ + beh/ + sub-01_task-speech_audio.wav + sub-01_task-speech_audio.json + sub-01_task-speech_events.tsv + sub-01_task-speech_events.json +``` + ## Example `_beh.tsv` ```tsv diff --git a/src/schema/objects/extensions.yaml b/src/schema/objects/extensions.yaml index d482f8849d..64c44bcd91 100644 --- a/src/schema/objects/extensions.yaml +++ b/src/schema/objects/extensions.yaml @@ -1,5 +1,11 @@ --- # This file describes valid file extensions in the specification. +avi: + value: .avi + display_name: Audio Video Interleave + description: | + An [Audio Video Interleave](https://en.wikipedia.org/wiki/Audio_Video_Interleave) video file. + This format is commonly used for behavioral video recordings. ave: value: .ave display_name: AVE # not sure what ave stands for @@ -153,6 +159,24 @@ md: display_name: Markdown description: | A Markdown file. +mkv: + value: .mkv + display_name: Matroska Video + description: | + A [Matroska](https://www.matroska.org/) video container file. + This format is commonly used for behavioral video recordings. +mp3: + value: .mp3 + display_name: MPEG Audio Layer III + description: | + An [MP3](https://en.wikipedia.org/wiki/MP3) audio file. + This format is commonly used for behavioral audio recordings. +mp4: + value: .mp4 + display_name: MPEG-4 Part 14 + description: | + An [MPEG-4 Part 14](https://en.wikipedia.org/wiki/MPEG-4_Part_14) video container file. + This format is commonly used for behavioral video recordings and may contain both video and audio streams. mefd: value: .mefd/ display_name: Multiscale Electrophysiology File Format Version 3.0 @@ -286,6 +310,12 @@ tsv_gz: A gzipped tab-delimited file. This file extension is only used for very large tabular data, such as physiological recordings. For smaller data, the unzipped `.tsv` extension is preferred. +wav: + value: .wav + display_name: Waveform Audio File Format + description: | + A [Waveform Audio File Format](https://en.wikipedia.org/wiki/WAV) audio file. + This format is commonly used for behavioral audio recordings. txt: value: .txt display_name: Text diff --git a/src/schema/objects/metadata.yaml b/src/schema/objects/metadata.yaml index acb7fa4447..85e00930ed 100644 --- a/src/schema/objects/metadata.yaml +++ b/src/schema/objects/metadata.yaml @@ -20,9 +20,8 @@ AcquisitionDuration: name: AcquisitionDuration display_name: Acquisition Duration description: | - Duration (in seconds) of volume acquisition. + Duration (in seconds) of scan acquisition, including all volumes for multi-volume scans. Corresponds to [DICOM Tag 0018, 9073](https://dicomlookup.com/dicomtags/(0018,9073)) `Acquisition Duration`. - This field is mutually exclusive with `"RepetitionTime"`. type: number exclusiveMinimum: 0 unit: s @@ -484,6 +483,42 @@ Code: revision-specific URIs are recommended. type: string format: uri +CodeValue: + name: CodeValue + display_name: Code Value + description: | + An identifier that is unambiguous within the Coding Scheme + denoted by Coding Scheme Designator and Coding Scheme Version. + Corresponds to [DICOM Tag 0008, 0100](https://dicomlookup.com/dicomtags/(0008,0100)) `Code Value`. + type: string +CodeMeaning: + name: CodeMeaning + display_name: Code Meaning + description: | + Text that has meaning to a human and conveys the meaning of the term + Corresponds to [DICOM Tag 0008, 0104](https://dicomlookup.com/dicomtags/(0008,0104)) `Code Meaning`. + type: string +CodingSchemeDesignator: + name: CodingSchemeDesignator + display_name: Coding Scheme Designator + description: | + The identifier of the coding scheme in which the coded entry is defined. + Corresponds to [DICOM Tag 0008, 0102](https://dicomlookup.com/dicomtags/(0008,0102)) + `Coding Scheme Designator`. + type: string +CodingSchemeVersion: + name: CodingSchemeVersion + display_name: Coding Scheme version + description: | + An identifier of the version of the coding scheme if necessary to resolve ambiguity. + Corresponds to [DICOM Tag 0008, 0103](https://dicomlookup.com/dicomtags/(0008,0103)) `Coding Scheme Version`. + type: string +CodeURL: + name: CodeURL + display_name: Code URL + description: URL where the code used to generate the dataset may be found. + type: string + format: uri CogAtlasID: name: CogAtlasID display_name: Cognitive Atlas ID @@ -519,6 +554,44 @@ Columns: type: array items: type: string +Container: + name: Container + display_name: Container + description: | + Used to specify the location and relevant attributes of software container image + used to produce the dataset. Valid keys in this object include `Type`, `Tag` and + [`URI`][uri] with [string][] values. + type: object + recommended: + - Type + - Tag + - URI + properties: + ContainerType: + $ref: objects.metadata.ContainerType + ContainerTag: + $ref: objects.metadata.ContainerTag + ContainerURI: + $ref: objects.metadata.ContainerURI +ContainerTag: + name: ContainerTag + display_name: Container Tag + description: | + Tag of software container image used to produce the dataset + type: string +ContainerType: + name: ContainerType + display_name: Container Type + description: | + Type of software container image used to produce the dataset + type: string +ContainerURI: + name: ContainerURI + display_name: Container URI + description: | + URI for software container image used to produce the dataset + type: string + format: uri ContinuousHeadLocalization: name: ContinuousHeadLocalization display_name: Continuous Head Localization @@ -550,6 +623,22 @@ DCOffsetCorrection: If the method used was subtracting the mean value for each channel, use "mean". type: string +Database: + name: Database + display_name: Database + description: | + [URI](SPEC_ROOT/common-principles.md#uniform-resource-indicator) + of database where the dataset is hosted. + type: string + format: uri +Dataset: + name: Dataset + display_name: Dataset + description: | + [URI](SPEC_ROOT/common-principles.md#uniform-resource-indicator) + where data can be retrieved. + type: string + format: uri DatasetDOI: name: DatasetDOI display_name: DatasetDOI @@ -583,6 +672,7 @@ DatasetType: enum: - $ref: objects.enums.raw.value - $ref: objects.enums.derivative.value + - $ref: objects.enums.study__datasettype.value DecayCorrectionFactor: name: DecayCorrectionFactor display_name: Decay Correction Factor @@ -610,38 +700,20 @@ DeidentificationMethodCodeSequence: type: array items: type: object - recommended_fields: + recommended: - CodeValue - CodeMeaning - CodingSchemeDesignator - CodingSchemeVersion properties: CodeValue: - name: CodeValue - type: string - description: | - An identifier that is unambiguous within the Coding Scheme - denoted by Coding Scheme Designator and Coding Scheme Version. - Corresponds to [DICOM Tag 0008, 0100](https://dicomlookup.com/dicomtags/(0008,0100)) `Code Value`. + $ref: objects.metadata.CodeValue CodeMeaning: - name: CodeMeaning - type: string - description: | - Text that has meaning to a human and conveys the meaning of the term - Corresponds to [DICOM Tag 0008, 0104](https://dicomlookup.com/dicomtags/(0008,0104)) `Code Meaning`. + $ref: objects.metadata.CodeMeaning CodingSchemeDesignator: - name: CodingSchemeDesignator - type: string - description: | - The identifier of the coding scheme in which the coded entry is defined. - Corresponds to [DICOM Tag 0008, 0102](https://dicomlookup.com/dicomtags/(0008,0102)) - `Coding Scheme Designator`. + $ref: objects.metadata.CodingSchemeDesignator CodingSchemeVersion: - name: CodingSchemeVersion - type: string - description: | - An identifier of the version of the coding scheme if necessary to resolve ambiguity. - Corresponds to [DICOM Tag 0008, 0103](https://dicomlookup.com/dicomtags/(0008,0103)) `Coding Scheme Version`. + $ref: objects.metadata.CodingSchemeVersion DelayAfterTrigger: name: DelayAfterTrigger display_name: Delay After Trigger @@ -700,6 +772,13 @@ Description: description: | Free-form natural language description. type: string +Description__GeneratedBy: + name: Description + display_name: Pipeline Description + description: | + Plain-text description of the pipeline or process that generated the outputs. + RECOMMENDED if `Name` is `"Manual"`. + type: string DetectorType: name: DetectorType display_name: Detector Type @@ -715,6 +794,26 @@ DetectorType: - type: string enum: - $ref: objects.enums.mixed.value +Descriptors: + name: Descriptors + display_name: Descriptors + description: | + List of relevant descriptors (for example, journal articles) for dataset + using a valid + [URI](SPEC_ROOT/common-principles.md#uniform-resource-indicator) + when possible. + anyOf: + - type: string + - type: array + items: + type: string +Device: + name: Device + display_name: Device + description: | + Free-form description of the device used to record the data + (for example, `"iPhone 12"`, `"Canon EOS R5"`). + type: string DeviceSerialNumber: name: DeviceSerialNumber display_name: Device Serial Number @@ -805,6 +904,13 @@ DispersionCorrected: Boolean flag specifying whether the blood data have been dispersion-corrected. NOTE: not customary for manual samples, and hence should be set to `false`. type: boolean +DOI: + name: DOI + display_name: DOI + description: | + A Digital Object Identifier (DOI) is a persistent, resolvable name for a digital resource. + (see [URI](SPEC_ROOT/common-principles.md#uniform-resource-indicator)) + type: string DoseCalibrationFactor: name: DoseCalibrationFactor display_name: Dose Calibration Factor @@ -1001,19 +1107,12 @@ EditPulse: type: object additionalProperties: type: object - recommended_fields: [FrequencyOffset, PulseDuration] + recommended: [FrequencyOffset, PulseDuration] properties: FrequencyOffset: - anyOf: - - type: number - unit: ppm - - type: array - items: - type: number - unit: ppm + $ref: objects.metadata.FrequencyOffset PulseDuration: - type: number - unit: ms + $ref: objects.metadata.PulseDuration EditTarget: name: EditTarget display_name: Editing Target @@ -1200,6 +1299,48 @@ FlipAngle: unit: degree exclusiveMinimum: 0 maximum: 360 +Format: + name: Format + display_name: Value format + description: | + Permitted formats for values in the described column. + type: string + # Keep synced with objects.formats + enum: + # Type formats + - string + - number + - integer + - boolean + # Numeric/alphanumeric strings + - index + - label + # Dates/times + - date + - datetime + - time + # Units + - unit + # URIs + - uri + - rrid + - bids_uri + # Paths + - dataset_relative + - file_relative + - participant_relative + - stimuli_relative + # Miscellaneous + - hed_version +FrameAcquisitionDuration: + name: FrameAcquisitionDuration + display_name: Frame Acquisition Duration + description: | + Duration (in seconds) of volume acquisition. + Corresponds to DICOM Tag 0018, 9220 `Frame Acquisition Duration`. + type: number + exclusiveMinimum: 0 + unit: s FrameDuration: name: FrameDuration display_name: Frame Duration @@ -1220,6 +1361,18 @@ FrameTimesStart: items: type: number unit: s +FrequencyOffset: + name: FrequencyOffset + display_name: Frequency Offset + description: | + List or value for frequency offset for spectral editing in ppm. See EditPulse + anyOf: + - type: number + unit: ppm + - type: array + items: + type: number + unit: ppm Funding: name: Funding display_name: Funding @@ -1241,46 +1394,15 @@ GeneratedBy: recommended: [Version] properties: Name: - name: Name - description: | - Name of the pipeline or process that generated the outputs. Use `"Manual"` to - indicate the derivatives were generated by hand, or adjusted manually after an - initial run of an automated pipeline. - type: string + $ref: objects.metadata.Name__GeneratedBy Version: - name: Version - description: Version of the pipeline - type: string + $ref: objects.metadata.Version__GeneratedBy Description: - name: Description - description: | - Plain-text description of the pipeline or process that generated the outputs. - RECOMMENDED if `Name` is `"Manual"`. - type: string + $ref: objects.metadata.Description__GeneratedBy CodeURL: - name: CodeURL - description: URL where the code used to generate the dataset may be found. - type: string - format: uri + $ref: objects.metadata.CodeURL Container: - name: Container - description: | - Used to specify the location and relevant attributes of software container image - used to produce the dataset. Valid keys in this object include `Type`, `Tag` and - [`URI`][uri] with [string][] values. - type: object - recommended: - - Type - - Tag - - URI - properties: - Type: - type: string - Tag: - type: string - URI: - type: string - format: uri + $ref: objects.metadata.Container GeneticLevel: name: GeneticLevel display_name: Genetic Level @@ -1306,31 +1428,11 @@ Genetics: required: [Dataset] properties: Dataset: - name: Dataset - description: | - [URI](SPEC_ROOT/common-principles.md#uniform-resource-indicator) - where data can be retrieved. - type: string - format: uri + $ref: objects.metadata.Dataset Database: - name: Database - description: | - [URI](SPEC_ROOT/common-principles.md#uniform-resource-indicator) - of database where the dataset is hosted. - type: string - format: uri + $ref: objects.metadata.Database Descriptors: - name: Descriptors - description: | - List of relevant descriptors (for example, journal articles) for dataset - using a valid - [URI](SPEC_ROOT/common-principles.md#uniform-resource-indicator) - when possible. - anyOf: - - type: string - - type: array - items: - type: string + $ref: objects.metadata.Descriptors GradientSetType: name: GradientSetType display_name: Gradient Set Type @@ -1728,6 +1830,15 @@ JNTANGChannelCount: Number of joint angle channels. type: integer minimum: 0 +Keywords: + name: Keywords + display_name: Keywords + description: | + A list of keywords that describe the content or subject matter of the dataset. + These may be used to enhance the findability of the dataset on public archives. + type: array + items: + type: string LabelingDistance: name: LabelingDistance display_name: Labeling Distance @@ -1878,10 +1989,9 @@ Levels: - type: object properties: TermURL: - type: string - format: uri + $ref: objects.metadata.TermURL Description: - type: string + $ref: objects.metadata.Description License: name: License display_name: License @@ -1987,6 +2097,54 @@ MEGREFChannelCount: `MEGREFChannelCount` should be set to `0`. type: integer minimum: 0 +# TODO: Change if needs to be broken down into ecephys and icephys +# TODO: Add links to the 2D coordinate system, Pixels and glossary entry. +# TODO: Add enums for the valid values specific to microephys +MicroephysCoordinateSystem: + name: MicroephysCoordinateSystem + display_name: Microephys Coordinate System + description: | + Defines the coordinate system for the microelectrode probes. + See the + [Coordinate Systems Appendix](SPEC_ROOT/appendices/coordinate-systems.md) + for a list of restricted keywords for coordinate systems. + If `"Other"`, provide definition of the coordinate system in + `"MicroephysCoordinateSystemDescription"`. If positions correspond to + pixel indices in a 2D image (of either a volume-rendering, + surface-rendering, operative photo, or operative drawing), + this MUST be "Pixels". For more information, see the section + on 2D coordinate systems. + For a list of valid values for this field, see the associated glossary entry. + anyOf: + # - $ref: objects.enums._MicroephysCoordSys + - $ref: objects.enums._StandardTemplateCoordSys +MicroephysCoordinateSystemDescription: + name: MicroephysCoordinateSystemDescription + display_name: Microephys Coordinate System Description + description: | + Free-form text description of the coordinate system. + May also include a link to a documentation page or paper describing the + system in greater detail. + type: string +MicroephysCoordinateUnits: + name: MicroephysCoordinateUnits + display_name: Microephys Coordinate Units + description: | + Units of the coordinates of `"MicroephysCoordinateSystem"`. + type: string +# TODO: Clarify if units need to be used or enums, if units, define pixels + enum: + - m + - mm + - cm + - pixels +MicroephysCoordinateSystemPhoto: + name: MicroephysCoordinateSystemPhoto + display_name: Microephys Coordinate System Photo + description: | + A link to a photo or drawing of the microelectrode probe system. + type: string + format: uri MISCChannelCount: name: MISCChannelCount display_name: Miscellaneous channel count @@ -2105,6 +2263,12 @@ ManufacturersModelName: description: | Manufacturer's model name of the equipment that produced the measurements. type: string +ManufacturersModelVersion: + name: ManufacturersModelVersion + display_name: Manufacturers Model Version + description: | + Manufacturer's model version of the equipment that produced the measurements. + type: string MatrixCoilMode: name: MatrixCoilMode display_name: Matrix Coil Mode @@ -2136,6 +2300,12 @@ MaxMovement: as measured by the head localization coils (for example, `4.8`). type: number unit: mm +Maximum: + name: Maximum + display_name: Maximum value + description: | + Maximum value a column entry is permitted to have. + type: number MeasurementToolMetadata: name: MeasurementToolMetadata display_name: Measurement Tool Metadata @@ -2147,10 +2317,9 @@ MeasurementToolMetadata: type: object properties: TermURL: - type: string - format: uri + $ref: objects.metadata.TermURL Description: - type: string + $ref: objects.metadata.Description MetaboliteAvail: name: MetaboliteAvail display_name: Metabolite Available @@ -2174,6 +2343,12 @@ MetaboliteRecoveryCorrectionApplied: If `true`, the `hplc_recovery_fractions` column MUST be present in the corresponding `*_blood.tsv` file. type: boolean +Minimum: + name: Minimum + display_name: Minimum value + description: | + Minimum value a column entry is permitted to have. + type: number MiscChannelCount: name: MiscChannelCount display_name: Misc Channel Count @@ -2249,6 +2424,14 @@ Name: description: | Name of the dataset. type: string +Name__GeneratedBy: + name: Name + display_name: Pipeline Name + description: | + Name of the pipeline or process that generated the outputs. Use `"Manual"` to + indicate the derivatives were generated by hand, or adjusted manually after an + initial run of an automated pipeline. + type: string NegativeContrast: name: NegativeContrast display_name: Negative Contrast @@ -2697,6 +2880,13 @@ PromptRate: type: array items: type: number +PulseDuration: + name: PulseDuration + display_name: Pulse Duration + description: | + Pulse duration if spectral editing was applied, in milliseconds + type: number + unit: ms PulseSequenceDetails: name: PulseSequenceDetails display_name: Pulse Sequence Details @@ -2861,6 +3051,12 @@ RecordingDuration: Length of the recording in seconds (for example, `3600`). type: number unit: s +RecordingSetupName: + name: RecordingSetupName + display_name: Recording Setup Name + description: | + Custom name of the recording setup. + type: string RecordingType: name: RecordingType display_name: Recording Type @@ -2983,9 +3179,13 @@ ResonantNucleus: For multi-nuclei experiments such as 1H-[13C] MR, an array can be used: `["1H", "13C"]`. Corresponds to [DICOM Tag 0018, 9100](https://dicomlookup.com/dicomtags/(0018,9100)) `Resonant Nucleus`. - type: array - items: - type: string + anyOf: + - type: string + - type: array + items: + type: string + minItems: 1 + maxItems: 2 RotationOrder: name: RotationOrder display_name: RotationOrder @@ -3410,7 +3610,7 @@ SourceDatasets: name: SourceDatasets display_name: Source Datasets description: | - Used to specify the locations and relevant attributes of all source datasets. + Used to specify the locations and relevant attributes of all source datasets (BIDS or not). Valid keys in each object include `"URL"`, `"DOI"` (see [URI](SPEC_ROOT/common-principles.md#uniform-resource-indicator)), and `"Version"` with @@ -3421,12 +3621,11 @@ SourceDatasets: type: object properties: URL: - type: string - format: uri + $ref: objects.metadata.URL DOI: - type: string + $ref: objects.metadata.DOI Version: - type: string + $ref: objects.metadata.Version__Dataset Sources: name: Sources display_name: Sources @@ -3543,10 +3742,15 @@ SpectrometerFrequency: to the resonant frequency of 1H. For multi-nuclei experiments such as 1H-[13C] MR at 3T, an array can be used: `[127.731, 32.125]`. - type: array - items: - type: number - unit: MHz + anyOf: + - type: number + unit: MHz + - type: array + items: + type: number + unit: MHz + minItems: 1 + maxItems: 2 SpoilingGradientDuration: name: SpoilingGradientDuration display_name: Spoiling Gradient Duration @@ -3652,6 +3856,13 @@ SubjectArtefactDescription: If this field is set to `"n/a"`, it will be interpreted as absence of major source of artifacts except cardiac and blinks. type: string +SupplementarySignals: + name: SupplementarySignals + display_name: Supplementary Signals + description: | + Description of the supplementary signal (additional modalities) recorded + in parallel and are also stored in the data file. + type: string TablePosition: name: TablePosition display_name: Table Position @@ -3681,7 +3892,7 @@ TaskName: description: | Name of the task. No two tasks should have the same name. - The task label included in the filename is derived from this `"TaskName"` field + The task label included in the filename MAY be derived from this `"TaskName"` field by removing all non-alphanumeric or `+` characters (that is, all except those matching `[0-9a-zA-Z+]`), and potentially replacing spaces with `+` to ease readability. For example `"TaskName"` `"faces n-back"` or `"head nodding"` could correspond to task labels @@ -3844,6 +4055,15 @@ Units: (see [Units](SPEC_ROOT/common-principles.md#units)). type: string format: unit +URL: + name: URL + display_name: URL + description: | + A Uniform Resource Locator (URL) is a structured reference to an object that + includes a resource transfer scheme and a location. + (see [URI](SPEC_ROOT/common-principles.md#uniform-resource-indicator)) + type: string + format: uri VascularCrushing: name: VascularCrushing display_name: Vascular Crushing @@ -3867,6 +4087,16 @@ VascularCrushingVENC: items: type: number unit: cm/s +Version__Dataset: + name: Version + display_name: Dataset Version + description: Version of the dataset. + type: string +Version__GeneratedBy: + name: Version + display_name: Pipeline Version + description: Version of the pipeline + type: string VolumeAffineMatrix: name: VolumeAffineMatrix display_name: Volume Affine Matrix @@ -4041,3 +4271,63 @@ iEEGReference: this field should have a general description and the channel specific reference should be defined in the `channels.tsv` file. type: string + +AudioChannelCount: + name: AudioChannelCount + display_name: Audio Channel Count + description: | + Number of audio channels in the recording (for example, `2` for stereo). + type: integer + minimum: 1 + +AudioDuration: + name: AudioDuration + display_name: Audio Duration + description: | + Duration of the audio recording in seconds. + type: number + exclusiveMinimum: 0 + unit: s + +AudioSampleRate: + name: AudioSampleRate + display_name: Audio Sample Rate + description: | + Sample rate of the audio recording in Hertz (for example, `44100`). + type: number + exclusiveMinimum: 0 + unit: Hz + +Duration: + name: Duration + display_name: Duration + description: | + Total duration of the audio or video recording in seconds. + type: number + exclusiveMinimum: 0 + unit: s + +FrameRate: + name: FrameRate + display_name: Frame Rate + description: | + Frame rate of the video recording in frames per second (for example, `30.0`). + type: number + exclusiveMinimum: 0 + unit: Hz + +Height: + name: Height + display_name: Video Height + description: | + Height of the video in pixels (for example, `1920`). + type: integer + minimum: 1 + +Width: + name: Width + display_name: Video Width + description: | + Width of the video in pixels (for example, `1080`). + type: integer + minimum: 1 diff --git a/src/schema/objects/suffixes.yaml b/src/schema/objects/suffixes.yaml index 1e2825ced2..6e6efb66b6 100644 --- a/src/schema/objects/suffixes.yaml +++ b/src/schema/objects/suffixes.yaml @@ -528,6 +528,13 @@ asllabeling: A deidentified screenshot of the planning of the labeling slab/plane with respect to the imaging slab or slices. This screenshot is based on DICOM macro C.8.13.5.14. +audio: + value: audio + display_name: Audio Recording + description: | + Audio recording of a behaving subject. + This may include vocalizations, speech, or environmental sounds. + The audio stream may be from a standalone audio recording or extracted from a video file. beh: value: beh display_name: Behavioral recording @@ -877,3 +884,11 @@ unloc: description: | MRS acquisitions run without localization. This includes signals detected using coil sensitivity only. +video: + value: video + display_name: Video Recording + description: | + Video recording of a behaving subject. + This may include both video and audio streams. + Video recordings are typically used to capture behavior, facial expressions, + or body movements during experimental tasks or rest. diff --git a/src/schema/rules/files/raw/beh.yaml b/src/schema/rules/files/raw/beh.yaml index 6193015295..ea17b69018 100644 --- a/src/schema/rules/files/raw/beh.yaml +++ b/src/schema/rules/files/raw/beh.yaml @@ -14,3 +14,42 @@ noncontinuous: task: required acquisition: optional run: optional + +# Audio recordings +audio: + suffixes: + - audio + extensions: + - .mp3 + - .wav + - .json + datatypes: + - beh + entities: + subject: required + session: optional + task: optional + acquisition: optional + run: optional + recording: optional + split: optional + +# Video recordings +video: + suffixes: + - video + extensions: + - .mp4 + - .mkv + - .avi + - .json + datatypes: + - beh + entities: + subject: required + session: optional + task: optional + acquisition: optional + run: optional + recording: optional + split: optional diff --git a/src/schema/rules/sidecars/beh.yaml b/src/schema/rules/sidecars/beh.yaml index f2d8410914..27fc43e12f 100644 --- a/src/schema/rules/sidecars/beh.yaml +++ b/src/schema/rules/sidecars/beh.yaml @@ -25,3 +25,24 @@ BEHInstitutionInformation: InstitutionName: recommended InstitutionAddress: recommended InstitutionalDepartmentName: recommended + +# Audio and Video metadata +AudioVideoDevice: + selectors: + - datatype == "beh" + - intersects([suffix], ["audio", "video"]) + fields: + Device: optional + DeviceSerialNumber: optional + +AudioVideoStreams: + selectors: + - datatype == "beh" + - intersects([suffix], ["audio", "video"]) + fields: + AudioChannelCount: optional + AudioSampleRate: optional + FrameRate: optional + Height: optional + Width: optional + Duration: optional From ea2e5d15ae841892a4b85b1a1ea888ec45b337f0 Mon Sep 17 00:00:00 2001 From: Ben Dichter Date: Sat, 25 Oct 2025 12:38:27 -0400 Subject: [PATCH 02/44] fix(metadata): remove invalid enum reference for DatasetType --- src/schema/objects/metadata.yaml | 1 - 1 file changed, 1 deletion(-) diff --git a/src/schema/objects/metadata.yaml b/src/schema/objects/metadata.yaml index 85e00930ed..5ce7ab51d4 100644 --- a/src/schema/objects/metadata.yaml +++ b/src/schema/objects/metadata.yaml @@ -672,7 +672,6 @@ DatasetType: enum: - $ref: objects.enums.raw.value - $ref: objects.enums.derivative.value - - $ref: objects.enums.study__datasettype.value DecayCorrectionFactor: name: DecayCorrectionFactor display_name: Decay Correction Factor From ffccafc98a5c01a5ffb35959c51a3f2559e88f49 Mon Sep 17 00:00:00 2001 From: "pre-commit-ci[bot]" <66853113+pre-commit-ci[bot]@users.noreply.github.com> Date: Sat, 25 Oct 2025 16:59:36 +0000 Subject: [PATCH 03/44] [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --- src/modality-specific-files/behavioral-experiments.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/modality-specific-files/behavioral-experiments.md b/src/modality-specific-files/behavioral-experiments.md index d1a1cad65f..327e50347a 100644 --- a/src/modality-specific-files/behavioral-experiments.md +++ b/src/modality-specific-files/behavioral-experiments.md @@ -86,7 +86,7 @@ body movements, or other behavioral aspects during experimental tasks or rest pe Data curators MUST take special care to ensure compliance with applicable privacy regulations (such as HIPAA in the United States, GDPR in the European Union, or other local data protection laws) when handling these recordings. - + These recordings are generally more suitable for internal use or for sharing non-human subject data, unless appropriate privacy protections are implemented. From 4707a3347511c06a1744fc957cd79c1f9295e7d0 Mon Sep 17 00:00:00 2001 From: Ben Dichter Date: Sat, 25 Oct 2025 13:54:54 -0400 Subject: [PATCH 04/44] Apply suggestion from @bendichter --- src/schema/objects/metadata.yaml | 48 -------------------------------- 1 file changed, 48 deletions(-) diff --git a/src/schema/objects/metadata.yaml b/src/schema/objects/metadata.yaml index 85e00930ed..da3660a884 100644 --- a/src/schema/objects/metadata.yaml +++ b/src/schema/objects/metadata.yaml @@ -2097,54 +2097,6 @@ MEGREFChannelCount: `MEGREFChannelCount` should be set to `0`. type: integer minimum: 0 -# TODO: Change if needs to be broken down into ecephys and icephys -# TODO: Add links to the 2D coordinate system, Pixels and glossary entry. -# TODO: Add enums for the valid values specific to microephys -MicroephysCoordinateSystem: - name: MicroephysCoordinateSystem - display_name: Microephys Coordinate System - description: | - Defines the coordinate system for the microelectrode probes. - See the - [Coordinate Systems Appendix](SPEC_ROOT/appendices/coordinate-systems.md) - for a list of restricted keywords for coordinate systems. - If `"Other"`, provide definition of the coordinate system in - `"MicroephysCoordinateSystemDescription"`. If positions correspond to - pixel indices in a 2D image (of either a volume-rendering, - surface-rendering, operative photo, or operative drawing), - this MUST be "Pixels". For more information, see the section - on 2D coordinate systems. - For a list of valid values for this field, see the associated glossary entry. - anyOf: - # - $ref: objects.enums._MicroephysCoordSys - - $ref: objects.enums._StandardTemplateCoordSys -MicroephysCoordinateSystemDescription: - name: MicroephysCoordinateSystemDescription - display_name: Microephys Coordinate System Description - description: | - Free-form text description of the coordinate system. - May also include a link to a documentation page or paper describing the - system in greater detail. - type: string -MicroephysCoordinateUnits: - name: MicroephysCoordinateUnits - display_name: Microephys Coordinate Units - description: | - Units of the coordinates of `"MicroephysCoordinateSystem"`. - type: string -# TODO: Clarify if units need to be used or enums, if units, define pixels - enum: - - m - - mm - - cm - - pixels -MicroephysCoordinateSystemPhoto: - name: MicroephysCoordinateSystemPhoto - display_name: Microephys Coordinate System Photo - description: | - A link to a photo or drawing of the microelectrode probe system. - type: string - format: uri MISCChannelCount: name: MISCChannelCount display_name: Miscellaneous channel count From 38310aacf6f19a46b344ca91604b81dbbb2307e7 Mon Sep 17 00:00:00 2001 From: Ben Dichter Date: Sat, 25 Oct 2025 13:55:22 -0400 Subject: [PATCH 05/44] Apply suggestion from @bendichter --- src/schema/objects/metadata.yaml | 6 ------ 1 file changed, 6 deletions(-) diff --git a/src/schema/objects/metadata.yaml b/src/schema/objects/metadata.yaml index da3660a884..fa13f47da7 100644 --- a/src/schema/objects/metadata.yaml +++ b/src/schema/objects/metadata.yaml @@ -2215,12 +2215,6 @@ ManufacturersModelName: description: | Manufacturer's model name of the equipment that produced the measurements. type: string -ManufacturersModelVersion: - name: ManufacturersModelVersion - display_name: Manufacturers Model Version - description: | - Manufacturer's model version of the equipment that produced the measurements. - type: string MatrixCoilMode: name: MatrixCoilMode display_name: Matrix Coil Mode From fa98d09a9452fc6c3753c610dbf62269be0d8c9c Mon Sep 17 00:00:00 2001 From: Ben Dichter Date: Sat, 25 Oct 2025 13:55:37 -0400 Subject: [PATCH 06/44] Apply suggestion from @bendichter --- src/schema/objects/metadata.yaml | 7 ------- 1 file changed, 7 deletions(-) diff --git a/src/schema/objects/metadata.yaml b/src/schema/objects/metadata.yaml index fa13f47da7..ad89182431 100644 --- a/src/schema/objects/metadata.yaml +++ b/src/schema/objects/metadata.yaml @@ -3802,13 +3802,6 @@ SubjectArtefactDescription: If this field is set to `"n/a"`, it will be interpreted as absence of major source of artifacts except cardiac and blinks. type: string -SupplementarySignals: - name: SupplementarySignals - display_name: Supplementary Signals - description: | - Description of the supplementary signal (additional modalities) recorded - in parallel and are also stored in the data file. - type: string TablePosition: name: TablePosition display_name: Table Position From 88849858ac71271e41a6961bf88b2f78b87d66da Mon Sep 17 00:00:00 2001 From: Ben Dichter Date: Sat, 25 Oct 2025 13:55:57 -0400 Subject: [PATCH 07/44] Apply suggestion from @bendichter --- src/schema/objects/metadata.yaml | 6 ------ 1 file changed, 6 deletions(-) diff --git a/src/schema/objects/metadata.yaml b/src/schema/objects/metadata.yaml index ad89182431..eba58d3809 100644 --- a/src/schema/objects/metadata.yaml +++ b/src/schema/objects/metadata.yaml @@ -2997,12 +2997,6 @@ RecordingDuration: Length of the recording in seconds (for example, `3600`). type: number unit: s -RecordingSetupName: - name: RecordingSetupName - display_name: Recording Setup Name - description: | - Custom name of the recording setup. - type: string RecordingType: name: RecordingType display_name: Recording Type From 827e7d90df0ca23166f3bdc20d714b3ea5a7bf88 Mon Sep 17 00:00:00 2001 From: Ben Dichter Date: Sun, 26 Oct 2025 12:55:37 -0400 Subject: [PATCH 08/44] refactor(beh): update title to 'Behavioral recordings' and use filetree macros - Change section title from 'Behavioral experiments' to 'Behavioral recordings' - Convert file tree examples to use MACROS___make_filetree_example for consistent rendering - Address review comments from @yarikoptic in PR #2231 --- .../behavioral-experiments.md | 88 ++++++++++++------- 1 file changed, 57 insertions(+), 31 deletions(-) diff --git a/src/modality-specific-files/behavioral-experiments.md b/src/modality-specific-files/behavioral-experiments.md index 327e50347a..52c2805cec 100644 --- a/src/modality-specific-files/behavioral-experiments.md +++ b/src/modality-specific-files/behavioral-experiments.md @@ -1,4 +1,4 @@ -# Behavioral experiments +# Behavioral recordings !!! example "Example datasets" @@ -115,31 +115,49 @@ Audio and video files MAY use the following entities: ### Examples -```Text -sub-01/ - beh/ - sub-01_task-rest_video.mp4 - sub-01_task-rest_video.json - sub-01_task-stroop_recording-face_video.mp4 - sub-01_task-stroop_recording-face_video.json - sub-01_task-stroop_recording-room_video.mp4 - sub-01_task-stroop_recording-room_video.json - sub-01_task-vocalization_audio.wav - sub-01_task-vocalization_audio.json -``` + +{{ MACROS___make_filetree_example( + { + "sub-01": { + "beh": { + "sub-01_task-rest_video.mp4": "", + "sub-01_task-rest_video.json": "", + "sub-01_task-stroop_recording-face_video.mp4": "", + "sub-01_task-stroop_recording-face_video.json": "", + "sub-01_task-stroop_recording-room_video.mp4": "", + "sub-01_task-stroop_recording-room_video.json": "", + "sub-01_task-vocalization_audio.wav": "", + "sub-01_task-vocalization_audio.json": "", + }, + }, + } +) }} For continuous recordings split into multiple files: -```Text -sub-01/ - ses-01/ - beh/ - sub-01_ses-01_task-freeplay_run-01_video/ - split-001.mp4 - split-002.mp4 - split-003.mp4 - sub-01_ses-01_task-freeplay_run-01_video.json -``` + +{{ MACROS___make_filetree_example( + { + "sub-01": { + "ses-01": { + "beh": { + "sub-01_ses-01_task-freeplay_run-01_video": { + "split-001.mp4": "", + "split-002.mp4": "", + "split-003.mp4": "", + }, + "sub-01_ses-01_task-freeplay_run-01_video.json": "", + }, + }, + }, + } +) }} ### Sidecar JSON for audio and video recordings @@ -198,14 +216,22 @@ but with the `_events` suffix. For example: -```Text -sub-01/ - beh/ - sub-01_task-speech_audio.wav - sub-01_task-speech_audio.json - sub-01_task-speech_events.tsv - sub-01_task-speech_events.json -``` + +{{ MACROS___make_filetree_example( + { + "sub-01": { + "beh": { + "sub-01_task-speech_audio.wav": "", + "sub-01_task-speech_audio.json": "", + "sub-01_task-speech_events.tsv": "", + "sub-01_task-speech_events.json": "", + }, + }, + } +) }} ## Example `_beh.tsv` From 7cacffdf82a25aad24134a367aed901d440b2590 Mon Sep 17 00:00:00 2001 From: Ben Dichter Date: Sun, 26 Oct 2025 12:56:46 -0400 Subject: [PATCH 09/44] fix(docs): update link to BIDS examples repository for behavioral datasets --- src/modality-specific-files/behavioral-experiments.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/modality-specific-files/behavioral-experiments.md b/src/modality-specific-files/behavioral-experiments.md index 52c2805cec..547ebfaa8a 100644 --- a/src/modality-specific-files/behavioral-experiments.md +++ b/src/modality-specific-files/behavioral-experiments.md @@ -3,7 +3,7 @@ !!! example "Example datasets" Datasets containing behavioral data can be found - in the [BIDS examples repository](https://bids-standard.github.io/bids-examples/#behavioral) + in the [BIDS examples repository](https://bids-website.readthedocs.io/en/latest/datasets/examples.html#behavioral) and can be used as helpful guidance when curating new datasets. {{ MACROS___make_filename_template("raw", datatypes=["beh"]) }} -Behavioral data MAY be stored in the `beh` directory, -The `beh` directory uses standard formats for event timing (`_events.tsv`), -metadata (`_events.json`), video recordings (`_video.*`), audio recordings (`_audio.*`), -physiological recordings (`_physio.tsv.gz`, `_physio.json`), -and other continuous recordings (`_stim.tsv.gz`, `_stim.json`). - -Additionally, events files -that do not include the mandatory `onset` and `duration` columns -MAY be included, +The `beh` directory MAY store behavioral recordings such as audio (`_audio.*`) and video (`_video.*`) recordings, physiological (`_physio.*`) recordings, and other continuous recordings (`_stim.tsv.gz`, `_stim.json`). +Audio and video recordings MAY be of subjects performing tasks, resting-state behavior, or recordings of stimuli being presented to the subject. +The `beh` directory MAY also contain event timing files (`_events.tsv`) and their associated metadata (`_events.json`) for behavioral experiments that do not have corresponding neuroimaging or functional data. + +Additionally, events files that do not include the mandatory `onset` and `duration` columns MAY be included, but MUST be labeled `_beh.tsv` rather than `_events.tsv`. The following OPTIONAL columns are pre-defined for behavioral data files: From 3ee2c6c4fb6a938e613cbaf020b1b02ac92fff13 Mon Sep 17 00:00:00 2001 From: Ben Dichter Date: Tue, 28 Oct 2025 09:39:44 -0400 Subject: [PATCH 11/44] Update src/modality-specific-files/behavioral-experiments.md Co-authored-by: Chris Markiewicz --- src/modality-specific-files/behavioral-experiments.md | 8 +++----- 1 file changed, 3 insertions(+), 5 deletions(-) diff --git a/src/modality-specific-files/behavioral-experiments.md b/src/modality-specific-files/behavioral-experiments.md index f50fa70c88..7c98daca0a 100644 --- a/src/modality-specific-files/behavioral-experiments.md +++ b/src/modality-specific-files/behavioral-experiments.md @@ -143,11 +143,9 @@ A guide for using macros can be found at "sub-01": { "ses-01": { "beh": { - "sub-01_ses-01_task-freeplay_run-01_video": { - "split-001.mp4": "", - "split-002.mp4": "", - "split-003.mp4": "", - }, + "sub-01_ses-01_task-freeplay_run-01_split-001_video.mp4": "", + "sub-01_ses-01_task-freeplay_run-01_split-002_video.mp4": "", + "sub-01_ses-01_task-freeplay_run-01_split-003_video.mp4": "", "sub-01_ses-01_task-freeplay_run-01_video.json": "", }, }, From 826e25a7c740ae110ef5657c169448673bc0fd0a Mon Sep 17 00:00:00 2001 From: Ben Dichter Date: Tue, 28 Oct 2025 09:44:32 -0400 Subject: [PATCH 12/44] Apply suggestion from @bendichter --- src/modality-specific-files/behavioral-experiments.md | 1 + 1 file changed, 1 insertion(+) diff --git a/src/modality-specific-files/behavioral-experiments.md b/src/modality-specific-files/behavioral-experiments.md index 7c98daca0a..2f2c3cd834 100644 --- a/src/modality-specific-files/behavioral-experiments.md +++ b/src/modality-specific-files/behavioral-experiments.md @@ -90,6 +90,7 @@ body movements, or other behavioral aspects during experimental tasks or rest pe Audio recordings MUST use one of the following extensions: +- `.flac` - Free Lossless Audio Codec - `.mp3` - MPEG Audio Layer III - `.wav` - Waveform Audio File Format From 00e3e265732bc82849271c1a5e05f004e5b76416 Mon Sep 17 00:00:00 2001 From: Ben Dichter Date: Tue, 28 Oct 2025 10:16:13 -0400 Subject: [PATCH 13/44] Apply suggestion from @bendichter --- src/schema/rules/files/raw/beh.yaml | 1 + 1 file changed, 1 insertion(+) diff --git a/src/schema/rules/files/raw/beh.yaml b/src/schema/rules/files/raw/beh.yaml index ea17b69018..4d5324d199 100644 --- a/src/schema/rules/files/raw/beh.yaml +++ b/src/schema/rules/files/raw/beh.yaml @@ -22,6 +22,7 @@ audio: extensions: - .mp3 - .wav + - .flac - .json datatypes: - beh From 25f65e1c3ecca2a99efcebd90c5d6ee1a7abc730 Mon Sep 17 00:00:00 2001 From: Ben Dichter Date: Tue, 28 Oct 2025 10:34:46 -0400 Subject: [PATCH 14/44] Add FLAC audio file extension support for behavioral recordings --- src/schema/objects/extensions.yaml | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/src/schema/objects/extensions.yaml b/src/schema/objects/extensions.yaml index 64c44bcd91..22ea426324 100644 --- a/src/schema/objects/extensions.yaml +++ b/src/schema/objects/extensions.yaml @@ -120,6 +120,12 @@ fif: display_name: Functional Imaging File Format description: | An MEG file format used by Neuromag, Elekta, and MEGIN. +flac: + value: .flac + display_name: Free Lossless Audio Codec + description: | + A [FLAC](https://en.wikipedia.org/wiki/FLAC) audio file. + This format is commonly used for behavioral audio recordings. jpg: value: .jpg display_name: Joint Photographic Experts Group Format From bf00bc64e3b9cbbcf5bd71482cf8f36086aa08d3 Mon Sep 17 00:00:00 2001 From: Ben Dichter Date: Tue, 28 Oct 2025 10:47:00 -0400 Subject: [PATCH 15/44] Add StartTime field for audio/video metadata in BIDS schema --- src/schema/objects/metadata.yaml | 16 ++++++++++++++++ src/schema/rules/sidecars/beh.yaml | 1 + 2 files changed, 17 insertions(+) diff --git a/src/schema/objects/metadata.yaml b/src/schema/objects/metadata.yaml index eba58d3809..920a8a7517 100644 --- a/src/schema/objects/metadata.yaml +++ b/src/schema/objects/metadata.yaml @@ -3744,6 +3744,22 @@ StartTime: where `s` reflects whole seconds, and `.000000` reflects OPTIONAL fractional seconds. type: number unit: s +StartTime__audiovideo: + name: StartTime + display_name: Start Time + description: | + Start time in seconds in relation to the start of acquisition of the first + data sample in the corresponding dataset (negative values are allowed). + This data MAY be specified with sub-second precision using the syntax `s[.000000]`, + where `s` reflects whole seconds, and `.000000` reflects OPTIONAL fractional seconds. + For audio/video files where precise temporal alignment is not possible or applicable, + this field MAY be set to `"n/a"`. + anyOf: + - type: number + unit: s + - type: string + enum: + - n/a StationName: name: StationName display_name: Station Name diff --git a/src/schema/rules/sidecars/beh.yaml b/src/schema/rules/sidecars/beh.yaml index 27fc43e12f..c283e70b73 100644 --- a/src/schema/rules/sidecars/beh.yaml +++ b/src/schema/rules/sidecars/beh.yaml @@ -40,6 +40,7 @@ AudioVideoStreams: - datatype == "beh" - intersects([suffix], ["audio", "video"]) fields: + StartTime__audiovideo: required AudioChannelCount: optional AudioSampleRate: optional FrameRate: optional From 540a1e79e69ca157844e597ac27b106b2d43bbd4 Mon Sep 17 00:00:00 2001 From: Ben Dichter Date: Tue, 28 Oct 2025 12:59:10 -0400 Subject: [PATCH 16/44] Revert "Add StartTime field for audio/video metadata in BIDS schema" This reverts commit bf00bc64e3b9cbbcf5bd71482cf8f36086aa08d3. --- src/schema/objects/metadata.yaml | 16 ---------------- src/schema/rules/sidecars/beh.yaml | 1 - 2 files changed, 17 deletions(-) diff --git a/src/schema/objects/metadata.yaml b/src/schema/objects/metadata.yaml index 920a8a7517..eba58d3809 100644 --- a/src/schema/objects/metadata.yaml +++ b/src/schema/objects/metadata.yaml @@ -3744,22 +3744,6 @@ StartTime: where `s` reflects whole seconds, and `.000000` reflects OPTIONAL fractional seconds. type: number unit: s -StartTime__audiovideo: - name: StartTime - display_name: Start Time - description: | - Start time in seconds in relation to the start of acquisition of the first - data sample in the corresponding dataset (negative values are allowed). - This data MAY be specified with sub-second precision using the syntax `s[.000000]`, - where `s` reflects whole seconds, and `.000000` reflects OPTIONAL fractional seconds. - For audio/video files where precise temporal alignment is not possible or applicable, - this field MAY be set to `"n/a"`. - anyOf: - - type: number - unit: s - - type: string - enum: - - n/a StationName: name: StationName display_name: Station Name diff --git a/src/schema/rules/sidecars/beh.yaml b/src/schema/rules/sidecars/beh.yaml index c283e70b73..27fc43e12f 100644 --- a/src/schema/rules/sidecars/beh.yaml +++ b/src/schema/rules/sidecars/beh.yaml @@ -40,7 +40,6 @@ AudioVideoStreams: - datatype == "beh" - intersects([suffix], ["audio", "video"]) fields: - StartTime__audiovideo: required AudioChannelCount: optional AudioSampleRate: optional FrameRate: optional From fc019ae17b254b9b1a9113f8a42ac51706b4021a Mon Sep 17 00:00:00 2001 From: Ben Dichter Date: Tue, 28 Oct 2025 13:01:41 -0400 Subject: [PATCH 17/44] Clarify behavioral recordings section by adding details on simultaneous recordings and timing alignment --- src/modality-specific-files/behavioral-experiments.md | 3 +++ 1 file changed, 3 insertions(+) diff --git a/src/modality-specific-files/behavioral-experiments.md b/src/modality-specific-files/behavioral-experiments.md index 2f2c3cd834..acf565dfe2 100644 --- a/src/modality-specific-files/behavioral-experiments.md +++ b/src/modality-specific-files/behavioral-experiments.md @@ -17,6 +17,9 @@ and a guide for using macros can be found at The `beh` directory MAY store behavioral recordings such as audio (`_audio.*`) and video (`_video.*`) recordings, physiological (`_physio.*`) recordings, and other continuous recordings (`_stim.tsv.gz`, `_stim.json`). Audio and video recordings MAY be of subjects performing tasks, resting-state behavior, or recordings of stimuli being presented to the subject. +Audio/video recordings MAY occur simultaneously with other recordings, such as BOLD or EEG. +Relative timing between files may be determined by consulting the `scans.tsv` file. +If no `scans.tsv` file is present, the alignment is undefined. The `beh` directory MAY also contain event timing files (`_events.tsv`) and their associated metadata (`_events.json`) for behavioral experiments that do not have corresponding neuroimaging or functional data. Additionally, events files that do not include the mandatory `onset` and `duration` columns MAY be included, From d7b127032085600a0a08d8ef2e7d8bdb7c304421 Mon Sep 17 00:00:00 2001 From: Ben Dichter Date: Wed, 29 Oct 2025 14:21:40 -0400 Subject: [PATCH 18/44] correct acquisition entity --- src/modality-specific-files/behavioral-experiments.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/modality-specific-files/behavioral-experiments.md b/src/modality-specific-files/behavioral-experiments.md index acf565dfe2..f4f096746b 100644 --- a/src/modality-specific-files/behavioral-experiments.md +++ b/src/modality-specific-files/behavioral-experiments.md @@ -108,7 +108,7 @@ Video recordings MUST use one of the following extensions: Audio and video files MAY use the following entities: - `task` - OPTIONAL for audio and video recordings -- `acquisition` - OPTIONAL, can distinguish different recording setups +- `acq` - OPTIONAL, can distinguish different recording setups - `run` - OPTIONAL, for multiple recordings with identical parameters - `recording` - OPTIONAL, to differentiate simultaneous recordings from different angles, locations, or devices - `split` - OPTIONAL, for continuous recordings split into multiple files From 4245b5d1b11cd9fbd671e4e66c1c10e0e7261a0f Mon Sep 17 00:00:00 2001 From: Ben Dichter Date: Mon, 3 Nov 2025 11:50:25 -0500 Subject: [PATCH 19/44] feat: add support for Ogg Vorbis audio format in behavioral recordings --- src/modality-specific-files/behavioral-experiments.md | 1 + src/schema/objects/extensions.yaml | 6 ++++++ src/schema/rules/files/raw/beh.yaml | 3 ++- 3 files changed, 9 insertions(+), 1 deletion(-) diff --git a/src/modality-specific-files/behavioral-experiments.md b/src/modality-specific-files/behavioral-experiments.md index f4f096746b..7f869ae852 100644 --- a/src/modality-specific-files/behavioral-experiments.md +++ b/src/modality-specific-files/behavioral-experiments.md @@ -95,6 +95,7 @@ Audio recordings MUST use one of the following extensions: - `.flac` - Free Lossless Audio Codec - `.mp3` - MPEG Audio Layer III +- `.ogg` - Ogg Vorbis - `.wav` - Waveform Audio File Format Video recordings MUST use one of the following extensions: diff --git a/src/schema/objects/extensions.yaml b/src/schema/objects/extensions.yaml index 22ea426324..6824adf224 100644 --- a/src/schema/objects/extensions.yaml +++ b/src/schema/objects/extensions.yaml @@ -231,6 +231,12 @@ nwb: A [Neurodata Without Borders](https://nwb-schema.readthedocs.io/en/latest/) file. Each recording consists of a single `.nwb` file. +ogg: + value: .ogg + display_name: Ogg Vorbis + description: | + An [Ogg Vorbis](https://en.wikipedia.org/wiki/Vorbis) audio file. + This format is commonly used for behavioral audio recordings. OMEBigTiff: value: .ome.btf display_name: Open Microscopy Environment BigTIFF diff --git a/src/schema/rules/files/raw/beh.yaml b/src/schema/rules/files/raw/beh.yaml index 1e143b1b70..cd9e681f4a 100644 --- a/src/schema/rules/files/raw/beh.yaml +++ b/src/schema/rules/files/raw/beh.yaml @@ -21,9 +21,10 @@ audio: suffixes: - audio extensions: + - .flac - .mp3 + - .ogg - .wav - - .flac - .json datatypes: - beh From aead0a455fdf9e7caa77e8ea08bfd404482980ee Mon Sep 17 00:00:00 2001 From: Ben Dichter Date: Mon, 3 Nov 2025 11:53:01 -0500 Subject: [PATCH 20/44] Apply suggestion from @bendichter --- src/schema/objects/extensions.yaml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/schema/objects/extensions.yaml b/src/schema/objects/extensions.yaml index 6824adf224..b7fc5621bb 100644 --- a/src/schema/objects/extensions.yaml +++ b/src/schema/objects/extensions.yaml @@ -176,7 +176,7 @@ mp3: display_name: MPEG Audio Layer III description: | An [MP3](https://en.wikipedia.org/wiki/MP3) audio file. - This format is commonly used for behavioral audio recordings. + This format is commonly used for behavioral audio recordings or auditory stimuli. mp4: value: .mp4 display_name: MPEG-4 Part 14 From 3756616f6144aea195f6f9c2699a75811c7f615d Mon Sep 17 00:00:00 2001 From: Ben Dichter Date: Wed, 3 Dec 2025 14:18:08 -0500 Subject: [PATCH 21/44] change example height value --- src/schema/objects/metadata.yaml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/schema/objects/metadata.yaml b/src/schema/objects/metadata.yaml index eba58d3809..a6e54de941 100644 --- a/src/schema/objects/metadata.yaml +++ b/src/schema/objects/metadata.yaml @@ -4253,7 +4253,7 @@ Height: name: Height display_name: Video Height description: | - Height of the video in pixels (for example, `1920`). + Height of the video in pixels (for example, `1080`). type: integer minimum: 1 From 272516c9172ff1fe799c4dffe084c5031b9e891f Mon Sep 17 00:00:00 2001 From: Ben Dichter Date: Wed, 3 Dec 2025 14:18:22 -0500 Subject: [PATCH 22/44] change example width value --- src/schema/objects/metadata.yaml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/schema/objects/metadata.yaml b/src/schema/objects/metadata.yaml index a6e54de941..14715cf07b 100644 --- a/src/schema/objects/metadata.yaml +++ b/src/schema/objects/metadata.yaml @@ -4261,6 +4261,6 @@ Width: name: Width display_name: Video Width description: | - Width of the video in pixels (for example, `1080`). + Width of the video in pixels (for example, `1920`). type: integer minimum: 1 From 0704e4aa53227d68045d20533461da0dcf6aaf38 Mon Sep 17 00:00:00 2001 From: Ben Dichter Date: Wed, 3 Dec 2025 14:33:28 -0500 Subject: [PATCH 23/44] swap example height and width --- src/modality-specific-files/behavioral-experiments.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/src/modality-specific-files/behavioral-experiments.md b/src/modality-specific-files/behavioral-experiments.md index 7f869ae852..86c405c5a1 100644 --- a/src/modality-specific-files/behavioral-experiments.md +++ b/src/modality-specific-files/behavioral-experiments.md @@ -185,8 +185,8 @@ For a video file containing both video and audio streams: "AudioChannelCount": 2, "AudioSampleRate": 48000, "FrameRate": 30.0, - "Height": 1920, - "Width": 1080, + "Height": 1080, + "Width": 1920, "Duration": 600.5 } ``` From b11d981580ae2493ac964ab3d2bdb457f3bb695a Mon Sep 17 00:00:00 2001 From: Ben Dichter Date: Mon, 15 Dec 2025 10:09:30 -0500 Subject: [PATCH 24/44] feat(schema): add audio bit depth and camera position metadata Introduce `AudioBitDepth` and `CameraPosition` object definitions, and allow them as optional fields in BEH sidecar `AudioVideoStreams` to capture richer audio recording parameters and camera setup context. --- src/schema/objects/metadata.yaml | 19 +++++++++++++++++++ src/schema/rules/sidecars/beh.yaml | 2 ++ 2 files changed, 21 insertions(+) diff --git a/src/schema/objects/metadata.yaml b/src/schema/objects/metadata.yaml index abe5469c97..73f5102e60 100644 --- a/src/schema/objects/metadata.yaml +++ b/src/schema/objects/metadata.yaml @@ -4400,6 +4400,25 @@ AudioSampleRate: exclusiveMinimum: 0 unit: Hz +AudioBitDepth: + name: AudioBitDepth + display_name: Audio Bit Depth + description: | + Number of bits per sample in the audio recording. + + Common values include `16`, `24`, or `32`. + type: integer + minimum: 1 + +CameraPosition: + name: CameraPosition + display_name: Camera Position + description: | + Free-form description of the camera placement relative to the subject or scene. + + Examples include "front", "profile-left", "ceiling", "room-corner", or "overhead". + type: string + Duration: name: Duration display_name: Duration diff --git a/src/schema/rules/sidecars/beh.yaml b/src/schema/rules/sidecars/beh.yaml index 27fc43e12f..c4abf6be49 100644 --- a/src/schema/rules/sidecars/beh.yaml +++ b/src/schema/rules/sidecars/beh.yaml @@ -42,7 +42,9 @@ AudioVideoStreams: fields: AudioChannelCount: optional AudioSampleRate: optional + AudioBitDepth: optional FrameRate: optional Height: optional Width: optional + CameraPosition: optional Duration: optional From bddeb17054d8b1012cf83b454fb788ec2d688340 Mon Sep 17 00:00:00 2001 From: Ben Dichter Date: Sat, 10 Jan 2026 13:24:37 -0500 Subject: [PATCH 25/44] feat(beh): add `_audiovideo` suffix for combined audio-video recordings - Add new `_audiovideo` suffix for files containing both audio and video streams - Update documentation to distinguish between audio-only, video-only, and combined recordings - Split AudioVideoStreams sidecar table into separate AudioStreams and VideoStreams tables - Add example files and JSON sidecars for audiovideo recordings - Update schema suffixes to include audiovideo definition --- .../behavioral-experiments.md | 45 ++++++++++++++----- src/schema/objects/suffixes.yaml | 6 +++ src/schema/rules/files/raw/beh.yaml | 20 +++++++++ src/schema/rules/sidecars/beh.yaml | 14 ++++-- 4 files changed, 71 insertions(+), 14 deletions(-) diff --git a/src/modality-specific-files/behavioral-experiments.md b/src/modality-specific-files/behavioral-experiments.md index 86c405c5a1..e328a78270 100644 --- a/src/modality-specific-files/behavioral-experiments.md +++ b/src/modality-specific-files/behavioral-experiments.md @@ -15,8 +15,8 @@ and a guide for using macros can be found at --> {{ MACROS___make_filename_template("raw", datatypes=["beh"]) }} -The `beh` directory MAY store behavioral recordings such as audio (`_audio.*`) and video (`_video.*`) recordings, physiological (`_physio.*`) recordings, and other continuous recordings (`_stim.tsv.gz`, `_stim.json`). -Audio and video recordings MAY be of subjects performing tasks, resting-state behavior, or recordings of stimuli being presented to the subject. +The `beh` directory MAY store behavioral recordings such as audio (`_audio.*`), video (`_video.*`), and combined audio-video (`_audiovideo.*`) recordings, physiological (`_physio.*`) recordings, and other continuous recordings (`_stim.tsv.gz`, `_stim.json`). +Audio, video, and audio-video recordings MAY be of subjects performing tasks, resting-state behavior, or recordings of stimuli being presented to the subject. Audio/video recordings MAY occur simultaneously with other recordings, such as BOLD or EEG. Relative timing between files may be determined by consulting the `scans.tsv` file. If no `scans.tsv` file is present, the alignment is undefined. @@ -71,10 +71,12 @@ A guide for using macros can be found at --> {{ MACROS___make_sidecar_table("beh.BEHInstitutionInformation") }} -## Audio and video recordings +## Audio, video, and audio-video recordings Audio and video recordings of behaving subjects MAY be stored in the `beh` directory -using the `_audio` and `_video` suffixes respectively. +using the `_audio`, `_video`, and `_audiovideo` suffixes. +The `_audio` suffix is for audio-only recordings, `_video` for video-only recordings, +and `_audiovideo` for recordings that contain both audio and video streams. These recordings are typically used to capture vocalizations, speech, facial expressions, body movements, or other behavioral aspects during experimental tasks or rest periods. @@ -98,7 +100,7 @@ Audio recordings MUST use one of the following extensions: - `.ogg` - Ogg Vorbis - `.wav` - Waveform Audio File Format -Video recordings MUST use one of the following extensions: +Video (and audio-video) recordings MUST use one of the following extensions: - `.mp4` - MPEG-4 Part 14 - `.mkv` - Matroska video container @@ -126,6 +128,8 @@ A guide for using macros can be found at "beh": { "sub-01_task-rest_video.mp4": "", "sub-01_task-rest_video.json": "", + "sub-01_task-interview_audiovideo.mp4": "", + "sub-01_task-interview_audiovideo.json": "", "sub-01_task-stroop_recording-face_video.mp4": "", "sub-01_task-stroop_recording-face_video.json": "", "sub-01_task-stroop_recording-room_video.mp4": "", @@ -158,9 +162,9 @@ A guide for using macros can be found at } ) }} -### Sidecar JSON for audio and video recordings +### Sidecar JSON for audio, video, and audio-video recordings -The following metadata fields are available for audio and video recordings: +The following metadata fields are available for audio, video, and audio-video recordings: {{ MACROS___make_sidecar_table("beh.AudioVideoDevice") }} -{{ MACROS___make_sidecar_table("beh.AudioVideoStreams") }} +The following fields are available for audio recordings (`_audio`) and audio-video recordings (`_audiovideo`): -### Example video sidecar JSON +{{ MACROS___make_sidecar_table("beh.AudioStreams") }} + +The following fields are available for video recordings (`_video`) and audio-video recordings (`_audiovideo`): + +{{ MACROS___make_sidecar_table("beh.VideoStreams") }} -For a video file containing both video and audio streams: +### Example audio-video sidecar JSON + +For an audio-video file containing both video and audio streams: ```JSON { @@ -191,6 +201,21 @@ For a video file containing both video and audio streams: } ``` +### Example video sidecar JSON + +For a video-only recording: + +```JSON +{ + "TaskName": "RestingState", + "Device": "Sony FDR-AX53", + "FrameRate": 30.0, + "Height": 1080, + "Width": 1920, + "Duration": 600.5 +} +``` + ### Example audio sidecar JSON For an audio-only recording: diff --git a/src/schema/objects/suffixes.yaml b/src/schema/objects/suffixes.yaml index 75f3b6b41b..099bd95c04 100644 --- a/src/schema/objects/suffixes.yaml +++ b/src/schema/objects/suffixes.yaml @@ -535,6 +535,12 @@ audio: Audio recording of a behaving subject. This may include vocalizations, speech, or environmental sounds. The audio stream may be from a standalone audio recording or extracted from a video file. +audiovideo: + value: audiovideo + display_name: Audio-Video Recording + description: | + Combined audio and video recording of a behaving subject. + This suffix is used for files that contain both audio and video streams. beh: value: beh display_name: Behavioral recording diff --git a/src/schema/rules/files/raw/beh.yaml b/src/schema/rules/files/raw/beh.yaml index cd9e681f4a..a033cc47b3 100644 --- a/src/schema/rules/files/raw/beh.yaml +++ b/src/schema/rules/files/raw/beh.yaml @@ -56,3 +56,23 @@ video: run: optional recording: optional split: optional + +# Combined audio-video recordings +audiovideo: + suffixes: + - audiovideo + extensions: + - .mp4 + - .mkv + - .avi + - .json + datatypes: + - beh + entities: + subject: required + session: optional + task: optional + acquisition: optional + run: optional + recording: optional + split: optional diff --git a/src/schema/rules/sidecars/beh.yaml b/src/schema/rules/sidecars/beh.yaml index c4abf6be49..ba9b9993eb 100644 --- a/src/schema/rules/sidecars/beh.yaml +++ b/src/schema/rules/sidecars/beh.yaml @@ -30,21 +30,27 @@ BEHInstitutionInformation: AudioVideoDevice: selectors: - datatype == "beh" - - intersects([suffix], ["audio", "video"]) + - intersects([suffix], ["audio", "video", "audiovideo"]) fields: Device: optional DeviceSerialNumber: optional + Duration: optional -AudioVideoStreams: +AudioStreams: selectors: - datatype == "beh" - - intersects([suffix], ["audio", "video"]) + - intersects([suffix], ["audio", "audiovideo"]) fields: AudioChannelCount: optional AudioSampleRate: optional AudioBitDepth: optional + +VideoStreams: + selectors: + - datatype == "beh" + - intersects([suffix], ["video", "audiovideo"]) + fields: FrameRate: optional Height: optional Width: optional CameraPosition: optional - Duration: optional From 7bb98a9439d4d1e6c491a37018bca3b89e890f9f Mon Sep 17 00:00:00 2001 From: Ben Dichter Date: Sat, 10 Jan 2026 17:09:15 -0500 Subject: [PATCH 26/44] feat(beh): add support for still image recordings in behavioral experiments Add `_image` suffix for storing still images captured during behavioral experiments in the `beh` directory. Changes include: - Add `.jpg` and `.png` as supported image file extensions - Document use cases: pose estimation training frames, behavioral setup snapshots, and extracted video frames - Update privacy/PII warnings to include images alongside audio/video - Add ImageProperties sidecar table and example files - Update AudioVideoDevice macro to AudioVideoImageDevice --- .../behavioral-experiments.md | 46 ++++++++++++++++--- src/schema/objects/suffixes.yaml | 7 +++ src/schema/rules/files/raw/beh.yaml | 18 ++++++++ src/schema/rules/sidecars/beh.yaml | 21 +++++++-- 4 files changed, 83 insertions(+), 9 deletions(-) diff --git a/src/modality-specific-files/behavioral-experiments.md b/src/modality-specific-files/behavioral-experiments.md index e328a78270..5ad2801c5a 100644 --- a/src/modality-specific-files/behavioral-experiments.md +++ b/src/modality-specific-files/behavioral-experiments.md @@ -15,8 +15,8 @@ and a guide for using macros can be found at --> {{ MACROS___make_filename_template("raw", datatypes=["beh"]) }} -The `beh` directory MAY store behavioral recordings such as audio (`_audio.*`), video (`_video.*`), and combined audio-video (`_audiovideo.*`) recordings, physiological (`_physio.*`) recordings, and other continuous recordings (`_stim.tsv.gz`, `_stim.json`). -Audio, video, and audio-video recordings MAY be of subjects performing tasks, resting-state behavior, or recordings of stimuli being presented to the subject. +The `beh` directory MAY store behavioral recordings such as audio (`_audio.*`), video (`_video.*`), combined audio-video (`_audiovideo.*`), and still image (`_image.*`) recordings, physiological (`_physio.*`) recordings, and other continuous recordings (`_stim.tsv.gz`, `_stim.json`). +Audio, video, audio-video, and image recordings MAY be of subjects performing tasks, resting-state behavior, or recordings of stimuli being presented to the subject. Audio/video recordings MAY occur simultaneously with other recordings, such as BOLD or EEG. Relative timing between files may be determined by consulting the `scans.tsv` file. If no `scans.tsv` file is present, the alignment is undefined. @@ -71,7 +71,7 @@ A guide for using macros can be found at --> {{ MACROS___make_sidecar_table("beh.BEHInstitutionInformation") }} -## Audio, video, and audio-video recordings +## Audio, video, and audio-video recordings and images Audio and video recordings of behaving subjects MAY be stored in the `beh` directory using the `_audio`, `_video`, and `_audiovideo` suffixes. @@ -80,9 +80,14 @@ and `_audiovideo` for recordings that contain both audio and video streams. These recordings are typically used to capture vocalizations, speech, facial expressions, body movements, or other behavioral aspects during experimental tasks or rest periods. +Still images captured during behavioral experiments MAY be stored in the `beh` directory +using the `_image` suffix. +These images are typically used for training frames for pose estimation, +snapshots of behavioral setups, or individual frames extracted from video recordings. + !!! warning "Privacy and personally identifiable information" - Audio and video recordings of human subjects often contain personally identifiable + Audio and video recordings and images of human subjects often contain personally identifiable information (PII) such as faces, voices, and other identifying features. Data curators MUST take special care to ensure compliance with applicable privacy regulations (such as HIPAA in the United States, GDPR in the European Union, or other @@ -100,12 +105,17 @@ Audio recordings MUST use one of the following extensions: - `.ogg` - Ogg Vorbis - `.wav` - Waveform Audio File Format -Video (and audio-video) recordings MUST use one of the following extensions: +Video and audio-video recordings MUST use one of the following extensions: - `.mp4` - MPEG-4 Part 14 - `.mkv` - Matroska video container - `.avi` - Audio Video Interleave +Image files MUST use one of the following extensions: + +- `.jpg` - JPEG image +- `.png` - Portable Network Graphics + ### Entities Audio and video files MAY use the following entities: @@ -134,6 +144,8 @@ A guide for using macros can be found at "sub-01_task-stroop_recording-face_video.json": "", "sub-01_task-stroop_recording-room_video.mp4": "", "sub-01_task-stroop_recording-room_video.json": "", + "sub-01_task-rest_image.jpg": "", + "sub-01_task-rest_image.json": "", "sub-01_task-vocalization_audio.wav": "", "sub-01_task-vocalization_audio.json": "", }, @@ -174,7 +186,9 @@ The definitions of the fields specified in these tables may be found in A guide for using macros can be found at https://github.com/bids-standard/bids-specification/blob/master/macros_doc.md --> -{{ MACROS___make_sidecar_table("beh.AudioVideoDevice") }} +{{ MACROS___make_sidecar_table("beh.AudioVideoImageDevice") }} + +{{ MACROS___make_sidecar_table("beh.AudioVideoDuration") }} The following fields are available for audio recordings (`_audio`) and audio-video recordings (`_audiovideo`): @@ -184,6 +198,12 @@ The following fields are available for video recordings (`_video`) and audio-vid {{ MACROS___make_sidecar_table("beh.VideoStreams") }} +The following fields are available for image files (`_image`): + +{{ MACROS___make_sidecar_table("beh.AudioVideoImageDevice") }} + +{{ MACROS___make_sidecar_table("beh.ImageProperties") }} + ### Example audio-video sidecar JSON For an audio-video file containing both video and audio streams: @@ -230,6 +250,20 @@ For an audio-only recording: } ``` +### Example image sidecar JSON + +For a still image: + +```JSON +{ + "TaskName": "Reaching", + "Device": "GoPro Hero 10", + "Height": 1080, + "Width": 1920, + "CameraPosition": "overhead" +} +``` + ### Annotations and events Behavioral annotations or event markers for audio and video recordings diff --git a/src/schema/objects/suffixes.yaml b/src/schema/objects/suffixes.yaml index 099bd95c04..174de3da17 100644 --- a/src/schema/objects/suffixes.yaml +++ b/src/schema/objects/suffixes.yaml @@ -541,6 +541,13 @@ audiovideo: description: | Combined audio and video recording of a behaving subject. This suffix is used for files that contain both audio and video streams. +behimage: + value: image + display_name: Behavioral Image + description: | + Still image captured during behavioral experiments. + This may include training frames for pose estimation, snapshots of behavioral setups, + or individual frames extracted from video recordings. beh: value: beh display_name: Behavioral recording diff --git a/src/schema/rules/files/raw/beh.yaml b/src/schema/rules/files/raw/beh.yaml index a033cc47b3..cd645255ed 100644 --- a/src/schema/rules/files/raw/beh.yaml +++ b/src/schema/rules/files/raw/beh.yaml @@ -76,3 +76,21 @@ audiovideo: run: optional recording: optional split: optional + +# Still images +image: + suffixes: + - behimage + extensions: + - .jpg + - .png + - .json + datatypes: + - beh + entities: + subject: required + session: optional + task: optional + acquisition: optional + run: optional + recording: optional diff --git a/src/schema/rules/sidecars/beh.yaml b/src/schema/rules/sidecars/beh.yaml index ba9b9993eb..e20efe5fa1 100644 --- a/src/schema/rules/sidecars/beh.yaml +++ b/src/schema/rules/sidecars/beh.yaml @@ -26,14 +26,20 @@ BEHInstitutionInformation: InstitutionAddress: recommended InstitutionalDepartmentName: recommended -# Audio and Video metadata -AudioVideoDevice: +# Audio, Video, and Image metadata +AudioVideoImageDevice: selectors: - datatype == "beh" - - intersects([suffix], ["audio", "video", "audiovideo"]) + - intersects([suffix], ["audio", "video", "audiovideo", "behimage"]) fields: Device: optional DeviceSerialNumber: optional + +AudioVideoDuration: + selectors: + - datatype == "beh" + - intersects([suffix], ["audio", "video", "audiovideo"]) + fields: Duration: optional AudioStreams: @@ -54,3 +60,12 @@ VideoStreams: Height: optional Width: optional CameraPosition: optional + +ImageProperties: + selectors: + - datatype == "beh" + - suffix == "behimage" + fields: + Height: optional + Width: optional + CameraPosition: optional From 877beebe913e4d8aba89e37de76d4098cf87ab3c Mon Sep 17 00:00:00 2001 From: Ben Dichter Date: Sat, 10 Jan 2026 17:16:07 -0500 Subject: [PATCH 27/44] feat(beh): add optional License field for audio/video/image recordings - Add License field to AudioVideoImageDevice sidecar schema - Update documentation to include images in audio/video section headings - Add note explaining licensing considerations for recordings containing identifiable participant data --- .../behavioral-experiments.md | 12 ++++++++++-- src/schema/rules/sidecars/beh.yaml | 1 + 2 files changed, 11 insertions(+), 2 deletions(-) diff --git a/src/modality-specific-files/behavioral-experiments.md b/src/modality-specific-files/behavioral-experiments.md index 5ad2801c5a..cf8cea09e5 100644 --- a/src/modality-specific-files/behavioral-experiments.md +++ b/src/modality-specific-files/behavioral-experiments.md @@ -174,9 +174,9 @@ A guide for using macros can be found at } ) }} -### Sidecar JSON for audio, video, and audio-video recordings +### Sidecar JSON for audio, video, audio-video recordings, and images -The following metadata fields are available for audio, video, and audio-video recordings: +The following metadata fields are available for audio, video, audio-video recordings, and images: {{ MACROS___make_sidecar_table("beh.AudioVideoImageDevice") }} +!!! note "Licensing for recordings containing participants" + + Audio, video, and image recordings of participants may have different licensing + restrictions than the main dataset due to privacy considerations. The optional + `License` field can be used to specify different terms for individual recordings + that contain identifiable participant data. If not specified, the recording + inherits the license from `dataset_description.json`. + {{ MACROS___make_sidecar_table("beh.AudioVideoDuration") }} The following fields are available for audio recordings (`_audio`) and audio-video recordings (`_audiovideo`): diff --git a/src/schema/rules/sidecars/beh.yaml b/src/schema/rules/sidecars/beh.yaml index e20efe5fa1..545e7646f5 100644 --- a/src/schema/rules/sidecars/beh.yaml +++ b/src/schema/rules/sidecars/beh.yaml @@ -34,6 +34,7 @@ AudioVideoImageDevice: fields: Device: optional DeviceSerialNumber: optional + License: optional AudioVideoDuration: selectors: From cc41d49a1de0f4408381dd4903dd926b9d3cfd02 Mon Sep 17 00:00:00 2001 From: Ben Dichter Date: Mon, 12 Jan 2026 13:22:23 -0500 Subject: [PATCH 28/44] fix(schema): rename behimage suffix to image for consistency --- src/schema/objects/suffixes.yaml | 2 +- src/schema/rules/files/raw/beh.yaml | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/src/schema/objects/suffixes.yaml b/src/schema/objects/suffixes.yaml index 174de3da17..e282331079 100644 --- a/src/schema/objects/suffixes.yaml +++ b/src/schema/objects/suffixes.yaml @@ -541,7 +541,7 @@ audiovideo: description: | Combined audio and video recording of a behaving subject. This suffix is used for files that contain both audio and video streams. -behimage: +image: value: image display_name: Behavioral Image description: | diff --git a/src/schema/rules/files/raw/beh.yaml b/src/schema/rules/files/raw/beh.yaml index cd645255ed..4d96c62b41 100644 --- a/src/schema/rules/files/raw/beh.yaml +++ b/src/schema/rules/files/raw/beh.yaml @@ -80,7 +80,7 @@ audiovideo: # Still images image: suffixes: - - behimage + - image extensions: - .jpg - .png From e9500ed6ee78c3b25ce125a2c8d25b9f477d64b0 Mon Sep 17 00:00:00 2001 From: Yaroslav Halchenko Date: Wed, 18 Mar 2026 16:50:46 -0400 Subject: [PATCH 29/44] Add common media file definitions (suffixes, extensions, metadata, sidecar rules) Introduce shared media file infrastructure for BEP044 (stimuli) and BEP047 (behavioral A/V). Both BEPs need overlapping audio/video/image support, so this extracts the common foundation: - Suffixes: audio, video, audiovideo, image - Extensions: .wav, .mp3, .aac, .ogg, .mp4, .avi, .mkv, .webm, .svg, .webp, .tiff - Metadata: Duration, FrameRate, Width, Height, AudioChannelCount, AudioSampleRate, VideoCodec, AudioCodec, VideoCodecRFC6381, AudioCodecRFC6381 - Sidecar rules (media.yaml): suffix-based rules that auto-apply to any datatype - Appendix (media-files.md): formats, codec identification, privacy, examples Co-Authored-By: Claude Code 2.1.63 / Claude Opus 4.6 --- mkdocs.yml | 1 + src/appendices/media-files.md | 162 +++++++++++++++++++++++++++ src/schema/objects/extensions.yaml | 62 ++++++++++ src/schema/objects/metadata.yaml | 89 +++++++++++++++ src/schema/objects/suffixes.yaml | 24 ++++ src/schema/rules/sidecars/media.yaml | 34 ++++++ 6 files changed, 372 insertions(+) create mode 100644 src/appendices/media-files.md create mode 100644 src/schema/rules/sidecars/media.yaml diff --git a/mkdocs.yml b/mkdocs.yml index c81a99bc26..de7ed1704e 100644 --- a/mkdocs.yml +++ b/mkdocs.yml @@ -47,6 +47,7 @@ nav: - Coordinate systems: appendices/coordinate-systems.md - Quantitative MRI: appendices/qmri.md - Arterial Spin Labeling: appendices/arterial-spin-labeling.md + - Media files: appendices/media-files.md - Cross modality correspondence: appendices/cross-modality-correspondence.md - Changelog: CHANGES.md - The BIDS Website: diff --git a/src/appendices/media-files.md b/src/appendices/media-files.md new file mode 100644 index 0000000000..535cdb2ebd --- /dev/null +++ b/src/appendices/media-files.md @@ -0,0 +1,162 @@ +# Media Files + +## Introduction + +Several BIDS datatypes make use of media files — audio recordings, video recordings, +combined audio-video recordings, and still images. +This appendix defines the common file formats, metadata conventions, +and codec identification schemes shared across all datatypes that use media files. + +Datatypes that incorporate media files (for example, behavioral recordings or stimuli) +define their own file-naming rules, directory placement, and datatype-specific metadata. +The conventions described here apply uniformly to all such datatypes. + +## Supported Formats + +### Audio formats + +| Format | Extension | Description | +|------------------------|-----------|-----------------------------------------------| +| Waveform Audio (WAV) | `.wav` | Uncompressed PCM audio; lossless, large files | +| MP3 | `.mp3` | Lossy compressed audio; widely supported | +| Advanced Audio Coding | `.aac` | Lossy compressed audio; successor to MP3 | +| Ogg Vorbis | `.ogg` | Open lossy compressed audio format | + +### Video container formats + +| Format | Extension | Description | +|------------------------|-----------|------------------------------------------| +| MPEG-4 Part 14 | `.mp4` | Widely supported multimedia container | +| Audio Video Interleave | `.avi` | Legacy multimedia container | +| Matroska | `.mkv` | Open, flexible multimedia container | +| WebM | `.webm` | Open format optimized for web delivery | + +### Image formats + +| Format | Extension | Description | +|---------------------------|-----------|----------------------------------------------| +| JPEG | `.jpg` | Lossy compressed photographic images | +| Portable Network Graphics | `.png` | Lossless compressed images with transparency | +| Scalable Vector Graphics | `.svg` | XML-based vector image format | +| WebP | `.webp` | Modern format supporting lossy and lossless | +| Tag Image File Format | `.tiff` | Lossless format common in scientific imaging | + +When choosing a format, consider the trade-off between file size and data fidelity. +Uncompressed or lossless formats (WAV, PNG, TIFF) preserve full quality +but produce larger files. +Lossy formats (MP3, AAC, JPEG) significantly reduce file size +at the cost of some data loss. + +## Media Stream Metadata + +Media files SHOULD be accompanied by a JSON sidecar file +containing technical metadata about the media streams. +The following metadata fields are defined for media files: + +### Duration + +| Field | Suffix | Requirement Level | +|------------|---------------------------------|-------------------| +| `Duration` | `audio`, `video`, `audiovideo` | RECOMMENDED | + +`Duration` is the total duration of the media file in seconds. +For audio-video files, this is the duration of the longest stream. + +### Audio stream properties + +| Field | Suffix | Requirement Level | +|---------------------|-----------------------|-------------------| +| `AudioCodec` | `audio`, `audiovideo` | RECOMMENDED | +| `AudioSampleRate` | `audio`, `audiovideo` | RECOMMENDED | +| `AudioChannelCount` | `audio`, `audiovideo` | RECOMMENDED | +| `AudioCodecRFC6381` | `audio`, `audiovideo` | OPTIONAL | + +### Visual properties + +| Field | Suffix | Requirement Level | +|----------|-------------------------------------|-------------------| +| `Width` | `video`, `audiovideo`, `image` | RECOMMENDED | +| `Height` | `video`, `audiovideo`, `image` | RECOMMENDED | + +### Video stream properties + +| Field | Suffix | Requirement Level | +|---------------------|-----------------------|-------------------| +| `VideoCodec` | `video`, `audiovideo` | RECOMMENDED | +| `FrameRate` | `video`, `audiovideo` | RECOMMENDED | +| `VideoCodecRFC6381` | `video`, `audiovideo` | OPTIONAL | + +## Codec Identification + +Codec identification uses two complementary naming systems: + +### FFmpeg codec names (RECOMMENDED) + +The `AudioCodec` and `VideoCodec` fields use +[FFmpeg codec names](https://www.ffmpeg.org/ffmpeg-codecs.html) as the RECOMMENDED +convention. These names are the de facto standard in scientific computing and can be +auto-extracted from media files using: + +```bash +ffprobe -v quiet -print_format json -show_streams +``` + +### RFC 6381 codec strings (OPTIONAL) + +The `AudioCodecRFC6381` and `VideoCodecRFC6381` fields use +[RFC 6381](https://datatracker.ietf.org/doc/html/rfc6381) codec strings. +These provide precise codec profile and level information useful for +web and broadcast interoperability. + +### Common codec reference + +| Codec | FFmpeg Name | RFC 6381 String | Notes | +|----------------|-------------|--------------------|-------------------------| +| H.264 / AVC | `h264` | `avc1.640028` | Most widely supported | +| H.265 / HEVC | `hevc` | `hev1.1.6.L93.B0` | High efficiency | +| VP9 | `vp9` | `vp09.00.10.08` | Open, royalty-free | +| AV1 | `av1` | `av01.0.01M.08` | Next-gen open codec | +| AAC-LC | `aac` | `mp4a.40.2` | Default audio for MP4 | +| MP3 | `mp3` | `mp4a.6B` | Legacy lossy audio | +| Opus | `opus` | `Opus` | Open, low-latency audio | +| FLAC | `flac` | `fLaC` | Open lossless audio | +| PCM 16-bit LE | `pcm_s16le` | — | Uncompressed (WAV) | + +The FFmpeg name column shows the value to use for `VideoCodec` or `AudioCodec`. +The RFC 6381 column shows the value for `VideoCodecRFC6381` or `AudioCodecRFC6381`. +RFC 6381 strings vary by profile and level; +the values shown are representative examples. + +## Privacy Considerations + +Media files — particularly audio and video recordings — may contain +personally identifiable information (PII), including but not limited to: + +- Voices and speech content +- Facial features and other physical characteristics +- Background environments that could identify locations +- Metadata embedded in file headers (for example, GPS coordinates, device identifiers) + +Researchers MUST ensure that sharing of media files complies with the +informed consent obtained from participants and with applicable privacy regulations. +De-identification techniques (for example, voice distortion, face blurring, +metadata stripping) SHOULD be applied where appropriate before data sharing. + +## Example + +A complete sidecar JSON file for an audio-video recording: + +```json +{ + "Duration": 312.5, + "VideoCodec": "h264", + "VideoCodecRFC6381": "avc1.640028", + "FrameRate": 30, + "Width": 1920, + "Height": 1080, + "AudioCodec": "aac", + "AudioCodecRFC6381": "mp4a.40.2", + "AudioSampleRate": 48000, + "AudioChannelCount": 2 +} +``` diff --git a/src/schema/objects/extensions.yaml b/src/schema/objects/extensions.yaml index 3c7ef248fa..26ce7a6eb0 100644 --- a/src/schema/objects/extensions.yaml +++ b/src/schema/objects/extensions.yaml @@ -1,5 +1,11 @@ --- # This file describes valid file extensions in the specification. +aac: + value: .aac + display_name: Advanced Audio Coding + description: | + An [Advanced Audio Coding](https://en.wikipedia.org/wiki/Advanced_Audio_Coding) + audio file. ave: value: .ave display_name: AVE # not sure what ave stands for @@ -7,6 +13,12 @@ ave: File containing data averaged by segments of interest. Used by KIT, Yokogawa, and Ricoh MEG systems. +avi: + value: .avi + display_name: Audio Video Interleave + description: | + An [Audio Video Interleave](https://en.wikipedia.org/wiki/Audio_Video_Interleave) + media container file. bdf: value: .bdf display_name: Biosemi Data Format @@ -153,6 +165,22 @@ md: display_name: Markdown description: | A Markdown file. +mkv: + value: .mkv + display_name: Matroska Video + description: | + A [Matroska](https://www.matroska.org/) media container file. +mp3: + value: .mp3 + display_name: MP3 Audio + description: | + An [MP3](https://en.wikipedia.org/wiki/MP3) audio file. +mp4: + value: .mp4 + display_name: MPEG-4 Part 14 + description: | + An [MPEG-4 Part 14](https://en.wikipedia.org/wiki/MP4_file_format) + media container file. mefd: value: .mefd/ display_name: Multiscale Electrophysiology File Format Version 3.0 @@ -201,6 +229,12 @@ nwb: A [Neurodata Without Borders](https://nwb-schema.readthedocs.io/en/latest/) file. Each recording consists of a single `.nwb` file. +ogg: + value: .ogg + display_name: Ogg Vorbis + description: | + An [Ogg](https://en.wikipedia.org/wiki/Ogg) audio file, + typically containing Vorbis-encoded audio. OMEBigTiff: value: .ome.btf display_name: Open Microscopy Environment BigTIFF @@ -249,6 +283,11 @@ snirf: display_name: Shared Near Infrared Spectroscopy Format description: | HDF5 file organized according to the [SNIRF specification](https://github.com/fNIRS/snirf) +svg: + value: .svg + display_name: Scalable Vector Graphics + description: | + A [Scalable Vector Graphics](https://en.wikipedia.org/wiki/SVG) image file. sqd: value: .sqd display_name: SQD @@ -263,6 +302,12 @@ tif: display_name: Tag Image File Format description: | A [Tag Image File Format](https://en.wikipedia.org/wiki/TIFF) file. +tiff: + value: .tiff + display_name: Tag Image File Format + description: | + A [Tag Image File Format](https://en.wikipedia.org/wiki/TIFF) image file. + The `.tiff` extension is the long form of `.tif`. trg: value: .trg display_name: KRISS TRG @@ -307,6 +352,23 @@ vmrk: A text marker file in the [BrainVision Core Data Format](https://www.brainproducts.com/support-resources/brainvision-core-data-format-1-0/). These files come in three-file sets, including a `.vhdr`, a `.vmrk`, and a `.eeg` file. +wav: + value: .wav + display_name: Waveform Audio + description: | + A [Waveform Audio File Format](https://en.wikipedia.org/wiki/WAV) + audio file, typically containing uncompressed PCM audio. +webm: + value: .webm + display_name: WebM + description: | + A [WebM](https://www.webmproject.org/) media container file, + typically containing VP8/VP9 video and Vorbis/Opus audio. +webp: + value: .webp + display_name: WebP Image + description: | + A [WebP](https://en.wikipedia.org/wiki/WebP) image file. Any: value: .* display_name: Any Extension diff --git a/src/schema/objects/metadata.yaml b/src/schema/objects/metadata.yaml index f7bdc4defa..b41029d1bb 100644 --- a/src/schema/objects/metadata.yaml +++ b/src/schema/objects/metadata.yaml @@ -237,6 +237,42 @@ AttenuationCorrectionMethodReference: description: | Reference paper for the attenuation correction method used. type: string +AudioChannelCount: + name: AudioChannelCount + display_name: Audio Channel Count + description: | + Number of audio channels in the audio or audio-video file + (for example, `1` for mono, `2` for stereo). + type: integer + minimum: 1 +AudioCodec: + name: AudioCodec + display_name: Audio Codec + description: | + The audio codec used to encode the audio stream, expressed as an + [FFmpeg codec name](https://www.ffmpeg.org/ffmpeg-codecs.html) + (for example, `"aac"`, `"mp3"`, `"opus"`, `"flac"`, `"pcm_s16le"`). + This value can be auto-extracted using + `ffprobe -v quiet -print_format json -show_streams`. + type: string +AudioCodecRFC6381: + name: AudioCodecRFC6381 + display_name: Audio Codec (RFC 6381) + description: | + The audio codec expressed as an + [RFC 6381](https://datatracker.ietf.org/doc/html/rfc6381) codec string + (for example, `"mp4a.40.2"` for AAC-LC). + This representation is useful for web and broadcast interoperability. + type: string +AudioSampleRate: + name: AudioSampleRate + display_name: Audio Sample Rate + description: | + Sampling frequency of the audio stream, in Hz + (for example, `44100`, `48000`, `96000`). + type: number + exclusiveMinimum: 0 + unit: Hz Authors: name: Authors display_name: Authors @@ -1028,6 +1064,15 @@ DoseCalibrationFactor: Corresponds to [DICOM Tag 0054, 1322](http://www.dicomlookup.com/dicomtags/(0054,1322)) `Dose Calibration Factor`. type: number +Duration: + name: Duration + display_name: Duration + description: | + Total duration of the media file, in seconds. + For audio-video files, this is the duration of the longest stream. + type: number + exclusiveMinimum: 0 + unit: s DwellTime: name: DwellTime display_name: Dwell Time @@ -1544,6 +1589,15 @@ FlipAngle: unit: degree exclusiveMinimum: 0 maximum: 360 +FrameRate: + name: FrameRate + display_name: Frame Rate + description: | + The video frame rate of the video stream, in Hz + (for example, `24`, `25`, `29.97`, `30`, `60`). + type: number + exclusiveMinimum: 0 + unit: Hz OnsetSource: name: OnsetSource display_name: Column Name of the Onset Source @@ -1767,6 +1821,14 @@ HardwareFilters: - type: string enum: - n/a +Height: + name: Height + display_name: Height + description: | + Height of the video frame or image, in pixels. + type: integer + minimum: 1 + unit: px HeadCircumference: name: HeadCircumference display_name: Head Circumference @@ -4496,6 +4558,25 @@ VisionCorrection: Equipment used to correct participant vision during an experiment. Example: "spectacles", "lenses", "none". type: string +VideoCodec: + name: VideoCodec + display_name: Video Codec + description: | + The video codec used to encode the video stream, expressed as an + [FFmpeg codec name](https://www.ffmpeg.org/ffmpeg-codecs.html) + (for example, `"h264"`, `"hevc"`, `"vp9"`, `"av1"`). + This value can be auto-extracted using + `ffprobe -v quiet -print_format json -show_streams`. + type: string +VideoCodecRFC6381: + name: VideoCodecRFC6381 + display_name: Video Codec (RFC 6381) + description: | + The video codec expressed as an + [RFC 6381](https://datatracker.ietf.org/doc/html/rfc6381) codec string + (for example, `"avc1.640028"` for H.264 High Profile Level 4.0). + This representation is useful for web and broadcast interoperability. + type: string VolumeTiming: name: VolumeTiming display_name: Volume Timing @@ -4531,6 +4612,14 @@ WholeBloodAvail: If `true`, the `whole_blood_radioactivity` column MUST be present in the corresponding `*_blood.tsv` file. type: boolean +Width: + name: Width + display_name: Width + description: | + Width of the video frame or image, in pixels. + type: integer + minimum: 1 + unit: px WithdrawalRate: name: WithdrawalRate display_name: Withdrawal Rate diff --git a/src/schema/objects/suffixes.yaml b/src/schema/objects/suffixes.yaml index 1fa2fb58f6..ea1029bae7 100644 --- a/src/schema/objects/suffixes.yaml +++ b/src/schema/objects/suffixes.yaml @@ -516,6 +516,18 @@ asl: The complete ASL time series stored as a 4D NIfTI file in the original acquisition order, with possible volume types including: control, label, m0scan, deltam, cbf. +audio: + value: audio + display_name: Audio file + description: | + An audio data file containing one or more audio streams. + Common formats include WAV (uncompressed), MP3, AAC, and Ogg Vorbis. +audiovideo: + value: audiovideo + display_name: Audio-video file + description: | + A media file containing both audio and video streams. + Common containers include MP4, MKV, AVI, and WebM. aslcontext: value: aslcontext display_name: Arterial Spin Labeling Context @@ -666,6 +678,12 @@ ieeg: display_name: Intracranial Electroencephalography description: | Intracranial electroencephalography recording data. +image: + value: image + display_name: Image file + description: | + A still image data file. + Common formats include JPEG, PNG, SVG, WebP, and TIFF. inplaneT1: value: inplaneT1 display_name: Inplane T1 @@ -897,3 +915,9 @@ unloc: description: | MRS acquisitions run without localization. This includes signals detected using coil sensitivity only. +video: + value: video + display_name: Video file + description: | + A video data file containing one or more video streams but no audio. + Common containers include MP4, MKV, AVI, and WebM. diff --git a/src/schema/rules/sidecars/media.yaml b/src/schema/rules/sidecars/media.yaml new file mode 100644 index 0000000000..b58ba4e743 --- /dev/null +++ b/src/schema/rules/sidecars/media.yaml @@ -0,0 +1,34 @@ +# +# Groups of related metadata fields for media files +# + +--- +MediaDuration: + selectors: + - intersects([suffix], ["audio", "video", "audiovideo"]) + fields: + Duration: recommended + +MediaAudioProperties: + selectors: + - intersects([suffix], ["audio", "audiovideo"]) + fields: + AudioCodec: recommended + AudioSampleRate: recommended + AudioChannelCount: recommended + AudioCodecRFC6381: optional + +MediaVisualProperties: + selectors: + - intersects([suffix], ["video", "audiovideo", "image"]) + fields: + Width: recommended + Height: recommended + +MediaVideoProperties: + selectors: + - intersects([suffix], ["video", "audiovideo"]) + fields: + VideoCodec: recommended + FrameRate: recommended + VideoCodecRFC6381: optional From bd553188d3695e298727d918bd8b3f20417afd46 Mon Sep 17 00:00:00 2001 From: Yaroslav Halchenko Date: Thu, 19 Mar 2026 11:33:51 -0400 Subject: [PATCH 30/44] Fix table separator padding for remark-lint compliance Add spaces between pipes and dashes in all separator rows (e.g., `| --- |` instead of `|---|`) to satisfy the remark-lint table-cell-padding rule. Co-Authored-By: Claude Code 2.1.63 / Claude Opus 4.6 --- src/appendices/media-files.md | 16 ++++++++-------- 1 file changed, 8 insertions(+), 8 deletions(-) diff --git a/src/appendices/media-files.md b/src/appendices/media-files.md index 535cdb2ebd..1fb95f245c 100644 --- a/src/appendices/media-files.md +++ b/src/appendices/media-files.md @@ -16,7 +16,7 @@ The conventions described here apply uniformly to all such datatypes. ### Audio formats | Format | Extension | Description | -|------------------------|-----------|-----------------------------------------------| +| ---------------------- | --------- | --------------------------------------------- | | Waveform Audio (WAV) | `.wav` | Uncompressed PCM audio; lossless, large files | | MP3 | `.mp3` | Lossy compressed audio; widely supported | | Advanced Audio Coding | `.aac` | Lossy compressed audio; successor to MP3 | @@ -25,7 +25,7 @@ The conventions described here apply uniformly to all such datatypes. ### Video container formats | Format | Extension | Description | -|------------------------|-----------|------------------------------------------| +| ---------------------- | --------- | ---------------------------------------- | | MPEG-4 Part 14 | `.mp4` | Widely supported multimedia container | | Audio Video Interleave | `.avi` | Legacy multimedia container | | Matroska | `.mkv` | Open, flexible multimedia container | @@ -34,7 +34,7 @@ The conventions described here apply uniformly to all such datatypes. ### Image formats | Format | Extension | Description | -|---------------------------|-----------|----------------------------------------------| +| ------------------------- | --------- | -------------------------------------------- | | JPEG | `.jpg` | Lossy compressed photographic images | | Portable Network Graphics | `.png` | Lossless compressed images with transparency | | Scalable Vector Graphics | `.svg` | XML-based vector image format | @@ -56,7 +56,7 @@ The following metadata fields are defined for media files: ### Duration | Field | Suffix | Requirement Level | -|------------|---------------------------------|-------------------| +| ---------- | ------------------------------- | ----------------- | | `Duration` | `audio`, `video`, `audiovideo` | RECOMMENDED | `Duration` is the total duration of the media file in seconds. @@ -65,7 +65,7 @@ For audio-video files, this is the duration of the longest stream. ### Audio stream properties | Field | Suffix | Requirement Level | -|---------------------|-----------------------|-------------------| +| ------------------- | --------------------- | ----------------- | | `AudioCodec` | `audio`, `audiovideo` | RECOMMENDED | | `AudioSampleRate` | `audio`, `audiovideo` | RECOMMENDED | | `AudioChannelCount` | `audio`, `audiovideo` | RECOMMENDED | @@ -74,14 +74,14 @@ For audio-video files, this is the duration of the longest stream. ### Visual properties | Field | Suffix | Requirement Level | -|----------|-------------------------------------|-------------------| +| -------- | ----------------------------------- | ----------------- | | `Width` | `video`, `audiovideo`, `image` | RECOMMENDED | | `Height` | `video`, `audiovideo`, `image` | RECOMMENDED | ### Video stream properties | Field | Suffix | Requirement Level | -|---------------------|-----------------------|-------------------| +| ------------------- | --------------------- | ----------------- | | `VideoCodec` | `video`, `audiovideo` | RECOMMENDED | | `FrameRate` | `video`, `audiovideo` | RECOMMENDED | | `VideoCodecRFC6381` | `video`, `audiovideo` | OPTIONAL | @@ -111,7 +111,7 @@ web and broadcast interoperability. ### Common codec reference | Codec | FFmpeg Name | RFC 6381 String | Notes | -|----------------|-------------|--------------------|-------------------------| +| -------------- | ----------- | ------------------ | ----------------------- | | H.264 / AVC | `h264` | `avc1.640028` | Most widely supported | | H.265 / HEVC | `hevc` | `hev1.1.6.L93.B0` | High efficiency | | VP9 | `vp9` | `vp09.00.10.08` | Open, royalty-free | From 0a9adddb69d3e47653e940d9cfeb522e82ecf06a Mon Sep 17 00:00:00 2001 From: Yaroslav Halchenko Date: Sat, 21 Mar 2026 11:45:21 -0400 Subject: [PATCH 31/44] Reuse existing RecordingDuration instead of introducing Duration Replace the newly added `Duration` metadata field with the existing `RecordingDuration` field, which already has the same semantics ("length of the recording in seconds") and unit. This avoids introducing a near-duplicate field for media files. Co-Authored-By: Claude Code 2.1.63 / Claude Opus 4.6 --- src/appendices/media-files.md | 13 +++++++------ src/schema/objects/metadata.yaml | 9 --------- src/schema/rules/sidecars/media.yaml | 2 +- 3 files changed, 8 insertions(+), 16 deletions(-) diff --git a/src/appendices/media-files.md b/src/appendices/media-files.md index 1fb95f245c..37f6a90532 100644 --- a/src/appendices/media-files.md +++ b/src/appendices/media-files.md @@ -55,12 +55,13 @@ The following metadata fields are defined for media files: ### Duration -| Field | Suffix | Requirement Level | -| ---------- | ------------------------------- | ----------------- | -| `Duration` | `audio`, `video`, `audiovideo` | RECOMMENDED | +| Field | Suffix | Requirement Level | +| ------------------- | ------------------------------ | ----------------- | +| `RecordingDuration` | `audio`, `video`, `audiovideo` | RECOMMENDED | -`Duration` is the total duration of the media file in seconds. -For audio-video files, this is the duration of the longest stream. +`RecordingDuration` is the total duration of the media file in seconds. +This reuses the existing BIDS metadata field already defined for +electrophysiology recordings (EEG, iEEG, MEG, and others). ### Audio stream properties @@ -148,7 +149,7 @@ A complete sidecar JSON file for an audio-video recording: ```json { - "Duration": 312.5, + "RecordingDuration": 312.5, "VideoCodec": "h264", "VideoCodecRFC6381": "avc1.640028", "FrameRate": 30, diff --git a/src/schema/objects/metadata.yaml b/src/schema/objects/metadata.yaml index b41029d1bb..14dd465823 100644 --- a/src/schema/objects/metadata.yaml +++ b/src/schema/objects/metadata.yaml @@ -1064,15 +1064,6 @@ DoseCalibrationFactor: Corresponds to [DICOM Tag 0054, 1322](http://www.dicomlookup.com/dicomtags/(0054,1322)) `Dose Calibration Factor`. type: number -Duration: - name: Duration - display_name: Duration - description: | - Total duration of the media file, in seconds. - For audio-video files, this is the duration of the longest stream. - type: number - exclusiveMinimum: 0 - unit: s DwellTime: name: DwellTime display_name: Dwell Time diff --git a/src/schema/rules/sidecars/media.yaml b/src/schema/rules/sidecars/media.yaml index b58ba4e743..e188221ee6 100644 --- a/src/schema/rules/sidecars/media.yaml +++ b/src/schema/rules/sidecars/media.yaml @@ -7,7 +7,7 @@ MediaDuration: selectors: - intersects([suffix], ["audio", "video", "audiovideo"]) fields: - Duration: recommended + RecordingDuration: recommended MediaAudioProperties: selectors: From 4faad34d9f6fe7a7f137cafcedb9ec0509ab9bbd Mon Sep 17 00:00:00 2001 From: Yaroslav Halchenko Date: Sat, 21 Mar 2026 11:45:43 -0400 Subject: [PATCH 32/44] Document AudioSampleRate vs SamplingFrequency distinction Add a note in the appendix explaining why AudioSampleRate is used instead of the existing SamplingFrequency: audio-video containers need to distinguish the audio sampling rate from the video frame rate, so the Audio prefix is necessary for multi-stream files. Co-Authored-By: Claude Code 2.1.63 / Claude Opus 4.6 --- src/appendices/media-files.md | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/src/appendices/media-files.md b/src/appendices/media-files.md index 37f6a90532..817eaee755 100644 --- a/src/appendices/media-files.md +++ b/src/appendices/media-files.md @@ -72,6 +72,10 @@ electrophysiology recordings (EEG, iEEG, MEG, and others). | `AudioChannelCount` | `audio`, `audiovideo` | RECOMMENDED | | `AudioCodecRFC6381` | `audio`, `audiovideo` | OPTIONAL | +Note: `AudioSampleRate` is used instead of the existing `SamplingFrequency` field +because audio-video files require distinguishing the audio sampling rate from the +video frame rate. The `Audio` prefix makes this unambiguous in multi-stream containers. + ### Visual properties | Field | Suffix | Requirement Level | From 56be0f60a557710567d177b325c618c9a283bba9 Mon Sep 17 00:00:00 2001 From: Yaroslav Halchenko Date: Sat, 21 Mar 2026 11:46:15 -0400 Subject: [PATCH 33/44] Include .tif alongside .tiff in image formats table The existing photo suffix rules use .tif, so document both .tif and .tiff as valid TIFF extensions for image contexts. This ensures consistency when BEPs define file rules for the image suffix. Co-Authored-By: Claude Code 2.1.63 / Claude Opus 4.6 --- src/appendices/media-files.md | 14 +++++++------- 1 file changed, 7 insertions(+), 7 deletions(-) diff --git a/src/appendices/media-files.md b/src/appendices/media-files.md index 817eaee755..21b65425e2 100644 --- a/src/appendices/media-files.md +++ b/src/appendices/media-files.md @@ -33,13 +33,13 @@ The conventions described here apply uniformly to all such datatypes. ### Image formats -| Format | Extension | Description | -| ------------------------- | --------- | -------------------------------------------- | -| JPEG | `.jpg` | Lossy compressed photographic images | -| Portable Network Graphics | `.png` | Lossless compressed images with transparency | -| Scalable Vector Graphics | `.svg` | XML-based vector image format | -| WebP | `.webp` | Modern format supporting lossy and lossless | -| Tag Image File Format | `.tiff` | Lossless format common in scientific imaging | +| Format | Extension | Description | +| ------------------------- | --------------- | -------------------------------------------- | +| JPEG | `.jpg` | Lossy compressed photographic images | +| Portable Network Graphics | `.png` | Lossless compressed images with transparency | +| Scalable Vector Graphics | `.svg` | XML-based vector image format | +| WebP | `.webp` | Modern format supporting lossy and lossless | +| Tag Image File Format | `.tif`, `.tiff` | Lossless format common in scientific imaging | When choosing a format, consider the trade-off between file size and data fidelity. Uncompressed or lossless formats (WAV, PNG, TIFF) preserve full quality From 8381389207ae92c83892c6f2cf93701d8cd6a3d0 Mon Sep 17 00:00:00 2001 From: Yaroslav Halchenko Date: Sat, 21 Mar 2026 11:46:46 -0400 Subject: [PATCH 34/44] Document relationship between media files and existing photo suffix Add a section explaining that the media file definitions generalize all media in BIDS. The existing photo suffix covers a narrower use case (still images in electrophysiology/microscopy) and predates this framework. A "photo" could equally be a video with narration, an audio description, or a drawing. The media suffixes should be adopted for new datatypes, and a future proposal may deprecate photo in favor of the broader image suffix with migration tooling. Co-Authored-By: Claude Code 2.1.63 / Claude Opus 4.6 --- src/appendices/media-files.md | 17 +++++++++++++++++ 1 file changed, 17 insertions(+) diff --git a/src/appendices/media-files.md b/src/appendices/media-files.md index 21b65425e2..3926c0a4c4 100644 --- a/src/appendices/media-files.md +++ b/src/appendices/media-files.md @@ -11,6 +11,23 @@ Datatypes that incorporate media files (for example, behavioral recordings or st define their own file-naming rules, directory placement, and datatype-specific metadata. The conventions described here apply uniformly to all such datatypes. +### Relationship to the `photo` suffix + +The media file definitions introduced here generalize the concept of all media in BIDS. +The existing `photo` suffix (used for photographs of anatomical landmarks, +head localization coils, and tissue samples) predates this framework and covers +a narrower use case — still images in specific electrophysiology and microscopy datatypes. + +The media suffixes (`audio`, `video`, `audiovideo`, `image`) are intended as the +general-purpose mechanism for all media content in BIDS. +In practice, a "photo" could equally be a video of an experimental setup with verbal +narration, an audio recording describing electrode placement, or a drawing rather than +a photograph. +The media file framework should be generally adopted for new datatypes, +and a future proposal may deprecate the `photo` suffix in favor of the broader `image` +suffix with appropriate migration tooling +(see [bids-utils](https://github.com/bids-standard/bids-utils)). + ## Supported Formats ### Audio formats From 311e335b2757fa212da12510dfecbfc8967d04ea Mon Sep 17 00:00:00 2001 From: Yaroslav Halchenko Date: Mon, 23 Mar 2026 13:09:41 -0400 Subject: [PATCH 35/44] Render media metadata tables from schema using macros Replace hand-written metadata tables with MACROS___make_sidecar_table() calls that pull field names, requirement levels, types, and descriptions directly from the schema (rules/sidecars/media.yaml + objects/metadata.yaml). This eliminates duplication between the appendix prose and the schema, addressing review feedback from @neuromechanist and @effigies. The suffix applicability is noted in prose above each table since the existing macro does not render a "Suffix" column. Format/extension tables remain as manual markdown since no macro exists for that layout. Co-Authored-By: Claude Code 2.1.63 / Claude Opus 4.6 --- src/appendices/media-files.md | 35 ++++++++++++++--------------------- 1 file changed, 14 insertions(+), 21 deletions(-) diff --git a/src/appendices/media-files.md b/src/appendices/media-files.md index 3926c0a4c4..a2170e1c7d 100644 --- a/src/appendices/media-files.md +++ b/src/appendices/media-files.md @@ -68,26 +68,22 @@ at the cost of some data loss. Media files SHOULD be accompanied by a JSON sidecar file containing technical metadata about the media streams. -The following metadata fields are defined for media files: +The following metadata fields are defined for media files. ### Duration -| Field | Suffix | Requirement Level | -| ------------------- | ------------------------------ | ----------------- | -| `RecordingDuration` | `audio`, `video`, `audiovideo` | RECOMMENDED | +Applies to suffixes: `audio`, `video`, `audiovideo`. -`RecordingDuration` is the total duration of the media file in seconds. -This reuses the existing BIDS metadata field already defined for +{{ MACROS___make_sidecar_table("media.MediaDuration") }} + +`RecordingDuration` reuses the existing BIDS metadata field already defined for electrophysiology recordings (EEG, iEEG, MEG, and others). ### Audio stream properties -| Field | Suffix | Requirement Level | -| ------------------- | --------------------- | ----------------- | -| `AudioCodec` | `audio`, `audiovideo` | RECOMMENDED | -| `AudioSampleRate` | `audio`, `audiovideo` | RECOMMENDED | -| `AudioChannelCount` | `audio`, `audiovideo` | RECOMMENDED | -| `AudioCodecRFC6381` | `audio`, `audiovideo` | OPTIONAL | +Applies to suffixes: `audio`, `audiovideo`. + +{{ MACROS___make_sidecar_table("media.MediaAudioProperties") }} Note: `AudioSampleRate` is used instead of the existing `SamplingFrequency` field because audio-video files require distinguishing the audio sampling rate from the @@ -95,18 +91,15 @@ video frame rate. The `Audio` prefix makes this unambiguous in multi-stream cont ### Visual properties -| Field | Suffix | Requirement Level | -| -------- | ----------------------------------- | ----------------- | -| `Width` | `video`, `audiovideo`, `image` | RECOMMENDED | -| `Height` | `video`, `audiovideo`, `image` | RECOMMENDED | +Applies to suffixes: `video`, `audiovideo`, `image`. + +{{ MACROS___make_sidecar_table("media.MediaVisualProperties") }} ### Video stream properties -| Field | Suffix | Requirement Level | -| ------------------- | --------------------- | ----------------- | -| `VideoCodec` | `video`, `audiovideo` | RECOMMENDED | -| `FrameRate` | `video`, `audiovideo` | RECOMMENDED | -| `VideoCodecRFC6381` | `video`, `audiovideo` | OPTIONAL | +Applies to suffixes: `video`, `audiovideo`. + +{{ MACROS___make_sidecar_table("media.MediaVideoProperties") }} ## Codec Identification From 4267efea661288110aa1dd92143a1e86b5d4fa5c Mon Sep 17 00:00:00 2001 From: Yaroslav Halchenko Date: Mon, 23 Mar 2026 13:10:59 -0400 Subject: [PATCH 36/44] Render media suffix definitions from schema using macro Add MACROS___make_suffix_table() call in the introduction to render the audio, video, audiovideo, and image suffix definitions directly from the schema, keeping the appendix in sync with suffixes.yaml. Co-Authored-By: Claude Code 2.1.63 / Claude Opus 4.6 --- src/appendices/media-files.md | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/src/appendices/media-files.md b/src/appendices/media-files.md index a2170e1c7d..43d7eadc72 100644 --- a/src/appendices/media-files.md +++ b/src/appendices/media-files.md @@ -7,6 +7,10 @@ combined audio-video recordings, and still images. This appendix defines the common file formats, metadata conventions, and codec identification schemes shared across all datatypes that use media files. +The following media suffixes are defined: + +{{ MACROS___make_suffix_table(["audio", "video", "audiovideo", "image"]) }} + Datatypes that incorporate media files (for example, behavioral recordings or stimuli) define their own file-naming rules, directory placement, and datatype-specific metadata. The conventions described here apply uniformly to all such datatypes. From 96dca84bb0662698c179f6cd11aae3f922822c1f Mon Sep 17 00:00:00 2001 From: Yaroslav Halchenko Date: Mon, 23 Mar 2026 18:43:14 -0400 Subject: [PATCH 37/44] Add make_extension_table macro and use it for media format tables Add MACROS___make_extension_table() that renders a table of file extensions from the schema (objects/extensions.yaml), with columns for format name, extension (linked to glossary), and description. Replace the 3 hand-written format tables in media-files.md (audio, video, image) with macro calls, eliminating duplication between the appendix prose and extensions.yaml. Other spec files with similar hand-written extension tables (EEG, iEEG, EMG, MEG appendix) can adopt this macro in follow-up PRs. Co-Authored-By: Claude Code 2.1.63 / Claude Opus 4.6 --- src/appendices/media-files.md | 22 ++------------ tools/mkdocs_macros_bids/macros.py | 46 ++++++++++++++++++++++++++++++ tools/mkdocs_macros_bids/main.py | 1 + 3 files changed, 50 insertions(+), 19 deletions(-) diff --git a/src/appendices/media-files.md b/src/appendices/media-files.md index 43d7eadc72..24ff988d9d 100644 --- a/src/appendices/media-files.md +++ b/src/appendices/media-files.md @@ -36,31 +36,15 @@ suffix with appropriate migration tooling ### Audio formats -| Format | Extension | Description | -| ---------------------- | --------- | --------------------------------------------- | -| Waveform Audio (WAV) | `.wav` | Uncompressed PCM audio; lossless, large files | -| MP3 | `.mp3` | Lossy compressed audio; widely supported | -| Advanced Audio Coding | `.aac` | Lossy compressed audio; successor to MP3 | -| Ogg Vorbis | `.ogg` | Open lossy compressed audio format | +{{ MACROS___make_extension_table(["wav", "mp3", "aac", "ogg"]) }} ### Video container formats -| Format | Extension | Description | -| ---------------------- | --------- | ---------------------------------------- | -| MPEG-4 Part 14 | `.mp4` | Widely supported multimedia container | -| Audio Video Interleave | `.avi` | Legacy multimedia container | -| Matroska | `.mkv` | Open, flexible multimedia container | -| WebM | `.webm` | Open format optimized for web delivery | +{{ MACROS___make_extension_table(["mp4", "avi", "mkv", "webm"]) }} ### Image formats -| Format | Extension | Description | -| ------------------------- | --------------- | -------------------------------------------- | -| JPEG | `.jpg` | Lossy compressed photographic images | -| Portable Network Graphics | `.png` | Lossless compressed images with transparency | -| Scalable Vector Graphics | `.svg` | XML-based vector image format | -| WebP | `.webp` | Modern format supporting lossy and lossless | -| Tag Image File Format | `.tif`, `.tiff` | Lossless format common in scientific imaging | +{{ MACROS___make_extension_table(["jpg", "png", "svg", "webp", "tif", "tiff"]) }} When choosing a format, consider the trade-off between file size and data fidelity. Uncompressed or lossless formats (WAV, PNG, TIFF) preserve full quality diff --git a/tools/mkdocs_macros_bids/macros.py b/tools/mkdocs_macros_bids/macros.py index 2e7c2f893e..7738dcf295 100644 --- a/tools/mkdocs_macros_bids/macros.py +++ b/tools/mkdocs_macros_bids/macros.py @@ -203,6 +203,52 @@ def make_suffix_table(suffixes, src_path=None): return table +def make_extension_table(extensions, src_path=None): + """Generate a markdown table of file extension information. + + Parameters + ---------- + extensions : list of str + A list of the extension keys to include in the table. + Keys correspond to entries in the schema's objects.extensions + (for example, ``["wav", "mp3", "aac", "ogg"]``). + src_path : str or None + The file where this macro is called, which may be explicitly provided + by the "page.file.src_path" variable. + + Returns + ------- + table : str + A Markdown-format table containing the extension information. + """ + if src_path is None: + src_path = _get_source_path() + + schema_obj = schema.load_schema() + ext_objects = schema_obj["objects"]["extensions"] + + # Compute the relative path to the glossary from the calling file + src_dir = os.path.dirname(src_path) + glossary_path = os.path.relpath("glossary.md", src_dir) + + rows = [] + for ext_key in extensions: + ext = ext_objects[ext_key] + value = ext["value"] + display_name = ext["display_name"] + # Collapse multi-line description to single line + description = " ".join(ext["description"].strip().split()) + + # Link to glossary anchor + link = f"[{value}]({glossary_path}#objects.extensions.{ext_key})" + + rows.append(f"| {display_name} | {link} | {description} |") + + header = "| **Format** | **Extension** | **Description** |" + separator = "| --- | --- | --- |" + return "\n".join([header, separator] + rows) + + def make_metadata_table(field_info, src_path=None): """Generate a markdown table of metadata field information. diff --git a/tools/mkdocs_macros_bids/main.py b/tools/mkdocs_macros_bids/main.py index 7fa873247a..e4cbd2ba70 100644 --- a/tools/mkdocs_macros_bids/main.py +++ b/tools/mkdocs_macros_bids/main.py @@ -38,6 +38,7 @@ def define_env(env): ) env.macro(macros.make_glossary, "MACROS___make_glossary") env.macro(macros.make_suffix_table, "MACROS___make_suffix_table") + env.macro(macros.make_extension_table, "MACROS___make_extension_table") env.macro(macros.make_metadata_table, "MACROS___make_metadata_table") env.macro(macros.make_json_table, "MACROS___make_json_table") env.macro(macros.make_sidecar_table, "MACROS___make_sidecar_table") From 933b390ff1b893b1ff9fb0a4fb9d402520544c80 Mon Sep 17 00:00:00 2001 From: Yaroslav Halchenko Date: Mon, 23 Mar 2026 21:06:46 -0400 Subject: [PATCH 38/44] Add test for make_extension_table macro Test that the macro correctly renders extension information from the schema, including display names, extension values, glossary links, and proper table structure. Follows the same pattern as existing render table tests. Co-Authored-By: Claude Code 2.1.63 / Claude Opus 4.6 --- .../tests/test_render_tables.py | 44 +++++++++++++++++++ 1 file changed, 44 insertions(+) diff --git a/tools/schemacode/src/bidsschematools/tests/test_render_tables.py b/tools/schemacode/src/bidsschematools/tests/test_render_tables.py index 22676689d0..7cd77ad951 100644 --- a/tools/schemacode/src/bidsschematools/tests/test_render_tables.py +++ b/tools/schemacode/src/bidsschematools/tests/test_render_tables.py @@ -1,8 +1,16 @@ """Tests for the bidsschematools package.""" +import sys +from pathlib import Path + from bidsschematools.render import tables from bidsschematools.render.utils import normalize_requirements +# Make mkdocs_macros_bids importable +_macros_dir = Path(__file__).parents[5] / "tools" / "mkdocs_macros_bids" +if str(_macros_dir) not in sys.path: + sys.path.insert(0, str(_macros_dir)) + def test_make_entity_table(schema_obj): """ @@ -145,3 +153,39 @@ def test_make_columns_table(schema_obj): assert level.upper() in render_row assert level_addendum.split("\n")[0] in render_row assert description_addendum.split("\n")[0] in render_row + + +def test_make_extension_table(schema_obj): + """Test whether expected extensions are present and listed correctly. + + This tests the make_extension_table macro from mkdocs_macros_bids. + """ + import macros as mkdocs_macros # type: ignore[import-not-found] + + target_extensions = ["wav", "mp4", "jpg"] + table = mkdocs_macros.make_extension_table( + target_extensions, + src_path="appendices/media-files.md", + ) + + rendered_lines = table.split("\n") + + # Header and separator + assert rendered_lines[0].startswith("| **Format**") + assert rendered_lines[1].startswith("| ---") + + # One data row per extension + assert len(rendered_lines) == len(target_extensions) + 2 + + # Check each extension is rendered with correct display name and value + expected = { + "wav": (".wav", "Waveform Audio"), + "mp4": (".mp4", "MPEG-4 Part 14"), + "jpg": (".jpg", "Joint Photographic Experts Group"), + } + for ext_key, render_row in zip(target_extensions, rendered_lines[2:]): + value, display_name = expected[ext_key] + assert display_name in render_row + assert value in render_row + # Glossary link + assert f"glossary.md#objects.extensions.{ext_key}" in render_row From be841b797a77aa3e4afd5e7fbeeb07983a1e7fd2 Mon Sep 17 00:00:00 2001 From: Yaroslav Halchenko Date: Wed, 3 Jun 2026 09:11:13 -0400 Subject: [PATCH 39/44] Remove overspecification for "photo" and clarify on variable rate Co-authored-by: Ben Dichter --- src/appendices/media-files.md | 3 --- src/schema/objects/metadata.yaml | 2 +- 2 files changed, 1 insertion(+), 4 deletions(-) diff --git a/src/appendices/media-files.md b/src/appendices/media-files.md index 24ff988d9d..7e830ac747 100644 --- a/src/appendices/media-files.md +++ b/src/appendices/media-files.md @@ -24,9 +24,6 @@ a narrower use case — still images in specific electrophysiology and microscop The media suffixes (`audio`, `video`, `audiovideo`, `image`) are intended as the general-purpose mechanism for all media content in BIDS. -In practice, a "photo" could equally be a video of an experimental setup with verbal -narration, an audio recording describing electrode placement, or a drawing rather than -a photograph. The media file framework should be generally adopted for new datatypes, and a future proposal may deprecate the `photo` suffix in favor of the broader `image` suffix with appropriate migration tooling diff --git a/src/schema/objects/metadata.yaml b/src/schema/objects/metadata.yaml index 14dd465823..480e2ff5f9 100644 --- a/src/schema/objects/metadata.yaml +++ b/src/schema/objects/metadata.yaml @@ -1585,7 +1585,7 @@ FrameRate: display_name: Frame Rate description: | The video frame rate of the video stream, in Hz - (for example, `24`, `25`, `29.97`, `30`, `60`). + (for example, `24`, `25`, `29.97`, `30`, `60`). For variable rate videos, this value should be the nominal frame rate. type: number exclusiveMinimum: 0 unit: Hz From aba8721acc651625e7df4039483bb5fcc34ad022 Mon Sep 17 00:00:00 2001 From: Yaroslav Halchenko Date: Wed, 3 Jun 2026 11:35:03 -0400 Subject: [PATCH 40/44] Clarify Width/Height and add PixelFormat Per PR review discussion: - Width and Height descriptions now explicitly state they correspond to the number of columns and rows in the stored pixel grid as captured, without applying any orientation correction (for example, EXIF Orientation tag). Addresses thread r2989971312 and yarikoptic's follow-up r3349597959. - Add PixelFormat (OPTIONAL) under MediaVideoProperties, using FFmpeg's pix_fmt string. A single value encodes color model, channel count, chroma subsampling, and bit depth, and is auto-extractable via ffprobe. Addresses thread r2989988246 (proposed by @h-mayorquin, refined by @bendichter in r3348715711; OPTIONAL per project convention rather than RECOMMENDED). Co-authored-by: Heberto Mayorquin Co-authored-by: Ben Dichter Co-Authored-By: Claude Code 2.1.161 / Claude Opus 4.7 --- src/schema/objects/metadata.yaml | 19 ++++++++++++++++++- src/schema/rules/sidecars/media.yaml | 1 + 2 files changed, 19 insertions(+), 1 deletion(-) diff --git a/src/schema/objects/metadata.yaml b/src/schema/objects/metadata.yaml index 480e2ff5f9..60d8f2db1a 100644 --- a/src/schema/objects/metadata.yaml +++ b/src/schema/objects/metadata.yaml @@ -1585,7 +1585,8 @@ FrameRate: display_name: Frame Rate description: | The video frame rate of the video stream, in Hz - (for example, `24`, `25`, `29.97`, `30`, `60`). For variable rate videos, this value should be the nominal frame rate. + (for example, `24`, `25`, `29.97`, `30`, `60`). + For variable rate videos, this value should be the nominal frame rate. type: number exclusiveMinimum: 0 unit: Hz @@ -1817,6 +1818,9 @@ Height: display_name: Height description: | Height of the video frame or image, in pixels. + Corresponds to the number of rows in the stored pixel grid as captured, + without applying any orientation correction that may be reported by + container metadata (for example, the EXIF `Orientation` tag). type: integer minimum: 1 unit: px @@ -3091,6 +3095,16 @@ PhysioType: enum: - $ref: objects.enums.PhysioTypeGeneric.value - $ref: objects.enums.PhysioTypeEyetrack.value +PixelFormat: + name: PixelFormat + display_name: Pixel Format + description: | + The pixel format of the video stream, as reported by FFmpeg's `pix_fmt` + field (for example, `"yuv420p"`, `"yuv420p10le"`, `"gray16le"`). + A single `pix_fmt` value encodes the color model, channel count, chroma + subsampling, and bit depth, and can be extracted automatically with + `ffprobe`. + type: string PixelSize: name: PixelSize display_name: Pixel Size @@ -4608,6 +4622,9 @@ Width: display_name: Width description: | Width of the video frame or image, in pixels. + Corresponds to the number of columns in the stored pixel grid as captured, + without applying any orientation correction that may be reported by + container metadata (for example, the EXIF `Orientation` tag). type: integer minimum: 1 unit: px diff --git a/src/schema/rules/sidecars/media.yaml b/src/schema/rules/sidecars/media.yaml index e188221ee6..a54fd04f2b 100644 --- a/src/schema/rules/sidecars/media.yaml +++ b/src/schema/rules/sidecars/media.yaml @@ -31,4 +31,5 @@ MediaVideoProperties: fields: VideoCodec: recommended FrameRate: recommended + PixelFormat: optional VideoCodecRFC6381: optional From a5b7aea7d33f1d17470fe83b4cf4ce5d2f0c775c Mon Sep 17 00:00:00 2001 From: Yaroslav Halchenko Date: Wed, 3 Jun 2026 11:49:55 -0400 Subject: [PATCH 41/44] Add VideoFrameCount; prefix-align FrameRate, Width, Height Following PR review discussion on naming consistency (yarikoptic's analysis in r3349532158 and r3349547943): - Rename FrameRate -> VideoFrameRate to match the existing Video* / Audio* prefix convention in MediaVideoProperties / MediaAudioProperties. - Add VideoFrameCount (RECOMMENDED) under MediaVideoProperties. Required for variable frame rate video where the count cannot be derived from VideoFrameRate * RecordingDuration; useful as an integrity check otherwise. Addresses thread r2989982185. - Rename MediaVisualProperties -> MediaImageProperties and rename Width/Height to ImageWidth/ImageHeight. The Image prefix disambiguates these pixel-grid dimensions from other notions of width/height (for example, physical object sizes in microscopy, field-of-view extents) and aligns with the schema-wide convention of family prefixes for generic terms. Example JSON in the appendix updated accordingly. Co-authored-by: Heberto Mayorquin Co-authored-by: Cody Baker Co-Authored-By: Claude Code 2.1.161 / Claude Opus 4.7 --- src/appendices/media-files.md | 11 ++-- src/schema/objects/metadata.yaml | 75 ++++++++++++++++------------ src/schema/rules/sidecars/media.yaml | 9 ++-- 3 files changed, 54 insertions(+), 41 deletions(-) diff --git a/src/appendices/media-files.md b/src/appendices/media-files.md index 7e830ac747..bda5936aab 100644 --- a/src/appendices/media-files.md +++ b/src/appendices/media-files.md @@ -74,11 +74,11 @@ Note: `AudioSampleRate` is used instead of the existing `SamplingFrequency` fiel because audio-video files require distinguishing the audio sampling rate from the video frame rate. The `Audio` prefix makes this unambiguous in multi-stream containers. -### Visual properties +### Image properties Applies to suffixes: `video`, `audiovideo`, `image`. -{{ MACROS___make_sidecar_table("media.MediaVisualProperties") }} +{{ MACROS___make_sidecar_table("media.MediaImageProperties") }} ### Video stream properties @@ -151,9 +151,10 @@ A complete sidecar JSON file for an audio-video recording: "RecordingDuration": 312.5, "VideoCodec": "h264", "VideoCodecRFC6381": "avc1.640028", - "FrameRate": 30, - "Width": 1920, - "Height": 1080, + "VideoFrameRate": 30, + "VideoFrameCount": 9375, + "ImageWidth": 1920, + "ImageHeight": 1080, "AudioCodec": "aac", "AudioCodecRFC6381": "mp4a.40.2", "AudioSampleRate": 48000, diff --git a/src/schema/objects/metadata.yaml b/src/schema/objects/metadata.yaml index 60d8f2db1a..6705e2d3e2 100644 --- a/src/schema/objects/metadata.yaml +++ b/src/schema/objects/metadata.yaml @@ -1580,16 +1580,6 @@ FlipAngle: unit: degree exclusiveMinimum: 0 maximum: 360 -FrameRate: - name: FrameRate - display_name: Frame Rate - description: | - The video frame rate of the video stream, in Hz - (for example, `24`, `25`, `29.97`, `30`, `60`). - For variable rate videos, this value should be the nominal frame rate. - type: number - exclusiveMinimum: 0 - unit: Hz OnsetSource: name: OnsetSource display_name: Column Name of the Onset Source @@ -1813,17 +1803,6 @@ HardwareFilters: - type: string enum: - n/a -Height: - name: Height - display_name: Height - description: | - Height of the video frame or image, in pixels. - Corresponds to the number of rows in the stored pixel grid as captured, - without applying any orientation correction that may be reported by - container metadata (for example, the EXIF `Orientation` tag). - type: integer - minimum: 1 - unit: px HeadCircumference: name: HeadCircumference display_name: Head Circumference @@ -1942,6 +1921,28 @@ ImageDecayCorrectionTime: `"TimeZero"` in the default unit seconds. type: number unit: s +ImageHeight: + name: ImageHeight + display_name: Image Height + description: | + Height of the video frame or image, in pixels. + Corresponds to the number of rows in the stored pixel grid as captured, + without applying any orientation correction that may be reported by + container metadata (for example, the EXIF `Orientation` tag). + type: integer + minimum: 1 + unit: px +ImageWidth: + name: ImageWidth + display_name: Image Width + description: | + Width of the video frame or image, in pixels. + Corresponds to the number of columns in the stored pixel grid as captured, + without applying any orientation correction that may be reported by + container metadata (for example, the EXIF `Orientation` tag). + type: integer + minimum: 1 + unit: px Immersion: name: Immersion display_name: Immersion @@ -4582,6 +4583,27 @@ VideoCodecRFC6381: (for example, `"avc1.640028"` for H.264 High Profile Level 4.0). This representation is useful for web and broadcast interoperability. type: string +VideoFrameCount: + name: VideoFrameCount + display_name: Video Frame Count + description: | + Total number of frames in the video stream. + For constant frame rate video this can be derived from `VideoFrameRate` + and `RecordingDuration`, but for variable frame rate (VFR) video the + derivation is undefined, so an explicit value is needed. + Also useful as an integrity check to detect truncated or corrupted files. + type: integer + minimum: 1 +VideoFrameRate: + name: VideoFrameRate + display_name: Video Frame Rate + description: | + The video frame rate of the video stream, in Hz + (for example, `24`, `25`, `29.97`, `30`, `60`). + For variable rate videos, this value should be the nominal frame rate. + type: number + exclusiveMinimum: 0 + unit: Hz VolumeTiming: name: VolumeTiming display_name: Volume Timing @@ -4617,17 +4639,6 @@ WholeBloodAvail: If `true`, the `whole_blood_radioactivity` column MUST be present in the corresponding `*_blood.tsv` file. type: boolean -Width: - name: Width - display_name: Width - description: | - Width of the video frame or image, in pixels. - Corresponds to the number of columns in the stored pixel grid as captured, - without applying any orientation correction that may be reported by - container metadata (for example, the EXIF `Orientation` tag). - type: integer - minimum: 1 - unit: px WithdrawalRate: name: WithdrawalRate display_name: Withdrawal Rate diff --git a/src/schema/rules/sidecars/media.yaml b/src/schema/rules/sidecars/media.yaml index a54fd04f2b..710f0d3b66 100644 --- a/src/schema/rules/sidecars/media.yaml +++ b/src/schema/rules/sidecars/media.yaml @@ -18,18 +18,19 @@ MediaAudioProperties: AudioChannelCount: recommended AudioCodecRFC6381: optional -MediaVisualProperties: +MediaImageProperties: selectors: - intersects([suffix], ["video", "audiovideo", "image"]) fields: - Width: recommended - Height: recommended + ImageWidth: recommended + ImageHeight: recommended MediaVideoProperties: selectors: - intersects([suffix], ["video", "audiovideo"]) fields: VideoCodec: recommended - FrameRate: recommended + VideoFrameRate: recommended + VideoFrameCount: recommended PixelFormat: optional VideoCodecRFC6381: optional From e576fea1d5296d2693ab2f7608dced096dd4f3f0 Mon Sep 17 00:00:00 2001 From: Yaroslav Halchenko Date: Wed, 3 Jun 2026 11:54:28 -0400 Subject: [PATCH 42/44] Rename PixelFormat to ImagePixelFormat and move to MediaImageProperties The pixel format (FFmpeg's pix_fmt) applies equally to single images and video frames: ffprobe reports it for both, and the encoded information (color model, channel count, chroma subsampling, bit depth) is the same concept in either case. Move the field out of MediaVideoProperties into MediaImageProperties (which already covers image, video, and audiovideo), and rename with the Image prefix to match the rest of the group (ImageWidth, ImageHeight). Description broadened from "video stream" to "video frame or image". Co-Authored-By: Claude Code 2.1.161 / Claude Opus 4.7 --- src/appendices/media-files.md | 1 + src/schema/objects/metadata.yaml | 20 ++++++++++---------- src/schema/rules/sidecars/media.yaml | 2 +- 3 files changed, 12 insertions(+), 11 deletions(-) diff --git a/src/appendices/media-files.md b/src/appendices/media-files.md index bda5936aab..4e6d98982c 100644 --- a/src/appendices/media-files.md +++ b/src/appendices/media-files.md @@ -155,6 +155,7 @@ A complete sidecar JSON file for an audio-video recording: "VideoFrameCount": 9375, "ImageWidth": 1920, "ImageHeight": 1080, + "ImagePixelFormat": "yuv420p", "AudioCodec": "aac", "AudioCodecRFC6381": "mp4a.40.2", "AudioSampleRate": 48000, diff --git a/src/schema/objects/metadata.yaml b/src/schema/objects/metadata.yaml index 6705e2d3e2..200fc57ca3 100644 --- a/src/schema/objects/metadata.yaml +++ b/src/schema/objects/metadata.yaml @@ -1932,6 +1932,16 @@ ImageHeight: type: integer minimum: 1 unit: px +ImagePixelFormat: + name: ImagePixelFormat + display_name: Image Pixel Format + description: | + The pixel format of the video frame or image, as reported by FFmpeg's + `pix_fmt` field (for example, `"yuv420p"`, `"yuv420p10le"`, `"gray16le"`, + `"rgb24"`). A single `pix_fmt` value encodes the color model, channel + count, chroma subsampling, and bit depth, and can be extracted + automatically with `ffprobe`. + type: string ImageWidth: name: ImageWidth display_name: Image Width @@ -3096,16 +3106,6 @@ PhysioType: enum: - $ref: objects.enums.PhysioTypeGeneric.value - $ref: objects.enums.PhysioTypeEyetrack.value -PixelFormat: - name: PixelFormat - display_name: Pixel Format - description: | - The pixel format of the video stream, as reported by FFmpeg's `pix_fmt` - field (for example, `"yuv420p"`, `"yuv420p10le"`, `"gray16le"`). - A single `pix_fmt` value encodes the color model, channel count, chroma - subsampling, and bit depth, and can be extracted automatically with - `ffprobe`. - type: string PixelSize: name: PixelSize display_name: Pixel Size diff --git a/src/schema/rules/sidecars/media.yaml b/src/schema/rules/sidecars/media.yaml index 710f0d3b66..173806df24 100644 --- a/src/schema/rules/sidecars/media.yaml +++ b/src/schema/rules/sidecars/media.yaml @@ -24,6 +24,7 @@ MediaImageProperties: fields: ImageWidth: recommended ImageHeight: recommended + ImagePixelFormat: optional MediaVideoProperties: selectors: @@ -32,5 +33,4 @@ MediaVideoProperties: VideoCodec: recommended VideoFrameRate: recommended VideoFrameCount: recommended - PixelFormat: optional VideoCodecRFC6381: optional From 399713d3d08a71eee120bf1044b4e2b6e095cdbf Mon Sep 17 00:00:00 2001 From: Yaroslav Halchenko Date: Wed, 3 Jun 2026 12:19:40 -0400 Subject: [PATCH 43/44] Minor wording tune up on the choices Co-authored-by: Yaroslav Halchenko --- src/appendices/media-files.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/appendices/media-files.md b/src/appendices/media-files.md index 4e6d98982c..21fed7a254 100644 --- a/src/appendices/media-files.md +++ b/src/appendices/media-files.md @@ -43,7 +43,7 @@ suffix with appropriate migration tooling {{ MACROS___make_extension_table(["jpg", "png", "svg", "webp", "tif", "tiff"]) }} -When choosing a format, consider the trade-off between file size and data fidelity. +When choosing a format, consider the trade-off between file size, data fidelity, openness and prevalence of the format in the domain of application. Uncompressed or lossless formats (WAV, PNG, TIFF) preserve full quality but produce larger files. Lossy formats (MP3, AAC, JPEG) significantly reduce file size From c29ed8bafcb45afa5aade72b20e5c01e8c669188 Mon Sep 17 00:00:00 2001 From: Yaroslav Halchenko Date: Wed, 3 Jun 2026 12:28:33 -0400 Subject: [PATCH 44/44] Add ImageBitDepth (OPTIONAL) under MediaImageProperties Per PR review discussion on thread r2989972681 (h-mayorquin proposed, @CodyCBakerPhD raised the image-vs-video tension on r3349547943): ImagePixelFormat (FFmpeg pix_fmt) deterministically encodes bit depth for any FFmpeg-readable file, so ImageBitDepth is redundant for video. However: - Common PIL modes (L, RGB, RGBA, P, ...) are implicitly 8-bit-per- channel and do not encode bit depth in the mode name. Image-domain tooling (Pillow, libtiff, PNG library) surfaces bit depth as a first-class integer rather than as part of a pix_fmt string. - For image-only sidecars whose producing tools do not naturally go through FFmpeg, ImagePixelFormat may be absent and bit depth is the only color-precision field available. - An integer is more directly discoverable for the typical researcher than the FFmpeg pix_fmt naming convention. Added as OPTIONAL with an explicit "redundant with ImagePixelFormat when both present; the two MUST agree" note in the description, so the redundancy is acknowledged rather than hidden. Co-authored-by: Heberto Mayorquin Co-authored-by: Cody Baker Co-Authored-By: Claude Code 2.1.161 / Claude Opus 4.7 --- src/appendices/media-files.md | 1 + src/schema/objects/metadata.yaml | 16 ++++++++++++++++ src/schema/rules/sidecars/media.yaml | 1 + 3 files changed, 18 insertions(+) diff --git a/src/appendices/media-files.md b/src/appendices/media-files.md index 21fed7a254..62759f3dfb 100644 --- a/src/appendices/media-files.md +++ b/src/appendices/media-files.md @@ -156,6 +156,7 @@ A complete sidecar JSON file for an audio-video recording: "ImageWidth": 1920, "ImageHeight": 1080, "ImagePixelFormat": "yuv420p", + "ImageBitDepth": 8, "AudioCodec": "aac", "AudioCodecRFC6381": "mp4a.40.2", "AudioSampleRate": 48000, diff --git a/src/schema/objects/metadata.yaml b/src/schema/objects/metadata.yaml index 200fc57ca3..48d2600383 100644 --- a/src/schema/objects/metadata.yaml +++ b/src/schema/objects/metadata.yaml @@ -1907,6 +1907,22 @@ ImageAcquisitionProtocol: [URI](SPEC_ROOT/common-principles.md#uniform-resource-indicator) (for example from [protocols.io](https://www.protocols.io/)). type: string +ImageBitDepth: + name: ImageBitDepth + display_name: Image Bit Depth + description: | + Bit depth per channel of the stored pixel data of the video frame or + image (for example, `8`, `10`, `12`, `16`). For multi-channel data + this is the depth of each individual channel. + When `ImagePixelFormat` is also provided, this field is redundant with + the bit depth encoded in the FFmpeg `pix_fmt` value (for example, + `yuv420p10le` -> 10) and the two MUST agree. `ImageBitDepth` is + nonetheless useful as a more directly discoverable summary, and as the + primary precision field for image-only sidecars whose producing tools + do not naturally surface `pix_fmt`. + type: integer + minimum: 1 + unit: bit ImageDecayCorrected: name: ImageDecayCorrected display_name: Image Decay Corrected diff --git a/src/schema/rules/sidecars/media.yaml b/src/schema/rules/sidecars/media.yaml index 173806df24..9e5dc25ed4 100644 --- a/src/schema/rules/sidecars/media.yaml +++ b/src/schema/rules/sidecars/media.yaml @@ -25,6 +25,7 @@ MediaImageProperties: ImageWidth: recommended ImageHeight: recommended ImagePixelFormat: optional + ImageBitDepth: optional MediaVideoProperties: selectors: