Skip to content

MSC4274: Inline media galleries via msgtypes #4274

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
Johennes wants to merge 2 commits into
base: main
Choose a base branch
from
Open

MSC4274: Inline media galleries via msgtypes #4274

Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
4edac2e
MSC4274: Inline media galleries via msgtypes
Johennes Mar 18, 2025
5ff56b9
Fix typo
Johennes Mar 18, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
111 changes: 111 additions & 0 deletions proposals/4274-inline-media-galleries.md
View file
Open in desktop
Copy link
Copy Markdown
Member

@turt2live turt2live Mar 18, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Implementation requirements:

  • Client

Copy link
Copy Markdown
Contributor Author

@Johennes Johennes Mar 24, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Copy Markdown

@bxdxnn bxdxnn Apr 13, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Original file line number Diff line number Diff line change
@@ -0,0 +1,111 @@
# MSC4274: Inline media galleries via msgtypes

Matrix allows sharing individual media files such as images or videos via
dedicated [`m.room.message` msgtypes]. This is, however, not practical for
sharing larger collections of files. For one thing, there is no mechanism to set
a joint caption for several media files. For another, there is no way to inform
other clients that a set of pictures belongs together and should be grouped
accordingly in the timeline. These shortcomings call for a built-in media
gallery feature as it is widely known from other messaging apps.

This proposal defines a compact and pragmatic variant of sharing media galleries
in a single event.

## Proposal

A new `msgtype` of `m.gallery` is introduced with the following properties in
Copy link
Copy Markdown
Contributor

@MurzNN MurzNN Apr 14, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If we are taking in the scope not only images, but also files (m.file), why do we call this a "gallery"?

From my point of view, the best term for this feature is "attachments", because actually on the technical side we have a regular m.message with just some attached entities (images, videos, files, anything else).

And the term "gallery" is related more to the displaying style, not to the message type, because a message with 5 attached .zip files will be hard to display as a gallery ;) Even the same "5 images" can be displayed as a list of file names, not as a gallery, if we have a text-based UI.

So, naming this not as a "gallery" but using a more general term, will extend the scope of such message type, and will allow, in the future, to put into itemtypes other interesting things like references to other messages, multiple forwarded messages, etc.

What do you think about this idea?

`content`:

- `body` (string): The caption of the gallery.
- `format` (string): The format used in `formatted_body`. Currently only
`org.matrix.custom.html` is supported.
- `formatted_body` (string): The formatted version of the body. Required if
`format` is specified.
- `itemtypes` (array): Ordered array of metadata for each item in the gallery.
Copy link
Copy Markdown

@ErikMichelson ErikMichelson Mar 24, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why is this called itemtypes and not simply items?

Copy link
Copy Markdown
Contributor Author

@Johennes Johennes Mar 24, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hm, I think I landed on this because itemtypes is basically the generalization of msgtype plus the associated data into an array. I agree that items would be clearer though.


The objects in the `itemtypes` array have the same schema as the `content` of
the msgtypes `m.image`, `m.video`, `m.audio` and `m.file`. Rather than
`msgtype`, they use `itemtype` though.

``` json5
{
"type": "m.room.message",
"content": {
"msgtype": "m.gallery",
"body": "Checkout my photos from [FOSDEM 2025](https://fosdem.org/2025/)",
"format": "org.matrix.custom.html",
"formatted_body": "Checkout my photos from <a href=\"https://fosdem.org/2025/\">FOSDEM 2025</a>",
"itemtypes": [{
"itemtype": "m.image",
"body": "filename.jpg",
"info": {
"h": 398,
"mimetype": "image/jpeg",
"size": 31037,
"w": 394
},
"url": "mxc://example.org/JWEIFJgwEIhweiWJE",
...
}, ...]
},
...
}
```

## Potential issues

The size of galleries under this proposal is limited by the event size limit
(65,536 bytes). An encrypted gallery event with a single image, including
thumbnail, and no caption measures about 2,000 bytes. Each additional image adds
roughly another 1,000 bytes. Therefore galleries would be capped to about 60
images. While this is a limitation, the amount seems practical for most use
cases.

Additionally, this proposal doesn't prescribe a fallback mechanism for clients
that don't support `m.gallery` msgtypes. Clients that care about backwards
compatibility could encode media links or a generic fallback text into `body` or
`formatted_body`, respectively. It is unclear, however, if a fallback is
actually desirable or not because unwrapping 60 images into somebody else's
timeline might be considered inappropriate if not spammy.

## Alternatives

This proposal is similar to the seemingly abandoned [MSC3382]. The latter
conflates galleries into `m.text` (or even arbitrary) msgtypes and reuses
`msgtype` inside the items which creates a circular dependency. These issues are
solved in the present proposal.

Rather than adding a new `msgtype`, galleries could be expressed via extensible
events by reusing parts of [MSC1767] and related proposals. This would only take
effect in a future room version, however, and can be covered in a separate MSC.

Another alternative is to define galleries as chains of related events as in
[MSC2881]. Compared to the inline approach taken in this proposal, this has
better fallback support, is more flexible and less prone to event size limits.
At the same time, it is significantly more complex to implement in terms of
replies, edits, redactions, forwarding, etc.

Finally, rather than defining galleries on the sender's side, receiving clients
could opportunistically group consecutive images in the timeline into galleries.
This might need specific rules such as grouping by sender or time windowing.
Different clients might prefer very different display rules, however. Therefore,
Copy link
Copy Markdown
Contributor

@MurzNN MurzNN Apr 14, 2026 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we should clarify here the displaying rules: where the "body" text should be displayed - above the attached items or below them, or maybe add a field that holds the display style.

Because this is crucial from the user point of view:

  • If we display the body above, it will be the main "body" of the message, and the attached images are just an addition (Slack style).

  • If we display the body below, the main content of the message will be files, and the "body" will be just a commenting text, in addition to the images (Telegram style).

Both variants of rendering are possible and available in different messengers, but they bear completely different meanings of the message.

Copy link
Copy Markdown
Contributor

@MurzNN MurzNN Apr 14, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I, personally, am a fan of the Slack style, not Telegram style ;)

Just imagine a message:
Variant 1:

Checkout my photos from FOSDEM 2025:
[photo 1]
[photo 2]
[photo 3]

Variant 2:

[photo 1]
[photo 2]
[photo 3]
Checkout my photos from FOSDEM 2025:

The order of things in variant 1 looks more correct, right?

Copy link
Copy Markdown
Contributor

@MurzNN MurzNN Apr 14, 2026 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actually, these are two totally different things from the logical point of view:

  • Above files - is the "message body" (main content).
  • Below files - is the "message caption" (secondary thing: signature, comment, subscription, not the main content).

And if we keep the text named as "body", we should put it above. Or, rename to "caption" then.

Or, maybe even keep both ones ;) with recommendations what to render where.

it seems more practical that the spec annotates galleries clearly while leaving
their UI treatment to clients themselves.

## Security considerations

None.

## Unstable prefix

Until this proposal is accepted into the spec, implementations SHOULD refer to
`m.gallery` as `dm.filament.gallery`.

## Dependencies

None.

[`m.room.message` msgtypes]: https://spec.matrix.org/v1.13/client-server-api/#mroommessage-msgtypes
[MSC3382]: https://github.com/matrix-org/matrix-spec-proposals/pull/3382
[MSC1767]: https://github.com/matrix-org/matrix-spec-proposals/pull/1767
[MSC2881]: https://github.com/matrix-org/matrix-spec-proposals/pull/2881