MSC-Draft: Message Sets by Relations

In many modern chat systems, users can upload multiple images at once, and they will appear in their friends' chat clients as one message containing many images. In Matrix, this has been previously excluded due to the philosophy of one event per entity. Additionally, the lack of albums causes inconsistencies when bridging between Matrix and an album-supporting service. This MSC aims to specify a means to create such albums in a way that is both backwards compatible and follows the Matrix philosophy.

➤Source (manually imported from Hedgedoc)

Proposal

This MSC introduces a MSC-1767 content block m.message_set, which serves as a marker for a set of events to relate back to, and the m.message_set relation type, which relates to an event with that content block.

ℹ️ Note: MSC-1767 is not required, although it is highly recommended for implementation as its MSC has already been approved and will be mandatory in future room versions.

➤MSC-1767

Message set content block

The m.message_set content block should contain the following keys:

• keys (required): an array of strings. These correspond to relation keys and are explained more below.
• type (optional): a string indicating the primary type of message to be expected. For image albums, this will likely be m.image (see MSC-3552). This is useful to allow clients to show rich status indicators about the message set.

An example of such a message:

{
    "type": "m.message_set",
    "content": {
        "m.message_set": {
            "keys": [
                "1",
                "2",
                "third"
            ],
            "type": "m.image"
        }
    }
}

Message set markers MUST NOT have fallback data, such as m.text, and therefore should not be displayed in clients that do not support message sets. The fallback behavior is that each part is displayed individually.

Additionally, clients MAY display a potentially-rich "sending" status (i.e. "uploading an album") while a message set is pending, and they MAY consider an incomplete message set no longer pending after some time. "Pending" is the state when a message set is incomplete but some events may still be uploading, and incompleteness can be determined by checking if the marker's m.message_set relations contain one valid message for each specified key.

Clients SHOULD NOT display incomplete message sets, unless to show that it is pending or incomplete. Additionally, when all events in a set have been received, they SHOULD be shown as one multi-part unit at the location of the marker event.

Clients MUST NOT send a message set with less than two events. Instead, send the one message part without a set marker, or do not send an event at all. Another MSC could implement an "uploading" state, which would likely be best done with an ephemeral event, like typing indicators.

If a client wishes to limit how many parts are displayed in the timeline, the full set SHOULD be advertised and readily available, such as with a "See the full album" button; however, it is NOT REQUIRED to display or use all parts of a set.

➤MSC-3552

Replies and threads

The marker event MAY be in reply to another message or in a thread. In these cases, the message set should display as a reply or in a thread, as expected.

Additionally, if the message set is a reply or in a thread, the m.in_reply_to and is_falling_back fields in m.relates_to should be set accordingly on each part and the marker (see the Rich Replies and Threading sections in the specification). This should work around an issue where if a message set is sent in a thread, it escapes the thread on clients that support threads but not message sets.

➤Rich Replies

➤Threading

Keys

Message set relation keys are arbitrary strings; however, they MUST be unique among a set. Keys could, for example, be set to the filename of a file that's being uploaded, a sequential number, or a random value. Clients MUST use the order of the keys in the m.message_set marker event to determine the order of parts displayed in a message set.

In the event a client wishes to separate any event from the message, the entire set still MUST strictly retain the order marked by the keys, dividing the message as necessary to do so.

Examples

{
    "type": "m.image",
    "content": {
        "m.text": [
            {"body": "example.png (128kb) example.org/_matrix/media/v3/download/example.org/abcd1234"}
        ],
        "m.file": {
            "url": "mxc://example.org/abcd1234",
            "name": "matrix.png",
            "mimetype": "image/png",
            "size": 12345
        },
        "m.relates_to": {
            "rel_type": "m.message_set",
            "key": "1",
            "event_id": "$SETMARKERID"
        }
    }
}

A sequence of messages, annotated.

{
    // Irrelevant fields excluded.
    "event_id": "$root",
    "type": "m.message_set",
    "content": {
        "m.message_set": {
            "keys": {
                "file1.jpg",
                "file1.jpg-2",
                "file2.gif",
                "__caption__"
            }
        }
    }
}

{
    "event_id": "$caption",
    "type": "m.text",
    "content": {
        m.text: [
            {
                "body": "A group of people on a plane.",
                "mime_type": "text/plain"
            }
        ],
        "m.relates_to": {
            "rel_type": "m.message_set",
            "key": "__caption__",
            "event_id": "$root"
        }
    }
}

{
    "event_id": "$part1",
    "type": "m.image",
    "content": {
        "m.text": [
            {
                "body": "file1.jpg (128kb) example.org/_matrix/media/v3/download/example.org/abcd1234"
            }
        ],
        "m.file": {
            "url": "mxc://example.org/abcd1234",
            "name": "matrix.png",
            "mimetype": "image/jpeg",
            "size": 12345
        },
        "m.relates_to": {
            "rel_type": "m.message_set",
            "key": "file1.jpg",
            "event_id": "$root"
        }
    }
}

{
    "event_id": "$part2",
    "type": "m.image",
    "content": {
        "m.relates_to": {
            "rel_type": "m.message_set",
            "key": "file2.gif",
            "event_id": "$root"
        },
        "m.text": [
            {
                "body": "file2.gif (128kb) example.org/_matrix/media/v3/download/example.org/abcd5678"
            }
        ],
        "m.file": {
            "url": "mxc://example.org/abcd5678",
            "name": "file2.gif",
            "mimetype": "image/gif",
            "size": 12345
        }
    }
}

{
    "event_id": "$part3",
    "type": "m.image",
    "content": {
        "m.relates_to": {
            "rel_type": "m.message_set",
            "key": "file1.jpg-2",
            "event_id": "$root"
        },
        "m.text": [
            {
                "body": "file1.jpg (128kb) example.org/_matrix/media/v3/download/example.org/abcd91011"
            }
        ],
        "m.file": {
            "url": "mxc://example.org/abcd91011",
            "name": "file1.jpg",
            "mimetype": "image/jpeg",
            "size": 12345
        }
    }
}

The end result of the above example should be that one message is displayed with the following parts, in this order:

1. file1.jpg (key: file1.jpg)

2. file1.jpg (key: file1.jpg-1)

3. file2.gif (key: file2.gif)

4. "A group of people on a plane." (key: caption)

Security Considerations

There is a risk of impersonation, as Matrix does not mandate limitation on who may add relations to a given message, and clients will display each set as one message, from one sender. Therefore, homeservers MAY, and clients MUST, reject any part which is not from the sender of the marker.

Additional Considerations

This MSC does not define what types of events may be in a set together. A future MSC could define rules for these events; for example, sets may only be allowed to contain one m.text event.

Clients may opt to drop some message parts if it can't display them properly, such as a m.text part mixed with m.image parts. Therefore, if a client wishes to send a captioned image or album this way, it is preferred to place the caption's key at the beginning or end of the key list. Do not break the key order. To this end, clients should be prepared to show album captions at either the top or bottom of the album.

The directionality of a set's part order is not defined. A good way to handle this would be based on the user's language and/or locale; i.e. in English, left-to-right then top-to-bottom. A future MSC could introduce a field which mandates or recommends a particular directionality.

This MSC does not define a timeout period, or what the behavior is when it becomes clear that a given set will not be completed.

Clients are free to decide how many, and in what layout, parts should be displayed in a message set. This could mean an m.image typed set shows the first 10 images in masonry layout.

Alternatives

• Sending every part all in one event. This is against the "event" philosophy.
• Relating each event to the last, as a reply chain. A version of this should be done alongside the m.message_set relation where applicable; see "Replies and threads" (paragraph 2) above.
• Sending a sequence of events as a thread. This might cause some clients to confusingly hide the set's parts, and users would have to open the thread to see all of the messages.

Unstable Prefix

While this MSC is not considered stable by the specification, the name org.matrix.msc9999.message_set should be used in place of m.message_set everywhere it appears in this document.

Glossary

• (Message) set: a collection of related events, displayed as one unit.
• (Message set) marker: an event that has a valid m.message_set content block. The other events in any given set relate back to this message.
• (Message set) part: an event that is correctly related to a message set marker, according to the rules set forth in this MSC.