Manageable Messaging Protocol (MMP): Core

➤Source file (imported manually from Hedgedoc)

This is a rough draft specification.

The Micro Messaging Protocol aims to enable secure, unencumbered, and interoperable instant messaging, built on top of and alongside existing standards. Existing standards in this space fail to accomplish their goals for various reasons: Matrix is encumbered by its event graph, making self-hosting prohibitively expensive, and a confusingly large and growing single specification with no extension standardization framework, and the Extensible Messaging and Presence Protocol (XMPP) is encumbered by years of rotting code, reliance on dated technology, and contradicting extensions in crucial areas.

MMP endeavors to avoid these shortcomings by providing extensibility governed by a standards body, and by considering features in the base protocol which enable more advanced features, while keeping those advanced features separate.

Concepts and Terminology

Frame

: A packet of data which contains the information in transit. This corresponds to a Protobuf message. It's a metaphor: a frame _contains_ a message. It also can contain other information about the message, including delivery information, sender, recipients, and what type of content is contained in the message.

Conversation

: A conceptual unit, determined by its participants (to and from) and its conversation ID which acts as a disambiguator; it can include messages and state. Most conversations have an ID of 0, as multiple conversations between the same group of people is not always useful but it can be.

: **Note:** Group chats could end up with a large conversation ID as a consequence of adding and removing participants. This practice is not recommended and a chat room, to be defined in a companion specification, should be used instead.

Message

: A single, user-specified, sequential element of a conversation. It may be a text message such as "Hello!", or a group of images. One cognitive unit should be represented per message: an album of images or a single image, a text message, or a captioned location are good examples.

General Rules of Handling MMP Protobuf

In both the client-server and server-server (federation) APIs, Protobuf is used as the wire format for frames. The following rules apply to all Protobuf "messages" as used in MMP:

• The field number `1` is always a string, containing a URI which represents what type of frame is being sent, and therefore which Protobuf message format to use. The frame type (the value of `1`) will always be listed alongside the message it represents.

* Tip: to parse this, use `message MMP_Ping` first to determine the type, then if the type is known, re-parse the frame with that type.

* Frame type URIs which refer to frames specified in this document are in the URN namespace `urn:mmp:mmp1:*`; for example, the "ping" message is `urn:mmp:mmp1:ping`.

• The field number `911` always holds an error code, encoded as a string URI. If there is not a response, the field number `1` should be set to `""` (the empty string).

* Extensions and specifications related to MMP **MUST NOT** use `911` as anything other than as specified here!

• If `911` is set and `1` is `""`, then `112` (another emergency phone number) MAY contain an error message, encoded as a string.

* Error messages MUST be generated and interpreted as plain text. They MAY, however, contain URLs which clients SHOULD make clickable.

* While error codes are specified, error messages are not. They may be in any language and include server or situation specific information. However, when in doubt: use English and be specific about the problem without giving away sensitive details.

Client to Server Protocol

MMP data is to be accepted by servers and sent to clients as Protocol Buffers (Protobuf) messages, specified below, over a secure WebSocket at a server-specified endpoint on the domain. This endpoint is specified relative to the given login domain as plain text at the well-known path `/.well-known/mmp-client-socket` over HTTPS. This is done to allow alternate APIs to be added on top.

Ping

An authenticated client MUST send a ping at least as often as once per 30 seconds. Clients SHOULD send pings as often as every 10 seconds. Servers SHOULD disconnect clients that do not send a heartbeat for 30 seconds. It's your loss if you decide not to do that.

The Protobuf message for a ping looks like this.

syntax = "proto3"

message MMP_Ping {
    string frameType = 1; // == "urn:mmp:mmp1:ping"
    optional string errorMessage = 112;
    optional string errorCode = 911;
}

Every frame type is based on this frame type; or, in other words, this is the most minimal frame type MMP allows.

Authentication

Servers are expected to authenticate clients via an HTTP header, typically either the Cookie or Authorization headers, when a client attempts to establish the WebSocket connection.

If a client is not authenticated and wishes to be (so that they can use the Client-Server API), it is to send an empty, unauthenticated HTTP POST request to the Client-Server websocket endpoint. The appropriate response is a Protobuf message according to the following:

message MMP_Auth {
    string frameType = 1; // == "urn:mmp:mmp1:auth"
    // First listed type is preferred.
    repeated AuthType loginType = 2;
    // If this is the empty string ("") or unset, registration is disabled.
    // If this is "urn:mmp:mmp1:auth#registration", in-band registration is to be used.
    // Otherwise, this SHOULD be an HTTPS URI to send the user to in a Web browser to complete registration.
    string registrationEndpoint = 3;
    // Required when using OAuth2, OpenID, or IndieAuth.
    string authEndpoint = 16;
    
    // If this is set, it MUST be presented to the user with a check-box before allowing them to register.
    optional string termsOfServiceUrl = 24;
    // If this is set, it MUST be presented to the user with a check-box before allowing them to register.
    optional string privacyPolicyUrl = 25;
    // Clients SHOULD present this to the user as a link before logging in or registering.
    // Recommended labels include "Server legal documents" or "example.org Legal".
    optional string legalUrl = 26;
    
    // A plain-text string, which MAY include clickable URLs,
    // documenting the requirements for a password if in-band
    // registration is in use.
    optional string passwordRequirements = 31;
    
    enum AuthType {
        OAUTH2 = 1;
        // Common in the decentralized world.
        // Use of either OpenID or IndieAuth
        // is recommended. 
        OPENID = 2;
        INDIEAUTH = 3;
        // If username and password authentication is expected.
        USERNAME_AND_PASSWORD = 16;
        // Should be treated identically to the above except that it shows an email box instead of a username box.
        // Prefer username and password over this, because that may skip a step on many clients.
        EMAIL_AND_PASSWORD = 17;
    }
    
    optional string errorMessage = 112;
    optional string errorCode = 911;
}

Password-based login

If a client wishes to log in with a password (assuming the server listed and allows it), the client is to send a POST request over valid HTTPS (with any content type, as it should be ignored) with a frame conforming to the following format:

message MMP_UsernameLogin {
    string frameType = 1; // == "urn:mmp:mmp1:auth#username" OR "urn:mmp:mmp1:auth#email"
    string username = 2; // email goes in this field too
    string password = 3;
}

The HTTP response, with a server-selected appropriate status code, will obey the following format:

message MMP_LoginResponse {
    string frameType = 1; // == "urn:mmp:mmp1:auth#granted"
    string accessToken = 2;
    optional string errorMessage = 112;
    optional string errorCode = 911;
}

The value of `accessToken`, if set, should be considered a Bearer token, and when connecting to the WebSocket, a header like this is to be set:

Authorization: Bearer ACCESS_TOKEN_HERE

Error codes that may be used here:

• `urn:mmp:mmp1:auth#incorrect`: The client supplied incorrect credentials. This should be the "default" error code to return if authentication fails for any reason.
• `urn:mmp:mmp1:auth#invalid`: The credentials supplied were in an incorrect format, such as a malformed email address.
• `urn:mmp:mmp1:error#disabled`: Authorization is not allowed. End users might see this error if the server changes its configuration to disallow password-based authentication while they are entering their credentials.
• `urn:mmp:mmp1:auth#banned`: Servers MAY return this code if the user or client attempting to log in has been banned or suspended from the server. The error message (field `112`) may include more details, such as suspension length or appeal instructions.

In-Band Account Registration

Registration, when done "in-band," occurs via the same HTTPS endpoint used for password-based login, if it is accepted (see `message MMP_Auth#registrationEndpoint` above).

At least one of email-and-password or username-and-password based login MUST be enabled to facilitate logging in after registration.

The POST body should contain the following frame:

message MMP_AuthRegistration {
    string frameType = 1; // == "urn:mmp:mmp1:auth#registering"
    // The username that the user requests.
    // This becomes part of the user's public ID.
    string username = 2;
    // The server MAY accept registrations without
    // this field, but doing so may mean users will
    // not be able to recover their account should
    // they lose their password.
    // This can be used for login but you probably
    // want it for administrative purposes.
    string email = 3;
    string password = 8;
}

The HTTP response, with appropriate status code, should conform to this frame:

message MMP_Success {
    string frameType = 1; // == "urn:mmp:mmp1:auth#registered"
    bool success = 2;
    optional string errorMessage = 112;
    optional string errorCode = 911;
}

• **NOTE:** The response MAY conform to `message MMP_LoginResponse` instead. In this case, the frame type will be `"urn:mmp:mmp1:auth#granted"`.

Assuming the response is `message MMP_Success` and no error code was given, the client should then attempt to log in with their credentials.

Error codes that may be used here include:

• <a id="urn:mmp:mmp1:auth%23invalid">`urn:mmp:mmp1:auth#invalid`</a>: The credentials supplied were in an incorrect format, such as a malformed email address or a username beginning with a forbidden character.
• `urn:mmp:mmp1:error#disabled`: Authorization is not allowed. End users might see this error if the server changes its configuration to disallow password-based authentication or registration while they are entering their credentials.
• `urn:mmp:mmp1:auth#banned`: Servers MAY return this code if the client's IP, IP range, or region has been banned or suspended from the server. The error message (field `112`) may include more details.
• `urn:mmp:mmp1:auth#badname`: The username includes a forbidden word. Servers MAY return this to indicate the user attempted to create an account using a banned word. [`#invalid`](#urn:mmp:mmp1:auth%23invalid) should be returned instead if the username begins or ends with a reserved character, or uses illegal characters -- servers MUST NOT use this error in that circumstance.
• `urn:mmp:mmp1:auth#badpassword`: The password does not meet the server's requirements. Field `112` SHOULD contain more details about the specific requirement the user did not meet.
• `urn:mmp:mmp1:auth#exists`: The user already exists. This may be due to a colliding username or, if the server chooses, it MAY throw this error in response to the same email address. The error message in field `112` may include more details.

Sending messages

To send a message, send a Message frame to the server.

message MMP_Message {
    string frameType = 1; // == "urn:mmp:mmp1:message"
    // An opaque ID.
    // Servers MUST include this.
    // Clients sending new messages MUST exclude this.
    // Clients sending edited messages MUST include this,
    // and it MUST match a message which the user is allowed to edit.
    // If a client sets this and it doesn't match an existing message,
    // servers SHOULD throw an error instead of ignoring it.
    string id = 2;
    // The sender's chat address. Clients MUST set this to the empty string
    // or an address they control (servers may allow, for example, plus-addressing).
    // If a server sees an invalid "from" address, it is expected to either reduce it to a form that is valid, replace it with a default address for the user, or throw an error.    
    string from = 3;
    // Addresses to send the message to.
    // Replies in a conversation MUST be treated as "reply all"
    // by default.
    // This may include a chatroom. If that is the case, it SHOULD
    // be the only "to" listed, even if you are the recipient.
    repeated string to = 4;
    // 5 and 6 are set aside for CC and BCC, respectively.
    // They are not implemented in this spec and they may never
    // be useful so they may never be used.
    
    // The address to send any replies to.
    // This may be useful to redirect replies into a separate channel,
    // such as if a message was forwarded.
    // If this is set, clients MUST obey it and make sure their users
    // are aware that the reply will be sent in a different chat.
    string replyTo = 7;
    
    // Servers MUST include this.
    // Clients MUST exclude this.
    // Represents the date and time the message was sent
    // (or, in an untrusted scenario, when it was first known to the server).
    Timestamp date = 13;
    
    // The MIME type (or Matrix type) of the content. No restrictions are placed on
    // what type a message can be. To have multiple equivalent bodies,
    // multipart/alternate is recommended.
    // This MUST be set. It cannot be blank.
    string mimeType = 14;
    // The body of the content. If a Matrix type is specified,
    // the body should be encoded as either JSON or MessagePack.
    bytes content = 15;
    
    // The conversation ID to which this message belongs.
    // See Conversation in the Concepts section.
    int32 conversation = 16;
    
    // In each flag, everything after the first = is considered a value.
    // Values are optional.
    // Flags are to be specfied by extensions.
    repeated string flags = 17;
    
    
    // These fields are only used in responses.
    // They MUST NOT be set when making a request.
    // If they are set, servers SHOULD ignore them.
    
    optional string errorMessage = 112;
    optional string errorCode = 911;
}

Servers MUST send a [`message MMP_Success`](#MMP_Success) response, indicating whether the message was sent and, if it wasn't, why not (via an error code and optional message).

HTML messages are NOT RECOMMENED due to the complexity of HTML. Instead, prefer Markdown.

Error codes that may be returned here:

• `urn:mmp:mmp1:profile#missing`: One or more of the addresses you tried to contact could not be reached. This is accompanied by a failure if all of the addresses failed or the server otherwise refused to send it.
• `urn:mmp:mmp1:error#delivery-failed`: The remote server could not be reached; or the remote server did not accept the message.
• `urn:mmp:mmp1:auth#banned`: The sender is not allowed to send messages to the remote server, either by the sender's server or by the remote server. If the sender's entire server is banned by the remote server, the response will typically be `urn:mmp:mmp1:error#delivery-failed` because the server will refuse to connect.
• `urn:mmp:mmp1:message#invalid`: Required fields were missing, or if the server is strict about the excluded fields, forbidden fields were set.
• `urn:mmp:mmp1:message#toobig`: The message is too large. This MUST be accompanied by a failure. The client should instead upload the content to a media server.

Retrieving messages

By message ID

If you have a message or event ID, you can try to find that message by sending this frame:

message MMP_GetMessageByID {
    string type = 1; // == "urn:mmp:mmp1:message#by-id"
    // The message ID to get.
    string id = 2;
    // The server MAY require this to distinguish
    // messages by conversation.
    // It's okay to omit this if you don't know it,
    // but you may not get a result.
    optional Conversation conversation;
    
    // The identifying bits of a conversation.
    message Conversation {
        string from = 3;
        repeated string to = 4;
        int32 conversation = 16;
    }
}

The response will be a [`message MMP_Message`](#MMP_Message) frame, which, if necessary, may contain an error.

Such errors may include:

• `urn:mmp:mmp1:error#missing`: The message could not be found, or you do not have access to the message.
• `urn:mmp:mmp1:message#ambiguous`: The server could not find the message by ID alone and requires a conversation ID.

By conversation

To retrieve a message by conversation, send a frame with this format:

message MMP_ConversationFrame {
    string frameType = 1; // == "urn:mmp:mmp1:message#by-conversation"
    // This always includes an address belonging to the current client.
    repeated string participants = 2;
}

This returns a message list frame:

message MMP_MessageList {
    string frameType = 1; // == "urn:mmp:mmp1:message#list-by-conversation"
    repeated Message messages = 2;
    optional string errorMessage = 112;
    optional string errorCode = 911;
    
    // This type is equivalent to MMP_Message but without
    // fields 1, 112, and 911 (because it's not a frame).
    // If this and MMP_Message mismatch apart from fields 1, 
    // 112, and 911, MMP_Message is the correct pattern to follow.
    // Report the issue if you see it.
    message Message {
        string id = 2;
        string from = 3;
        repeated string to = 4;
        string replyTo = 7;
        Timestamp date = 13;
        string mimeType = 14;
        bytes content = 15;
        int32 conversation = 16;
        repeated string flags = 17;
    }
}

User IDs

TODO

Appendix A: Errors

Errors may be under any namespace, but generic errors defined in this document use the `urn:mmp:mmp1:error#*` namespace.

General errors

`urn:mmp:mmp1:error#missing`

: The requested resource could not be found.

`urn:mmp:mmp1:error#delivery-failed`

: The remote server could not be reached or delivery failed for another reason.

`urn:mmp:mmp1:error#disabled`

: The requested feature is supported, but the server administrator has disabled it.

`urn:mmp:mmp1:error#unimplemented`

: The requested feature, frame type, or other action has not been implemented.

Auth errors

`urn:mmp:mmp1:auth#incorrect`

: The supplied credentials are in the correct format but do not match any records available to the server.

`urn:mmp:mmp1:auth#invalid`

: The supplied credentials are malformed. Reasons may include:

• An email address field received something that isn't an email address.
• A supplied password does not meet the server's minimum requirements.
• A required field was not supplied.

`urn:mmp:mmp1:auth#banned`

: The user was banned from the server.

: The IP address, IP address range, internet service provider, or location was banned from the server; typically, this is a rate limit or a spam protection, but may also be enacted for legal reasons.

: A remote server being interacted with has banned the user from participating in conversations on that server.

`urn:mmp:mmp1:auth#badname`

: The requested username includes a banned word. Servers may return this to refuse names including certain words such as slurs, curse words, or reserved system terms, which should be accompanied by a (temporary) IP block, during registration.

• `urn:mmp:mmp1:auth#invalid` MUST be returned instead if the username is blocked due to a forbidden character or forbidden character placement.

`urn:mmp:mmp1:auth#exists`

: The requested username is already in use.

: The email address given is already associated with an account.

Message errors

`urn:mmp:mmp1:message#invalid`

: TODO: this should be a generic error

: Required fields were missing.

: Fields excluded from certain requests were set.

`urn:mmp:mmp1:message#toobig`

: The message submitted exceeds the maximum size allowed by the server.

`urn:mmp:mmp1:message#ambiguous`

: The message requested matches more than one message.

• The server may return this error even if the message ID is unique on the server, if that server expects the message IDs to be unique only per conversation. This isn't recommended, but it is allowed, and should be accounted for.
• This may be solved by supplying conversation data.

Profile errors

`urn:mmp:mmp1:profile#missing`

: The user exists but does not have profile information.

: Profile information for the user is not known to the server.

: The user is missing (especially when other things can be missing).

• The error message (field `112`) or frame type may help to discern what's going on.