Implement XMPP federation

[stevenroose]

I can't remember where or from who, but a few months ago I picked up a message that Wire wanted to implement federation between Wire servers.

I just want to give you guys a giant heads-up for the idea and meanwhile suggest using the XMPP protocol specifically for doing so. While it might seem a bit harder at first than just specifying your own custom protocol, XMPP is a very solid and extensible protocol with 20 years of R&D invested into it.

It's possible to cherry-pick extensions that make sense for Wire and there is a lot of tooling available for speaking XMPP ([1], [2], [3]) in Haskell.

As an added benefit, federating over XMPP allows all Wire users to chat with existing users on Jabber and to join existing public Jabber chat rooms. It would be an awesome step in the direction of an open federated chat network instead of a group of silo'd private solutions.

I'm sure if you have questions or doubts regarding this, there would be plenty of people with thorough knowledge on XMPP federation that are willing to help you out. ([1], [2], [3])

EDIT: some more thoughts can be found in this comment: #631 (comment)

[neongreen]

If I understand correctly, this involves basically ditching our own existing E2E protocol (https://github.com/wireapp/proteus), moving to something like OMEMO, and implementing OMEMO support in all our clients.

Then, whenever we want to improve our protocol, we need to publish a XEP so that other Jabber clients can implement this XEP (if they want to be able to talk with Wire users). Alternatively we might have our own protocol that we can iterate on, and keep OMEMO as a fallback for non-Wire users.

This sounds rather significantly more involved than "might seem a bit harder at first".

Or am I missing something?

[neongreen]

Also: I'm not familiar with XMPP federation, so I'm curious to know whether adopting XMPP will make federation easier for us even if we decide that we still need to run our custom protocol on top of it.

[Zash]

It depends on how much interoperability with the existing XMPP world you want. It's perfectly reasonable to have your own private extensions. OMEMO is an extension, there are other E2EE extensions.
In any case, being able to draw from 20 years of experience can't hurt, can it?

[weiss]

whenever we want to improve our protocol, we need to publish a XEP [...]. This sounds rather significantly more involved than "might seem a bit harder at first".

Yes: Standardization, negotiation with other players, debugging interop issues etc. certainly involves additional effort. I think interop with other systems is clearly in the interest of customers (and their external contacts), but obviously it's less clear from the vendor's point of view. Capitalism sucks 😄

However, advantages might include:

• The "20 years of experience" point that has been made above. You might avoid some pitfalls by not reinventing the federation wheel from scratch.
• The external feedback you get when publishing spec proposals/updates.
• Additional reputation in some parts of the more 'geeky' audience, which might help with marketing.
• As your competitors don't seem to be convinced of this model, it might be another unique selling point 😅

[neongreen]

Oh, just in case: I wasn't implying "this is not trivial so why are you even proposing this". Far from it. I will, however, admit that I'm sceptical about standardization not being an incredibly time- and money-consuming process. That's why "a bit harder" felt out of place in the original post. Please feel free to use this issue to post links / papers / reading material of any kind that might help us make a more informed decision (when we actually get to discussing federation internally).

…

On Thu, Feb 14, 2019, 19:18 Holger Weiß ***@***.*** wrote: whenever we want to improve our protocol, we need to publish a XEP [...]. This sounds rather significantly more involved than "might seem a bit harder at first". Yes: Standardization, negotiation with other players, debugging interop issues etc. certainly involves additional effort. I think interop with other systems is clearly in the interest of customers (and their external contacts), but obviously it's less clear from the vendor's point of view. Capitalism sucks 😄 However, advantages might include: - The "20 years of experience" point that has been made above. You might avoid some pitfalls by not reinventing the federation wheel from scratch. - The external feedback you get when publishing spec proposals/updates. - Additional reputation in some parts of the more 'geeky' audience, which might help with marketing. - As your competitors don't seem to be convinced of this model, it might be another unique selling point 😅 — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#631 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ABc-aqzTRxpdQ4HoK4ZyhcNGVVzBDRyLks5vNahpgaJpZM4a8BgZ> .

[stevenroose]

If I understand correctly, this involves basically ditching our own existing E2E protocol (https://github.com/wireapp/proteus), moving to something like OMEMO, and implementing OMEMO support in all our clients.

Not necessarily. You can decide both ways: either (1) you keep using your own E2EE scheme and implement it as a proprietary extension (when using a good XMPP library, it should be trivial to wrap that scheme in an extension); this has the downside that users will only be able to use this E2EE with other Wire servers that know this module. Or (2) you decide that it's valuable for Wire users to be able to chat E2EE with non-Wire users as well and implement OMEMO. It's of course also possible to have both so that users with older Wire clients can still use the Wire E2EE with other Wire users, while users that updated their client can use OMEMO. It's even possible to use both depending on what your contact supports.

(Related: Like Proteus, OMEMO is also based on the Signal double ratchet E2EE protocol (Axolotl), it might be possible to make them compatible even.)

Then, whenever we want to improve our protocol, we need to publish a XEP so that other Jabber clients can implement this XEP (if they want to be able to talk with Wire users). Alternatively we might have our own protocol that we can iterate on, and keep OMEMO as a fallback for non-Wire users.

This very much depends on how much you want to give back to the community. Like I mentioned above, it's certainly possible to just craft your own extensions without having to share them. However all functionality that you want to be adopted by the wider XMPP community so that it is compatible with other servers will have to go through the XEP standards process. Most likely there will already be a XEP for most features you'd want to implement, though.

Also, note that I'm only talking about the server-to-server XMPP protocol. A lot of the XMPP complexity is in the client-to-server protocol, which you don't have to implement. It's perfectly reasonable to decide that Wire users should only use the official Wire app and it's OK to have your own c2s protocol between the Wire server and client.

[neongreen]

Also, note that I'm only talking about the server-to-server XMPP protocol. A lot of the XMPP complexity is in the client-to-server protocol, which you don't have to implement. It's perfectly reasonable to decide that Wire users should only use the official Wire app and it's OK to have your own c2s protocol between the Wire server and client.

Oh, that's interesting to know. I didn't realise this.

[weiss]

I disagree with many details; e.g., while you consider it "completely unacceptable" to not enforce E2EE "in 2019", for me it's unacceptable to have my communication rely on closed services that allow me to talk to their own customers only, in 2019. As E2EE is especially hard to enforce on open networks, there's a trade-off, so I don't think there's a "right or wrong" here. The difference is that E2EE can be implemented on an open network, while there's no way for us to interop with closed silos.

That said, I do agree with your general stance: Fragmentation can obviously result in annoying UX, and XMPP suffers from this. The main problem here is missing manpower (i.e., money) being invested into the ecosystem, hence only some implementations are currently up to par with modern (mobile) features, so users must pick the right combinations, which is not something they want to be bothered with. However, I disagree the protocol's modularity is the culprit here. Quite the opposite, the extensibility is essential for moving the protocol forward, i.e. for adapting to new and different use cases (such as today's mobile IM requirements) without breaking interop with the existing ecosystem. Due to feature negotiation, it's of course easy to check whether a certain functionality (such as push notifications) is available. (No, users shouldn't have to check this, the apps they install should just offer sane services by default.)

Anyway, this discussion seems quite off-topic to me, as this issue is not about talking anyone into another solution or comparing Wire to XMPP. Your preferences are different from mine, and that's obviously fine. It's just about whether it would be nice to open an existing solution for communication with others, so that the two of us could chat with each other (just like we could call us by phone). If enforcing E2EE is considered essential by one of the vendors involved, just don't offer a knob to disable it in your client. Or wouldn't that solve your problem?

[swedneck]

I just want to also bring https://matrix.org/docs/spec/ to attention, it's in my opinion better than XMPP.

[akuckartz]

it's in my opinion better than XMPP.

How?

[jonaharagon]

If enforcing E2EE is considered essential by one of the vendors involved, just don't offer a knob to disable it in your client. Or wouldn't that solve your problem?

What about incoming messages without E2EE enabled, or messages to XMPP users who’s clients don’t support E2EE? I’m not really familiar with XMPP or how it would handle those scenarios, but it doesn’t seem like the UX Wire users are looking for: seamless always-on E2EE. And I’d consider it unacceptable for E2EE to be disabled by either client in either scenario.

I’d imagine the Wire server federation being planned will work a lot more seamlessly than trying to force XMPP to act the way Wire expects it to. Otherwise why put all this work into Wire’s server and encryption in the first place?

[iNPUTmice]

The basic XMPP specification requires clients to be able to talk in cleartext.

Can you point me to the paragraph in the XMPP spec were it says clients MUST be able to send and receive plain text messages?