(2.12) Linearizable Message Get request

[MauriceVanVeen]

Support for read-after-write and monotonic reads was added in #6970. This allows you to guarantee these properties, given you have access to the minimum last sequence to observe all writes up to that point.

If you don't have access to this sequence, for example when receiving an event via a side channel (that doesn't contain a sequence) you can't use these mechanisms. This PR adds support for Linearizable reads through single-message get requests. An alternative method that doesn't require any knowledge about a sequence, but has the additional cost of requiring replication to ensure the data is most recent.

An overview of consistency levels provided depending on which settings are used:

• By default, a replicated stream is eventually consistent when using a direct get request. Especially if reads are served by a mirror stream. (Message get requests are technically answered by "the leader", but may still be observed as eventually consistent if an outdated leader from a previous term answers. Usually though, they'll be observed as linearizable, but not strictly guaranteed during restarts or leader elections.)
• Read-after-write and monotonic reads can be achieved (even on mirrors), by using the MinLastSeq constraint in either JSApiMsgGetRequest or ConsumerConfig. Requiring the server that answers reads or serves consumer data to contain all writes up to and including the specified sequence.
• Linearizability can be achieved by passing the Linearizable constraint in the JSApiMsgGetRequest. This ensures the most recent real-time data is returned, at the cost of: requiring replication when the stream is replicated, only allowing message get requests (not direct get), and the read to be served only by the stream leader.

Consumers only offer read-after-write and monotonic reads through the use of MinLastSeq. But consumers can be made to guarantee linearizability if the first request to a stream is done using the Linearizable constraint to get the last message of the stream with JSApiMsgGetRequest. This sequence can then be used in the consumer create request to guarantee linearizability regardless of on which replica it is placed.

The default should be used when eventual consistency is desired for highest possible scale and speed. Read-after-write and monotonic reads can be used when desiring a minimal level of consistency, ensuring a specified amount of writes can be observed, allowing for high scale and speed, but optionally needing to handle additional error conditions or minor speed reductions. Linearizability adds the highest level of consistency at the tradeoff of scale and speed, only the original stream leader can answer these message get requests. This flexibility lets user choose between scale, speed, and levels of consistency/constraints on the topology.

Signed-off-by: Maurice van Veen [email protected]

[derekcollison]

Have this queued up to take a look - just fyi.

[MauriceVanVeen]

Have this queued up to take a look - just fyi.

Thank you!

No real rush (also why it's in draft still), we'll need to discuss this design a bit further with the client team as well probably. Although the design (read-after-write/monotonic reads/linearizable) gives a lot of control and flexibility, we'll need to decide whether this adds a bunch of complexity which would be (too) hard to understand by an end-user, in which case we might need to revisit this.

Happy about your feedback on this as well, of course!

[MauriceVanVeen]

Closing for now, see also: #7167