An update on Redis Streams developmentI saw multiple users asking me what is happening with Streams, when they’ll be ready for production uses, and in general what’s the ETA and the plan of the feature. This post will attempt to clarify a bit what comes next.
To start, in this moment Streams are my main priority: I want to finish this work that I believe is very useful in the Redis community and immediately start with the Redis Cluster improvements plans. Actually the work on Cluster has already started, with my colleague Fabio Nicotra that is porting redis-trib, the Cluster management tool, inside the old and good redis-cli. This step involves translating the code from Ruby to C. In the meantime, a few weeks ago I finished writing the Streams core, and I deleted the “streams” feature branch, merging everything into the “unstable” branch.
Later I reviewed again, several times actually, the specification for consumer groups. A few weeks ago I finally was happy with the result, so I started the implementation of this specification: https://gist.github.com/antirez/68e67f3251d10f026861be2d0fe0d2f4. Be aware that command names changed quite a bit… With the API being more like this: https://gist.github.com/antirez/4e7049ce4fce4aa61bf0cfbc3672e64d.
I’m halfway, after the first 500 lines of code I today was able to see clients using the XREADGROUP command to see only their local history when fetching old messages. This was a milestone in the implementation. Also XACK is now available. Basically all the data structures supporting the streams consumer groups are working, but I need to finish all the commands implementations, support the blocking operations in XREADGROUP and so forth, handle replication, RDB and AOF persistence. More or less in one month I should have everything ready.
Because of consumer groups, the RDB format of the Streams is going to change a lot compared to what “unstable” is using right now. And since there is to break compatibility, we’ll also add the ability of RDB files to persist information about keys last access, so that a Redis restart (or slave loading) involving RDB will get all the info needed in order to continue doing the eviction of keys in the correct way. Right now all that information would be lost instead. Because of this radical RDB changes the plan changed a bit: Streams are going to appear in Redis 5.0 and not 4.0, but 5.0 will be released as GA in about two months: this is possible because 5.0 will be just a 4.0 plus streams, no other major features inside, if not for the RDB changes…
So TLDR: I’m working to Streams almost full time, if not for some time to check PRs / Issues for critical stuff, and in two months if you upgrade to Redis 5.0 you’ll have production ready streams with consumer groups fully implemented. There is also quite of a documentation effort to perform, I’ll write both the command manuals and an extensive introduction to streams once we go RC1, which is in about one month, so that people will have documentation to support their tests during the release candidate stage.
I hope this clarifies what’s happening with Streams. See you soon with Redis 5.0 :-) Comments