Ever since Redis Modules were released into the wild, I wanted to write something nice and short and see how easy and fun it is to significantly extend Redis.

It also helps that Dvirsky, my friend, works at Redis Labs and wrote RediSearch - a kick ass full text indexing and search engine that kicks all the other search engines’ performance ass (you should definitly try it out).

For a while now, to try out new languages/frameworks/whatever I’ve been using Twitter’s Snowflake. This case was no different as everyone needs unique ids at some point.

A bit about Snowflake - it’s a network service (micro service) that generates unique ID numbers at high scale with a few guarantees.

Some of these guarantees include:

  • Uncoordinated - multiple Snowflake services running in parallel do not need to communicate with each other to generate unique ids.
  • Time ordered (sort of) - The IDs are based, among other things, on the current time (make sure all the services run on machines with NTP) and that generates Ids that are k-sorted within a reasonable bound (1 second)
  • Directly Sortable - the ids can be sorted since it is guaranteed that an older ID is smaller than a newer ID (up to 1 second). You don’t need the complete dataset to correctly sort a bunch of Ids.
  • Compact - Ids are kept at 64bit size compared to other unique IDs such as GUIDs which are 128bit.

Read more about Snowflake here

Writing Redis modules is rather easy. For best performance you’ll need to write it in C.

I’ve found a great C implementation of Snowflake written by Dwayn (thanks Dwayn) and wrapped it with the relevant Redis Module parts.

Get the module here, build it and play with it.

The nice benefits of using it are:

  • No additional service - if you already have Redis running you can utilize it instead of adding another service to the system. You won’t need to add discovery or extra configuration as it will all just work using the same way you use to discover and connect your redis instances.

  • Get all the Snowflake benefits for free - you get all the above benefits for free such as no-lock ID generation, time ordered, directly sortable and all of that in a nice 64bit compact version.