Cirrus

Cirrus lets you manage your Snowflake node IDs.

Etymology

High-level clouds, called cirrus clouds, can reach heights of 20,000 feet (6,000 meters) and are typically thin. They do not produce rain and often indicate fair weather. They are usually made up of ice.

Not Production Ready

See TODO list below.

What's Snowflake?

Snowflake IDs are special 8-byte IDs generated by Twitter's Snowflake algorithm.

Each Snowflake ID is a 63 bit int.

• 41 bits for a timestamp (milliseconds since a custom epoch)
• 10 bits for a node ID (ranging from 0 to 1023)
• 12 bits for a sequence number (ranging from 0 to 4095)

Why Cirrus?

If you're planning on horizontally scaling (i.e., replicating) your app, you'll need a way of doling out node IDs.

In the era of Docker and Kubernetes, containers and pods are expected to be ephemeral. They can go down at any moment. If that happens, they have no way of relinquishing their node IDs, and so that node ID is essentially lost.

Fortunately, checking for "stale" IDs is super easy, so we can ensure we have a dynamic and reliable way of re-requisitioning and reallocating node IDs.

What does Cirrus do?

Cirrus maps node IDs to app IDs.

An app ID is a string that can be anything (e.g., an int, a UUID, a hash, etc).

A node ID is an int ranging from 0 to 1023.

Apps must periodically send heartbeats to Cirrus, otherwise their node IDs get requisitioned.

TODO

Questions

• How do we know if a node has actually died? What if it's just busy and stays alive? There might be a moment where 2 apps generate PKs with the same node ID? That would be disastrous.

Single point of failure

Cirrus needs to be able scale out so it's not a single point of failure.

We need a couple things:

• Replace AvailableNodeIds channel with a distributed queue system (e.g., RabbitMQ, Cherami, etc)
• Replace Heartbeats map with a distributed store (e.g., etcd)

Usability

• Usage section in README
• Configurable durations
• Dockerfile
• Helm chart