Redis Cluster / Redis Cluster Master-Slave Model: The Ultimate Architecture of Redis

The Redis cluster is the ultimate architecture of Redis. It allows for horizontal scaling of Redis.

In Redis cluster, we decide to spread the data we are storing across multiple machines, which is known as Sharding. So each such Redis instance in the cluster is considered a shard of the whole data.

The Redis Cluster uses algorithmic sharding. To find the shard for a given key, we hash the key and mod the total result by the number of shards. Then, using a deterministic hash function, meaning that a given key will always map to the same shard, we can reason about where a particular key will be when we read it in the future.

Redis Cluster Architecture in System Design

To handle further addition of shards into the system (resharding), the Redis cluster uses Hashslot, to which all of the data is mapped. Thus, when we add new shards, we simply move hashslots from shard to shard and simplify the process of adding new primary instances into the cluster. And to the advantage, this is possible without any downtime, and minimal performance hit. Let’s look at an example below:

Consider the number of hashslots to be 10K.
Instance1 contains hashslots from 0 to 5000
Instance2 contains hashslots from 5001 to 10000.

Now, let’s say we need to add another instance, now the distribution of hashslots comes to,

Instance1 contains hashslots from 0 to 3333.
Instance2 contains hashslots from 3334 to 6667.
Instance3 contains hashslots from 6668 to 10000.

What is gossiping in the Redis cluster?

To determine the entire cluster’s health, the redis cluster uses gossiping. In the example below, we have 3 main instances and 3 secondary nodes of them. All these nodes constantly determine which nodes are currently available to serve requests. Suppose, if enough shards agree that instance1 is not responsive, they can promote instance1’s secondary into a primary to keep the cluster healthy. As a general rule of thumb, it is essential to have an odd number of primary nodes and two replicas each for the most robust and fault-tolerant network.

Redis and its role in System Design

Redis is an open-source, in-memory data structure store used as a database, cache, and message broker. It is widely used for its fast performance, flexibility, and ease of use. 

  • What is Redis
  • Redis Data Types
  • Benefits of using Redis
  • Working Architecture of Redis
    • 1. Single Redis Instance
    • 2. Redis HL (High Availability)
    • 3. Redis Sentinel
    • 4. Redis Cluster / Redis Cluster Master-Slave Model
      • What is gossiping in the Redis cluster?
  • Types of Redis Persistence Models
    • 1. RDB (Real-time Data Base) Persistence Model:
      • Snapshotting in RDB
      • Advantages of RDB(Real-time database)
      • Disadvantages of RDB(Real-time database)
    • 2. AOF (Append-Only File) Persistence Model
      • How AOF works?
      • Advantages of AOF
      • Disadvantages of AOF
    • 3. No Persistence Model
    • 4. Hybrid (RDB+AOF) Persistence Model
  • Availability, Consistency, and Partitioning in Redis
  • Can we use Redis as an alternative to the original DB?
  • Conclusion

Similar Reads

Redis Data Storage Types

Redis allows developers to store, retrieve, and manipulate data in various data structures such as strings, bitmaps, bitfields, hashes, lists, sets, sorted sets, geospatial, hyperlogs, and streams....

Benefits of using Redis

All Redis data resides in the server’s main memory, in contrast to databases such as PostgreSQL, SQL Server, and others that store most data on disk. Redis can therefore support higher orders of magnitude of operations and faster response times. Thus, it results in super-fast performance with average read and writes operations taking less than milliseconds, and thus accordingly supports millions of operations per second....

Working Architecture of Redis

There are several Redis architectures, depending on the use case and scale:...

1. Single Redis Instance

This is the most straightforward Redis deployment. It involves users setting up and running small instances that can help them grow and speed up their services. However, it has its own drawback, as all calls made to Redis would fail if this running instance crashes or is unavailable. Thus there is a degradation in the overall performance and speed of the system....

2. Redis HA (High Availability)

Another popular setup is the main deployment with a secondary deployment that is always kept in sync with the replication. The secondary instances can be one or more instances in our deployment, which helps in scale reads from Redis, and provide failover in the case when the main is lost....

3. Redis Sentinel

Sentinel corresponds to a distributed system. It is designed in a way where there is a cluster of sentinel processes working together for coordination of state to provide constant availability of the Redis system. Here are the responsibilities of the sentinel: Monitoring: Ensuring main and secondary instances are working as expected. Notification: Notify all the system admins about the events occurring in Redis instances. Management during failure: Sentinel nodes can start a process during failure if the primary instance is not available for long enough, and enough nodes agree that it is true....

4. Redis Cluster / Redis Cluster Master-Slave Model: The Ultimate Architecture of Redis

The Redis cluster is the ultimate architecture of Redis. It allows for horizontal scaling of Redis....

Types of Redis Persistence Models

Redis provides two main persistence options to save data to disk: RDB and AOF. Both options have their own advantages and disadvantages, and which one to use depends on the specific needs of the application. Given below are the several persistence options listed:...

1. RDB (Real-time Data Base) Persistence Model:

RDB is a point-in-time snapshot of the Redis dataset stored as a binary file. The RDB file contains a representation of the dataset at a particular point in time and can be used to restore the dataset in case of a server crash or restart. RDB is very efficient in terms of disk space usage and performance, as it uses a binary format to store the data....

2. AOF (Append-Only File) Persistence Model

AOF logs all write operations to a file in a human-readable format. This file contains a record of all the write operations performed on the dataset since the last save, making it possible to reconstruct the dataset in case of a crash. AOF provides better durability than RDB, as it logs every write operation to disk....

3. No Persistence Model

Redis also provides an option to disable persistence altogether, in which case the data is stored only in memory. This option is useful when Redis is used as a cache, and the data can be regenerated if lost....

4. Hybrid (RDB+AOF) Persistence Model

Redis provides an option to use both RDB and AOF persistence together, which is known as hybrid persistence. This option provides the benefits of both RDB and AOF, as the AOF log is used to replay write operations after a restart, and the RDB snapshot is used to restore the dataset at a specific point in time....

Availability, Consistency, and Partitioning in Redis

Here’s a brief overview of how Redis handles availability, consistency, and partitioning:...

Can we use Redis as an alternative to the original DB?

Based on the above discussion, Redis seems to be a better option for the original DB as it provides faster retrievals. Even then Redis is not used as a primary option for the database in the system....

Conclusion

Overall, Redis is a powerful tool for system design, but it may not be suitable for all use cases. It is important to carefully consider its limitations when deciding whether to use Redis in a particular application....

Contact Us