I have set up 3 Sentinel Redis servers where I have one main Redis instance and 2 secondary Redis instances. Sentinel enables Redis secondary instance to get promoted to main on when the main instance goes down. I have two questions around this setup:
My application should always be communicating with the main instance right?
All three Redis servers have different IP addresses. If my main redis server goes down, I will need to do several checks:
a. The first one will acknowledge that the redis server is down and can’t process requests (i.e my request will fail)
b. I will need to hit one of the other two servers to check which one is the main one.
c. Send the request to the main redis instance
Is there a simpler way around this?
Yeah, if a write fails, I need to call this API “Get Master Addr By Name” and for this I would need to ping one of the other servers. Once I get the the Address, I would then send the request to the Address. These are three network calls to the Redis cluster. Is there a simpler way?
You could load balance your reads across replicas with something like ‘read.my-redis-cluster01’ and your writes with ‘write.my-redis-cluster01’ where the load balancer is responsible for routing to the active leader. Sentinel can notify about problems with redis instances.
But if you want to talk directly with the cluster I think your only option is to query for the active leader either before you make a write request or after write failure and retry.
It depends on your environment, setting up proxy/LBs might be overkill, if you are running containers then an ambassador might be nice but could also be overkill as well, the simplest lowest effort would probably be a decorator for your write function if your Redis client doesn’t provide any leader discovery features.
