Circuit Breaker is a software design pattern used in software development especially in Microservices based software development. Circuit breaker works as same as how an electrical circuit breaker works. When multiple services
communicatewith each other to serve the number of request, there are many possibilities that the services are unavailable and or not responding state. So, when service-A is busy trying to communicate the other ‘unavailable’ Service-B, Service-A can also go ‘unavailable’. Similarly, all the service which is trying to communicate ‘unavailable’ service will go cascade unavailable. In this article, let’s discuss, what is Circuit Breaker Design Pattern and how this will solve the above issue.
Problem statement Explanation
We all know that Microservices Design pattern and service-oriented architecture is having more advantage over the monolithic design pattern. Like, High availability, Easy deployment, Good Fault tolerance. But at the same time, this microservice and service-oriented design pattern have the biggest drawback that it will get into the cascade failure loop as service may be in a
Assume a Service oriented software application and following scenario
- We have four different services namely Service A, B, C & D.
- Because of the Client request, Service A will connect with Service B and B to C and C to D.
- But, Service D failed due to some reason.
- So, Connections made to Service D by other Services will get to wait and unresponsive service D will remain in the same state.
- Therefore, Service C will continuously retry to make the connection with Service D. As so, Service B and Service A.
- This will cause Cascade failure and entire application, or the part of service will go unavailable
Solution for Cascade failure
Assume the same scenario mentioned above. Additionally, we are going to introduce a proxy like a system called Circuit breaker. This Circuit Breaker will act as same as the electrical circuit breaker. This means when the number of consecutive failures reaches the defined threshold level and the connection thread of the services will immediately break. After the defined time, The Circuit breaker will allow a
So, Continuing with the solution to the above scenario.
- All the connections between services will go through Circuit Breaker.
- When Service D fails and defined the
maximumnumber of failures is reached Circuit breaker for Service C, Circuit Breaker will trip, and connection will immediately stop.
- Then the Opened Connection will slowly start communicating with limited allowed number of connection to test the stability of unavailable service.
- If the test connections are succeeded, then, communication across the Services will resume normally.
- If the Connections are failed, then Circuit Breaker will create Monitor & Alert event or if we have any Self-healing implementation, will be triggered.
Reference Architecture of Circuit Breaker.
Let’s discuss a simple workflow or Architecture of the Circuit Breaker design pattern. The Circuit Breaker should have three
- Circuit Breaker – Closed – Allow Connection through
- The Circuit Breaker – Open – Break the connection
- Circuit breaker – Limited Closed – Allow Limited Number of connection to Test
When Connection request occurs, it will cross Circuit Breaker first.
- If the connection gets succeeded, Circuit Breaker will remain closed and allow the communication.
- When Connection fails and the number of attempts has not exceeded the limit, then the Circuit Breaker will remain closed and it will allow the communication.
- If Connection fails and the number of attempts exceeds the limit, then the Circuit Breaker will trip, and it will open. So, all the connection request will be stopped.
- Once the Circuit breaker Opened, then it will try to enable the Limited retry for connection. Which means, it will start allowing a defined limited number of retry to test the connection status.
- The Circuit Breaker will close and enable further communication when the retrying connections are getting succeeded.
- If still the connection is getting failed, then the Circuit Breaker will move to open state and trigger the Monitoring and Alert system, or it will trigger the self-healing or preventive mechanism.
Example of Circuit Breaker Implementation
Let’s see an example of a circuit breaker in python. There is an awesome module available in python to implement Circuit Breaker called “
As the first step, we have to create an instance of the Circuit Breaker.
import pybreaker db_breaker = pybreaker.CircuitBreaker(fail_max=5, reset_timeout=60)
So, In this instance, we have defined the maximum number of failure as 5 times and the reset timeout as 60 seconds.
To introduce this Circuit breaker to the actual logic, we are going to use python decorator.
@db_breaker def example_db_operation(var): # Your Logic goes here pass example_db_obj = example_db_operation(pass_var)
Similarly, if we are not going to use python decorator, then,
def example_db_operation(var): # Your Logic goes here pass example_db_obj = db_breaker.call(example_db_operation, pass_var)
So, if the method “example_db_operation” fails continuously for 5 times, then Circuit breaker will automatically open and stop all the communication to “example_db_operation”.
There is another module in python which implements circuit breaker is “circuitbreaker”.
The Circuit breaker is the design pattern. So, it does not mean that we have to use the one and only suggested a way to implement it. Circuit Breaker can be implemented in any platform or any language. Especially, where two services or instance or physical network communicate with each other. It is very useful especially when you have multiple services rely on each other like microservices. It’s about your creativity to use Circuit breaker design pattern in
In this article, we have discussed what is Circuit Breaker Design Pattern and example implementation of it. In out upcoming articles, we will discuss more on Circuit Breaker and other microservice implementation patters. Stay tuned and subscribe DigitalVarys for more articles and study materials on DevOps, Agile, DevSecOps and App Development.
Experienced DevSecOps Practitioner, Tech Blogger, Expertise in Designing Solutions in Public and Private Cloud. Opensource Community Contributor.