Software Architecture Design - Load Balancing

Hi, I am Ashraf Atef with you today trying to give a good explanation for the Load Balancing concept which is of the core concepts in Software Engineering and you may use or will use or even heard about it in meetup or something.

Table of Content

• What is Load Balancing
• History of the load balancer
• Load Balancer Types
• Load Balancers Algorithms

What is a load balancer?

let us start with a story of someone providing some service and creating a center for support and complaints and start with a single employee to respond to customers who come to the center, but as service is coming popular and his customers exponentially increased one customer service employee won't be enough to receive these visitors so he hires a complete team for customer support to handle all these visitors with this team he hires someone to manage/control/distribute any new visitor will go to which customer service employee, there are a lot of methodologies(algorithms) this person can use to manage this load of visitors. This person called Load Balancer

I know it seems that I picked the wrong example for controlling traffic load :D But load balancer is a Traffic Cop

So, How can we technically define a Load Balancer?!!

The load balancer is Balancing the load :D

Really I am not joking, it's responsible for distributing/controlling/dividing user concurrent requests load that sent to your service. The beauty of load balancing is that is not between user request and your web service only but also it can be between two inner components in your system

• Between user requests and web service instances
• Between web service and database instances
• Between internal application layers

Reader: it seems that you are talking about distributed systems so it's a little bit modern concept, right? Ashraf: No, let me tell you a little about load balancing history

History of Load Balancer

I was surprised while I am doing my research that the load balancer is from the 90s exactly 1996. of course not like today but the idea is the same.

This time was on the hardware level, there were devices for load balancing. At first, it starts with DNS load balancing, wait what ?! how that works?

when you enter in your browser example.com the DNS returns a list of servers' IPs that map to servers that have your app instances running on it. for example return [IP_1, IP_2, IP_3], another user hit the same URL the DNS will return the same array but in a different order [IP_2, IP_3, IP_1] and so on.

For sure that was revolutionary and make the world a better place but it has critical drawbacks, for example, the DNS load balancer doesn't know anything about server status If it is down it will redirect the user to nothing!! , or even load on each server and the processing time rate for the server, so for example maybe this happen -> the user machine will cache the IP address of the website but the server which holds the website instance is not live anymore which is bad behavior

Despite these limitations but companies still use it because it is less expensive :)

Don't worry that is not the only type there are other types used in meanwhile

Load Balancers Types

• DNS Load Balancer

We already discussed it above.

• Hardware Load Balancer

Hardware load balancers are hardware devices that have specific operating systems that work to distribute web application traffic to a cluster of servers, which increases the performance of the application availability.

Hardware load balancers are put in hardware racks in data centers. So this type needs special support from the IT team to maintain these devices.

But if you have for example a certain time you know you will have peak traffic (you will provide offers for a certain period) so you will need to provide hardware load balancers that handle this traffic but after the peak traffic ended, you won't need them anymore that is wasting of resources, it is not flexible or auto grow on demand. Here comes Software Load Balancers.

• Software Load Balancer

Software Load Balancer is software you can install on any hardware or virtual machine, it can be customized based on the traffic of your web application.

There are a lot of providers that you can use for LB. Software load balancers consider different parameters in their decision-making like content hosted by the servers, cookies, HTTP headers, CPU and memory utilization, load on the network, and so on to route traffic across the servers.

Load Balancer Algorithms

1- Round Robin

This algorithm makes sure that make all servers have equal requests. loop on all servers sequentially when the load balancer receives a request. this algorithm doesn't take care of any servers' status or parameters.

2- Weighted Round Robin

In this algorithm, the load balancer takes into account the server's status about how this server handles traffic load, based on this parameter weights are assigned to servers and traffics is routed to them based on that. Servers who can handle the high traffic load will take more traffic requests.

3- Least connections

There are two approaches for implementing this algorithm :

load balancer based on server opened connections will route traffic requests, servers who have a smaller number of opened connections will receive the requests because it is ready and can handle more requests.

the load balancer in this approach takes into account CPU utilization and processing time of servers. server with the shortest processing time with the smallest CPU utilization with the least opened connections will be the perfect server to handle coming requests.

4- Random

In this approach, the load balancer route requests randomly to servers, even in case take servers' parameters and all are equal then it will send it randomly

5- Hash

In this algorithm, the source client IP and request URL hashed to make any request that comes from this user will be routed to a certain server. This approach helo in reducing the redundancy of users' cached data across different servers, and make the user be able to re-establish the connection to the same server if the connection dropped.

Conclution

As we saw through this article about the importance of load balancers, how load balancing started, how it works, and algorithms you can apply in the load balancer.

All information in this article is based on my understanding and research so if I miss something don't hesitate to leave a comment and I will update the article