System Design FAQ: Top Questions

Load Balancing is the process of distributing incoming network traffic across multiple servers to ensure no single server becomes a bottleneck. It enhances system scalability, availability, and fault tolerance by preventing overloads and enabling redundancy.

🎯 Why Load Balancing is Important

• Ensures High Availability: Redirects traffic when one instance fails.
• Improves Performance: Prevents overloading and enables optimal resource usage.
• Enables Scalability: Allows traffic to be split as more servers are added.

🧠 Types of Load Balancers

• Layer 4 (Transport): Operates on TCP/UDP (e.g., AWS NLB)
• Layer 7 (Application): Works with HTTP headers, cookies, etc. (e.g., NGINX, AWS ALB)

⚙️ Common Load Balancing Algorithms

• Round Robin: Sends each request to the next server in a loop
• Least Connections: Sends traffic to the server with the fewest active connections
• IP Hash: Ensures user sessions are sticky to a backend
• Weighted Round Robin: Prioritizes powerful servers more frequently

🔧 Sample NGINX Load Balancer Config (Round Robin)

http {
    upstream app_servers {
        server backend1.myapp.com;
        server backend2.myapp.com;
        server backend3.myapp.com;
    }

    server {
        listen 80;
        location / {
            proxy_pass http://app_servers;
            proxy_set_header Host $host;
            proxy_set_header X-Real-IP $remote_addr;
        }
    }
}
        

🔐 Sample HAProxy Config (Least Connections)

frontend http_front
    bind *:80
    default_backend http_back

backend http_back
    balance leastconn
    server app1 192.168.1.101:80 check
    server app2 192.168.1.102:80 check
        

☁️ Cloud-Based Load Balancing

• AWS ALB: Application-level with host/path-based routing, SSL termination
• GCP Load Balancing: Global HTTP(S) load balancer
• Azure Application Gateway: Layer 7 WAF-enabled load balancer

🧪 Monitoring Metrics

• Request per second (RPS)
• HTTP errors (4xx/5xx)
• Latency and response time
• Active connections per backend

📦 Best Practices

• Use health checks to detect and remove unhealthy servers
• Enable sticky sessions only when necessary (e.g., for legacy auth)
• Terminate SSL at the load balancer for centralized certificate management

🚫 Common Pitfalls

• No health checks = downtime when backend fails
• Overusing sticky sessions = poor load distribution
• No monitoring = unable to detect overload or failure early

📌 Final Insight

Load balancing is more than just splitting traffic — it’s about resiliency and efficient routing. Whether you use open-source tools like NGINX/HAProxy or cloud-native offerings, always validate with metrics and prepare for failover.