3.4Load balancing and content delivery

Load balancing is the foundational technique for "distributing work across multiple servers to lower the load per server and raise availability," but in practice the key design decisions are "at which layer (L4 or L7) should traffic be balanced" and "which distribution method fits the requirement." This section systematically covers load balancer mechanics and content delivery optimization via CDN and reverse proxies—the knowledge needed to make those decisions.

3.4.1L4 and L7 load balancers

• An L4 load balancer distributes traffic using only transport-layer information (TCP/UDP IP addresses and port numbers). Because it does not inspect packet contents (application data), it is fast and low-latency, well suited to handling a large number of connections at high throughput.
• An L7 load balancer inspects application-layer content (HTTP headers, URL, cookies, etc.) to make its distribution decision. It enables advanced content-aware control such as routing by URL path (e.g., /api to the API server pool, /static to the static-content server pool) and session persistence based on cookies, but the inspection overhead means higher processing load and latency than L4.

3.4.2Distribution methods and health checks

• Round robin is the simplest method, distributing requests to each server in turn, equally. It works well when server capacity is uniform, but load can pile up on a particular server if long-running requests happen to cluster there.
• Least connections preferentially routes to the server currently handling the fewest active connections. When request processing time varies, it distributes load more in line with actual server load than round robin does.
• Weighted distribution assigns a weight to each server based on its processing capacity (spec differences), routing more requests to more capable servers. It is combined with round robin or least connections (e.g., weighted round robin).
• A health check is the load balancer periodically probing each server (ping, TCP connection check, HTTP response check, etc.) and automatically excluding any server that fails to respond from the distribution pool. It prevents requests from piling up on a failed server and preserves availability.

3.4.3Session persistence, reverse proxy, CDN, and NTP

• Session persistence (sticky sessions) always routes requests from the same client to the same server. It is needed when a server holds session state locally (e.g., login/cart state), but has the trade-off of tending to concentrate load on a particular server.
• A reverse proxy receives client requests and relays them to internal servers. A CDN (Content Delivery Network) caches static content (images, video, JS/CSS, etc.) on edge servers geographically close to end users, reducing load on the origin server and reducing latency.
• NTP (Network Time Protocol) synchronizes the clocks of network devices and servers with an NTP server. Time synchronization is essential for correlating log timestamps, judging certificate validity periods, and maintaining consistency in distributed systems.

Suppose an EC site's infrastructure lead is asked to review the current setup (an L4 load balancer doing round-robin distribution). The requirement is: "route API requests under /api to a pool of high-performance API servers, and static content such as /images to a separate pool of lightweight servers." Because routing by URL path requires inspecting HTTP header content, this cannot be done at L4, and switching to an L7 load balancer is the necessary judgment. Next, if the design holds a logged-in user's cart information in the application server's memory, routing each request to a different server would fail to find the cart data—so session persistence (sticky sessions) must be implemented using the L7 load balancer's cookie-inspection capability. However, if a particular heavy user's session stays pinned to one server indefinitely, load can become imbalanced, so a more desirable long-term design is to move session state to an external shared store such as Redis, moving toward "statelessness" where any server can look up the state regardless of which one handles the request. Further, for static content like images and video, deploying a CDN to cache content on geographically nearby edge servers avoids repeated access to the origin, improving both origin load and perceived user speed. Finally, to accurately correlate logs across multiple servers by timestamp when analyzing the effect of this load-balancing redesign, all servers' clocks must first be synchronized via NTP. Load-balancing design thus starts with "what criterion to route on (L4 or L7)" and requires consistent judgment through session management, content delivery, and time synchronization.

3.4.4Section summary

• L4 is fast, using only the transport layer; L7 inspects the application layer for advanced routing (URL path, cookies) but is slower
• A health check automatically removes failed servers; session persistence trades off against load balance
• A CDN caches static content at the edge; NTP synchronizes clocks across servers to enable log correlation