5.4Network architecture design

A system architect designs the network architecture—how to route traffic from users across a server fleet, where to place content, and how to switch paths during a failure—by weighing performance, availability, and cost. This section covers the difference in load balancing by layer, the roles of CDN and DNS, and judgment for redundant paths and latency optimization.

5.4.1Load balancing: L4 and L7

• L4 load balancing (transport layer) decides the routing destination by looking only at source/destination IP addresses, port numbers, and TCP/UDP connection information. Because it does not parse packet contents (such as HTTP headers), it is lightweight, fast, and delivers excellent throughput, but it cannot route flexibly based on application content, such as URL path or cookies (e.g., sending only a specific path to a different server group).
• L7 load balancing (application layer) decides the routing destination by parsing content up through the application layer—HTTP headers, URL path, cookies, and so on. It enables flexible routing, such as "route everything under /api/ to the API server group" or "route users with a specific cookie to a specific server version (for A/B testing, etc.)," but because it parses packet contents, it carries greater processing overhead and lower throughput than L4. A configuration that terminates SSL/TLS (decrypting encrypted traffic) at the load balancer is also often classified as L7.
• The judgment axis is "do we simply want to route traffic fast?" versus "do we need flexible control based on URL path or user attributes?" Simple routing where high throughput is the top priority favors L4, and cases needing flexibility such as path-based routing or A/B testing favor L7—the choice depends on the requirement (combining both is also common: a multi-tier setup where L4 does coarse distribution first, and L7 further routes by application behind it).

5.4.2CDN and DNS

• A CDN (Content Delivery Network) is a network that caches static content (images, video, JS/CSS, etc.) on edge servers geographically close to users and delivers it from there. By reducing the physical distance to the origin server (the true source of the content), it shortens communication latency, while also reducing direct access to the origin server, lightening the origin's load. Dynamically changing content (personalized order information, etc.) does not suit caching, and even via a CDN it must be designed to query the origin on every request.
• DNS (Domain Name System) is a hierarchical name-resolution mechanism that translates domain names into IP addresses. From an availability standpoint, making authoritative DNS servers redundant (and geographically distributed) eliminates a single point of failure. DNS responses can also return multiple IP addresses, or return the IP of a nearby location based on the user's geographic location (a GeoDNS-like mechanism), letting DNS also serve as an entry point for load balancing or failover during an outage. However, a DNS change is cached for the duration of its TTL (Time To Live) and can be slow to take effect, so relying on DNS alone as the means of immediate failover during an outage is impractical—in practice it is combined with the redundant paths discussed next.

5.4.3Redundant paths and latency optimization

• Redundant paths is a design where multiple paths are prepared so that service can continue even if a single communication path (a line or device) fails. Measures include contracting two or more circuits, dual-homing routers/switches with automatic failover to a detour path on failure, and placing the system across multiple data centers/regions so traffic is redirected to the other if one becomes unavailable. Avoiding a single point of failure (SPOF) is the crux of the design, and it is preferable to build in automatic failover at the path level rather than relying on DNS switching alone.
• Latency optimization is a design that shortens response time to users. Representative measures include reducing physical distance (edge placement via a CDN, placing the system in a region close to users), reducing the number of hops or congestion along the path, and lowering connection-establishment cost (reusing TLS sessions, connection multiplexing via HTTP/2 or HTTP/3). Note that latency measures not accompanied by capacity planning (estimating where bottlenecks will occur relative to expected access volume) can easily overlook congestion at peak times.

Suppose a system architect is designing the network architecture for a nationwide video-streaming service. For the load-balancing layer, they first considered a simple policy of routing all requests uniformly, but because the requirement called for flexible routing based on URL path—"route requests under /api/ to the dynamic API server group, and requests for other static assets to a separate server group"—they adopted L7 load balancing, which can parse packet contents. However, concerned that processing all requests through L7 alone would drop overall throughput due to parsing overhead, they achieved both throughput and flexibility with a multi-tier setup: L4 load balancing first does coarse distribution across multiple L7 load balancers, and L7 then finely routes by path behind it. Next, for the video files, which are static content, they judged that if users nationwide accessed a single origin server in Tokyo directly, latency would grow larger for geographically distant users, and the origin's load would also become excessive, so they introduced a CDN, caching video on edge servers in each region to achieve both shorter latency and reduced origin load. Meanwhile, dynamic content such as a user's viewing history or personalized recommendations does not suit caching on a CDN, so this was left querying the origin server each time. To further improve availability, they placed an origin replica in the Osaka region in addition to Tokyo, and built in redundant paths where DNS returning multiple IPs or geographic routing normally directs users to the nearest location, while automatically switching to the Osaka region at the path level if the Tokyo region fails. Here, aware that DNS's TTL-based caching means a switchover does not take effect immediately, they did not rely on DNS switching alone, layering failure detection and automatic failover on the load-balancer side as well. As this shows, the core of network architecture design is stacking measures per challenge: a multi-tier L4/L7 setup for routing-flexibility requirements, a CDN for geographic latency and origin load, and redundant paths plus defense-in-depth that accounts for DNS's limits for eliminating a single point of failure.

5.4.4Section summary

• L4 load balancing is fast using only IP/port but inflexible; L7 load balancing is flexible via application content but carries more overhead — choose per requirement, and a multi-tier setup is also common
• A CDN achieves both shorter latency and reduced origin load via edge caching; dynamic content does not suit caching
• Because a DNS switchover can be delayed by its TTL, eliminating a single point of failure requires combining it with path-level redundant paths (automatic failover)