2.3Web multi-tier architecture

When the number of users on a Web system surges, a system architect faces the question of whether simply adding more servers can absorb the load. Whether that question can be answered with an easy "yes" actually hinges on each server's stateless design. If a server holds per-client state (login information or in-progress work data) inside itself, simply adding more servers will not work correctly. This section covers the tiered structure of Web multi-tier architecture and the stateless design and session externalization that make scale-out possible.

2.3.1Presentation, application, and data tiers

• The presentation tier handles screen display and input (the web server/front end). The application tier executes business logic (the application server). The data tier handles data persistence and management (the database server). This is the Web-specific realization of the three-tier client/server concept, and each tier can scale and change independently.
• The biggest benefit of separating tiers is that tiers with different load characteristics can be scaled independently—for example, a surge in access may require scaling out the presentation and application tiers, but the data tier (especially a write-heavy RDBMS) is often hard to simply distribute horizontally, so the architect must identify which tier is likely to bottleneck and concentrate countermeasures there.

2.3.2Stateless design and session externalization

• Stateless design is a policy in which no server retains client-specific state (login status, cart contents, etc.) across requests. Because any server can correctly handle a request, the number of servers behind a load balancer can be freely scaled up or down (making scale-out easy). Conversely, if a server holds state statefully in its own memory, the same client must always be routed to the same server (session affinity), reducing the freedom to load-balance.
• Session externalization is a design technique in which the application server itself does not hold session information but instead offloads it to an external shared store (a session store), such as Redis. This makes each application server effectively stateless: even if a server is replaced due to a failure, another server can reference the session store and continue processing, preserving the freedom to scale out.

A system architect has been assigned to prepare a membership e-commerce site for a traffic surge—ten times peak traffic during a sale event. The current application servers hold logged-in users' session information (cart contents, login status) statefully, in each application server's own memory, and the load balancer is configured to always route a given user's requests to the same application server (session affinity). Simply adding more application servers under this design leaves a problem: a user starting a new session might land on a new server, but a user with an existing session stays pinned to their original server, so load keeps concentrating on specific servers. Furthermore, with session affinity in effect, if an application server fails and stops, the session information held in that server's memory is lost, forcibly logging out active users and losing their cart contents. To fundamentally solve both problems, the architect proposes migrating to a design that removes session information from application-server memory and externalizes it to a shared session store (such as Redis). This makes each application server effectively stateless, and the load balancer can now freely route each request to any available server, so during the sale, simply adding more application server instances mechanically handles the peak traffic. And if an application server goes down, because session information persists in the store, another application server can take over the session and continue processing, so users never experience a forced logout—an availability gain that comes along with the scalability gain. For the data tier (the RDBMS holding member and order data), on the other hand, because it is write-heavy it cannot be horizontally distributed as simply as the application servers, so the architect instead planned to add read-only replicas to distribute read load while separately estimating the write-heavy primary's performance through capacity planning—using a different scaling strategy for each tier.

2.3.3Section summary

• Web multi-tier architecture separates the presentation, application, and data tiers, letting each scale and change independently
• Stateless design plus session externalization lets any application server handle any request, making scale-out easy
• Because of its write-heavy nature, the data tier is hard to distribute horizontally and tends to bottleneck, requiring a different scaling strategy per tier