6.1Inter-system integration

When integrating multiple systems, simply "connecting them via an API" often fails to meet the requirements. A systems architect must judge which integration approach (REST API, ETL, EAI) to use, and whether to make it synchronous or asynchronous, based on constraints such as data volume, how real-time the processing must be, and differences in availability between the caller and the callee. Getting this judgment wrong leads to failures such as cascading response delays or unhandled temporary data inconsistency.

6.1.1REST APIs, ETL, and EAI

• REST APIs represent resources via HTTP methods (GET/POST/PUT/DELETE, etc.) and URLs, integrating systems loosely coupled. The caller and provider can be developed and deployed independently, and the general-purpose reachability from web/mobile clients is an advantage—but calling one record at a time tends to be inefficient for bulk data integration.
• ETL (Extract/Transform/Load) processes data from multiple systems in the order extract, then transform, then load, bulk-transferring it into a data warehouse or another system. It suits use cases that periodically consolidate large volumes of data, such as nightly batches, but offers low real-time responsiveness.
• EAI (Enterprise Application Integration) mediates heterogeneous in-house systems (accounting, inventory, HR, etc.) through a hub-like platform, so each individual system only needs to connect to the hub. This keeps the number of connections down compared to wiring each system to every other system point-to-point (a "spaghetti" of links), and changes to integration rules can be concentrated in the hub.

6.1.2Choosing synchronous vs. asynchronous integration

• Synchronous integration has the caller wait for the callee to finish processing and respond before moving on. It suits use cases where the result must be used immediately (e.g., confirming an order on the spot after checking stock), but comes with the trade-off that a delay or failure in the callee propagates directly into a delay or failure in the caller.
• Asynchronous integration has the caller move on to its next task without waiting once it has handed a request to a message queue or similar, receiving or polling for the result later. It decouples the caller from the callee's temporary delays or outages (improving loose coupling and fault tolerance), but there is a window between the request and when the result is reflected during which data is temporarily inconsistent (eventual consistency), and the business requirements must be checked to confirm this is acceptable.

Suppose a systems architect is designing the integration for an e-commerce site's order-confirmation processing and the point-award processing that follows an order. Because order confirmation must show the result of the stock check to the user on the spot and let the user decide whether to confirm, the call to the inventory system uses synchronous integration—confirming the order without waiting for the stock-check result would risk accepting an order for an out-of-stock item. The point-award processing, on the other hand, has a requirement that order confirmation itself must not be delayed even if the point-calculation service is temporarily congested or down, so the architect uses asynchronous integration, posting a point-award request to a message queue after the order is confirmed. This is acceptable because the business side has confirmed that it is fine (eventual consistency is acceptable) if points do not appear on screen immediately but are reflected a few seconds to a few minutes later. Had synchronous integration been chosen instead, a delay in the point-calculation service would have propagated directly into the customer-facing order-confirmation delay, translating into lost sales. For the daily sales-data feed into the existing accounting system, because a large volume of order data must be aggregated and transformed in bulk once a day and fed into the accounting system, the architect uses ETL rather than individual API calls, designing it as a batch process of extract, transform, and load. Judging the real-time responsiveness, data volume, and tolerable degree of inconsistency required by each process, and choosing the integration approach accordingly, is what the practice looks like.

6.1.3Section summary

• REST API = loosely coupled point-to-point integration, ETL = periodic bulk extract/transform/load, EAI = a hub keeps connection count down
• Synchronous integration = the result is usable immediately but delays propagate, asynchronous integration = improves loose coupling/fault tolerance but presupposes eventual consistency
• Choose the integration approach by working backward from real-time needs, data volume, and the tolerable degree of inconsistency