What's changed: Initial: 5 sections for Domain 2 (implementation and integration)
2.2Model Deployment Strategies
Learn FM deployment matched to app needs: on-demand invocation with Lambda, Bedrock provisioned throughput, SageMaker endpoints, containers (GPU/memory/token throughput), and model cascading.
Deploying FMs has challenges beyond traditional ML inference (huge models, GPUs, token processing, cold starts). Choose the method by traffic profile and latency needs.
2.2.1Choosing a deployment method
- Lambda on-demand invocation: for sporadic/bursty traffic; call Bedrock on-demand, elastic and serverless.
- Bedrock provisioned throughput: for steady high throughput or strict latency SLAs; reserve capacity.
- SageMaker endpoints: host self-managed/custom models or hybrid setups; real-time/async/batch.
- Model cascading: route easy queries to small/cheap models and only hard ones to large models—balancing cost and quality.
Common: sporadic traffic = Lambda on-demand, steady high throughput / latency SLA = provisioned throughput, host custom models = SageMaker endpoints, cheap small model for easy queries = model cascading.
Decide deployment by the kind of FM and the traffic profile. Call Bedrock managed FMs on-demand (elastic/pay-per-use) or via provisioned throughput (reserved capacity, stable latency), exposing the app from Lambda/ECS/Fargate through API Gateway. Host self-managed/open-weight models or special runtimes on SageMaker endpoints (real-time/async/batch transform), designing containers for GPU utilization, memory, and token-processing capacity and using model-loading strategies to reduce cold starts. For cost, use model cascading (route easy queries to small models) and task-specific small pre-trained models. Rather than running one giant model at full capacity 24/7, the professional pattern is to be elastic with traffic and tier models by difficulty.
| Traffic/need | Choose | Why |
|---|---|---|
| Sporadic/bursty | Lambda on-demand | Elastic, pay-per-use |
| Steady high throughput | Provisioned throughput | Reserved capacity, stable latency |
| Custom/self-managed model | SageMaker endpoints | Real-time/async/batch |
| Balance cost and quality | Model cascading | Tier models by difficulty |
Trap: “always serve all traffic with provisioned throughput” is wrong—on-demand is cheaper for sporadic traffic; reserve capacity only for steady high load. Match the method to the requirement.
2.2.2Section summary
- Sporadic = Lambda on-demand / steady = provisioned throughput / custom = SageMaker
- Cost-optimal = model cascading (tier models by difficulty)
Sign in to track progress — Log in.
Quick check
(just a quick review)Q1. You want to handle sporadic Bedrock inference requests from an internal tool elastically without managing servers. Best option?
Q2. For a high-traffic production app, you need stable throughput on Bedrock that meets a strict latency SLA. Best option?
Q3. Most queries are simple but a few need advanced reasoning. Which deployment strategy keeps cost low while preserving quality?
Keep track of your progress
The full study guide is free to read. Sign up free to practice with the question bank, track what you have read, review your mistakes, and highlight passages.