What's changed: In-scope coverage: deployment/runtime/IaC services
3.3Safe Model Deployment Strategies
Understand low-risk model deployment: blue/green, canary/linear traffic shifting, and shadow/A-B testing (multi-variant endpoints).
Switch to a new model gradually and with monitoring, so you can roll back fast on issues. SageMaker supports these strategies natively.
3.3.1Deployment strategies
- Blue/Green: stand up old/new fleets and switch traffic; rollback is just switching back—easy.
- Canary/Linear: shift traffic gradually, watch metrics, and auto-rollback on alarm.
- Shadow/A-B test: compare without impacting prod (shadow), or compare models on a multi-variant endpoint (A-B).
Model updates follow "gradual, monitored, instantly reversible." SageMaker supports endpoint updates as deployment guardrails: blue/green (stand up old/new fleets, switch all-at-once or in steps, revert by switching back), canary (send a small slice to the new model first, then expand if healthy), and linear (increase by a fixed percentage at intervals). All support auto-rollback via CloudWatch alarms and a baking period (monitoring wait). To observe a new model without impacting production, use shadow testing (duplicate production traffic to the new model but discard its responses); to compare models on live traffic, use A-B with production variants (multi-variant). After deployment, Model Monitor watches data/model-quality, bias, and feature-attribution drift, triggering retraining or rollback on degradation. The axes: "gradual + auto-rollback = canary/linear," "all-at-once + easy revert = blue/green," "no-impact validation = shadow," "live comparison = A-B."
| Requirement | Strategy |
|---|---|
| Gradual + auto-rollback | Canary / Linear |
| All-at-once + easy revert | Blue/Green |
| Validate without prod impact | Shadow testing |
| Compare models on live traffic | Production variants (A-B) |
Scenario: ship a new model safely. First shadow test by duplicating production traffic to validate response quality and latency without impact. If healthy, canary shift 5%→25%→100%, with CloudWatch alarms (error rate/latency) and a baking period for auto-rollback as a safety net. If you need to judge against the old model, compare with production variants (A-B).
Q. Gradual + auto-rollback? Canary/linear. Q. All-at-once + easy revert? Blue/green. Q. No-impact validation? Shadow testing. Q. Compare on live traffic? Production variants (A-B). Q. Detect post-deploy degradation? Model Monitor.
Watch the mix-ups: (1) Shadow testing doesn’t return responses to users (A-B actually serves some users)—different purposes. (2) Auto-rollback needs CloudWatch alarms + a baking period configured. (3) Even canary/linear miss issues if the monitored metric is wrong. (4) Blue/green temporarily costs for the extra new fleet.
Common on MLA: gradual shift with auto-rollback = canary/linear, instant switch, easy rollback = blue/green, validate without prod impact = shadow test, compare models on live traffic = multi-variant (A-B).
3.3.2In-scope services for deployment, runtime, and IaC
For inference runtime, serverless is AWS Lambda, containers are Amazon Elastic Container Service (ECS, minimal ops with Fargate) or Amazon Elastic Kubernetes Service (EKS, Kubernetes-standard), and images live in Amazon Elastic Container Registry (ECR). Public exposure uses Amazon API Gateway (HTTP API with auth/throttling), and on-prem connectivity uses AWS Direct Connect. Codify infrastructure and releases: IaC with AWS CloudFormation (declarative templates) and the AWS Cloud Development Kit (CDK, programming language); dependency packages with AWS CodeArtifact; builds with AWS CodeBuild; controlled deploys with AWS CodeDeploy. Self-service distribution of approved environments uses AWS Service Catalog, reusable serverless apps use AWS Serverless Application Repository, and distributed tracing uses AWS X-Ray.
3.3.3Section summary
- Reduce risk with blue/green, canary/linear, shadow/A-B
- Auto-rollback via metric monitoring + alarms
- Runtime = Lambda/ECS/EKS/ECR; IaC = CloudFormation/CDK; CI/CD = CodeArtifact/Build/Deploy
Sign in to track progress — Log in.
Quick check
(just a quick review)Q1. You want to gradually shift traffic to a new model and auto-rollback if metrics worsen. Which fits best?
Q2. You want to validate a new model on production traffic without impacting it. Which fits best?
Q3. You want to compare two models on live traffic simultaneously. What in SageMaker do you use?