What's changed: Initial: 3 sections for Domain 4 (operational efficiency and optimization)
4.1Cost Optimization and Resource Efficiency
Learn strategies to reduce FM cost: token efficiency (context optimization, compression, pruning), model tiering, prompt caching/semantic caching, batch inference, and provisioned throughput optimization.
FM cost is driven mainly by token volume. Cut input/output tokens, avoid unnecessary calls, and tier models by difficulty to lower cost while preserving quality.
4.1.1Levers to reduce cost
- Token efficiency: optimize the context window, compress prompts, prune context, and limit response size to cut tokens.
- Model tiering: route by query complexity between small/cheap and large models (choose by price-performance).
- Caching: avoid unnecessary calls with prompt caching (reuse common prefixes) and semantic caching (reuse results for semantically identical queries).
- Batch/reserved: process bulk cheaply with batch inference, optimize provisioned throughput for steady load, and monitor utilization with Auto Scaling.
Common: reuse results for same/similar queries = semantic caching, skip recomputing common prefixes = prompt caching, cheap bulk async = batch inference, small model for easy queries = model tiering, cut tokens = context optimization/compression.
Cost optimization multiplies four levers: fewer tokens, fewer calls, the right model, and the right purchase mode. Cut tokens via context optimization (pass only needed evidence), prompt compression, context pruning, and response-size limits. Cut calls with semantic caching (reuse results for semantically identical queries) and prompt caching (skip recomputing common prefixes like system prompts), using result fingerprints/deterministic hashes as cache keys. Optimize price-performance with model tiering (easy → small, hard → large). For purchase mode, use batch inference (cheaper) for bulk/async, optimize provisioned throughput capacity to utilization for steady high load, and keep on-demand for bursts. Monitor anomalous spend with AWS Cost Anomaly Detection and Cost Explorer. The key is cutting redundant tokens and duplicate calls without lowering quality.
| Goal | Lever | Point |
|---|---|---|
| Cut duplicate calls | Semantic/prompt caching | Reuse identical/common prefixes |
| Cut tokens | Context optimization/compression | Pass only needed evidence |
| Cheap bulk | Batch inference | Lower unit cost via async |
| Right model | Model tiering | Pick model by difficulty |
Trap: “always use the smallest model to cut cost” is wrong—using a small model for hard queries lowers quality and raises cost via retries; use model tiering. Also “prompt caching equals semantic caching” is wrong—the former skips recomputing common prefixes; the latter reuses results for semantically identical queries.
4.1.2Section summary
- Fewer calls = semantic/prompt caching / fewer tokens = context optimization/compression
- Right model = model tiering / cheap bulk = batch inference / steady = provisioned optimization
Sign in to track progress — Log in.
Quick check
(just a quick review)Q1. A FAQ bot receives many semantically identical questions phrased differently, inflating FM cost. Most effective cost reduction?
Q2. A nightly job summarizes hundreds of thousands of documents. Real-time is not needed and you want minimal cost. Best option?
Q3. You process all queries with the largest model, and cost is high. You want to cut cost while preserving quality. Best option?
Keep track of your progress
The full study guide is free to read. Sign up free to practice with the question bank, track what you have read, review your mistakes, and highlight passages.