What's changed: Initial: 3 sections for Domain 4 (operational efficiency and optimization)
4.2Optimizing Application Performance
Learn to improve GenAI app responsiveness: latency-optimized models, parallel requests, streaming, tuning temperature/top-k/top-p, retrieval optimization, and Auto Scaling.
Generation is inherently slow. Optimize UX by reducing perceived latency, increasing throughput, and tuning output quality via parameters.
4.2.1Levers to improve performance
- Lower latency: latency-optimized models, streaming, pre-computation for predictable queries, and parallel requests for complex workflows.
- Output parameters: tune temperature/top-k/top-p to requirements (low temperature for determinism, higher for diversity).
- Retrieval optimization: speed/accuracy of RAG via index optimization, query preprocessing, and custom-scored hybrid search.
- Throughput/scale: token-processing optimization, batch inference, concurrent-invocation management, and Auto Scaling for GenAI traffic.
Common: perceived speed = streaming + latency-optimized models, deterministic output = low temperature, diverse output = higher temperature/top-p, predictable queries = pre-computation, faster RAG = index/query preprocessing.
Performance optimization means “do not block, return fast, output that fits requirements.” Lower perceived latency with streaming (incremental display) and latency-optimized models (for time-sensitive use), pre-compute predictable queries, and parallelize complex workflows. Tune output with temperature (low = deterministic/factual, high = creative) and top-k/top-p (sampling breadth), validated via A/B testing. Fix slow RAG with index optimization, query preprocessing, and custom-scored hybrid search. Raise throughput with token-processing optimization, batch inference, and concurrent-invocation management, and secure capacity with Auto Scaling tuned to GenAI traffic and provisioned-throughput optimization. Profile APIs (prompt/completion patterns) to find bottlenecks and apply LLM-inference-specific latency reductions.
| Problem | Fix | Note |
|---|---|---|
| Feels slow | Streaming + latency-optimized model | Incremental display |
| Unstable/diffuse output | Tune temperature/top-p | Low temperature for determinism |
| Slow RAG | Index/query optimization | Hybrid search |
| Variable load | Auto Scaling | Tuned to GenAI traffic |
Trap: “raise temperature to improve accuracy of factual answers” is wrong—use low temperature for deterministic/factual output; higher temperature increases diversity but also hallucination risk. Also “streaming reduces total processing time” is wrong—it reduces perceived latency, not total time.
4.2.2Section summary
- Perceived = streaming + latency-optimized model / output = tune temperature/top-k/top-p
- RAG = index/query optimization / scale = Auto Scaling + concurrency management
Sign in to track progress — Log in.
Quick check
(just a quick review)Q1. For a factual business assistant that needs consistent answers, you want deterministic output that does not vary. Correct parameter setting?
Q2. For a real-time chat where responsiveness matters, you want to reduce perceived latency for long answers. Best combination?
Q3. In a RAG app, the retrieval step is slow and drives overall latency. How do you make retrieval faster and more accurate?
Keep track of your progress
The full study guide is free to read. Sign up free to practice with the question bank, track what you have read, review your mistakes, and highlight passages.