What's changed: Added per-section figures (cert-figure-retrofit). New AI-901 Chapter 2 (Domain 2: Foundry overview = project/model catalog/playground/connected services/evaluation & monitoring/portal vs SDK; model selection criteria and deployment = endpoint + key, token billing, context length)
2.2Selecting and Deploying Models
Understand how to choose a model from the catalog (capability, modality, context length, cost), the flow of deploying to get an endpoint + key, the basics of tokens and cost, and trying/evaluating in the playground.
Foundry’s model catalog lists many pretrained models differing in purpose, size, and provider. The first implementation step is choosing a model that fits the task. Look at four things: (1) capability (chat/generation, summarization, code, reasoning strength), (2) modality (text only, or multimodal with image/audio), (3) context length (max input tokens at once), and (4) cost (price per token, speed). Bigger "smarter" models tend to be pricier and slower, so pick the smallest model that is good enough.
2.2.1Deployment — making a model usable
After choosing, you deploy the model. Deployment yields an endpoint (URL) and a key (credential) to call it. Apps and the SDK send requests using "endpoint + key" (you can also use Microsoft Entra ID auth instead of a key). At deploy time you give a deployment name and, if needed, set throughput/capacity. You can create multiple deployments of the same model for different uses.
2.2.2Tokens and cost basics
Generative AI billing is based on tokens. A token is a processing unit like a "word fragment," and you are billed for both input (prompt) and output (response) tokens. Longer prompts and responses cost more and run slower. So cost optimization means trimming unnecessary context, choosing an appropriately sized model, and using large models only when needed. Each model has a context-length (token) limit, and input exceeding it cannot be passed.
| Criterion | Question | Example |
|---|---|---|
| Capability | What task? | Summary/chat/code/advanced reasoning |
| Modality | What input? | Text-only vs image/audio (multimodal) |
| Context length | How much at once? | Short prompt vs long-document summary |
| Cost/speed | Budget and latency? | Bigger = more accurate, pricier, slower |
Scenario: build an internal FAQ chat. (1) Pick a mid-size chat-capable model from the catalog (no advanced reasoning needed → favor cost) → (2) deploy to get endpoint + key → (3) tune prompts in the playground → (4) embed in the app via the Foundry SDK → (5) evaluate/monitor response quality and cost (tokens), switching to a larger model only if needed.
Watch out: (1) "choose a model (catalog)" and "make it usable (deploy)" are separate steps; you only get an endpoint after deploying. (2) Billing covers both input and output tokens (not output only). (3) Context length = the max you can pass at once; excess cannot be sent (memory does not grow automatically). (4) Bigger is not always better—the smallest sufficient model is cost-optimal.
Common: (1) "what an app needs to call a model" = endpoint + key (or Entra ID auth). (2) "billing unit for generative AI" = tokens (input + output). (3) "can it process a long document at once" = judged by context length. (4) "I also want image input" = choose a multimodal model. (5) "reduce cost" = smaller model, shorter prompts.
2.2.3Section summary
- Four selection criteria: capability / modality / context length / cost & speed; pick the smallest sufficient model
- Deploy to get endpoint + key → apps/SDK call it (Entra ID auth also possible)
- Billing is per token (input + output); context length = max passable at once
- Flow: choose → deploy → tune in playground → implement with SDK → evaluate & monitor
Sign in to track progress — Log in.
Quick check
(just a quick review)Q1. What combination does an app need to call a deployed model?
Q2. Generative AI usage is mainly billed by which unit?
Q3. Which metric determines whether a long document can be passed to a model at once?
Q4. To accept both image and text input, what model characteristic should you choose?
Q5. Which is correct about "choosing from the catalog" vs "deploying"?
Q6. What is the basic principle for choosing a model to keep cost low while meeting quality?