What's changed: Added per-section figures (cert-figure-retrofit). New AI-901 Chapter 6 filling the missing Domain 2 sub-areas: implementing text and speech (Azure AI Language; Azure AI Speech = STT/TTS/translation; multimodal models for spoken prompts) and implementing computer vision and image generation (multimodal/Azure AI Vision = OCR/object detection; image-generation models). Appended as ch6 to keep existing ch1-5 relatedSections non-breaking.
6.2Implementing Computer Vision and Image Generation
Understand how to implement two capabilities in Foundry—"understanding images" (visual input) and "creating images" (image generation). Cover image interpretation/OCR via multimodal models or Azure AI Vision, and text-to-image generation via generative models, with selection guidance.
Image-related implementation has two distinct capabilities: (1) understand images (visual input)—recognize, describe, or classify the content of a photo or diagram; and (2) create images (image generation)—generate new images from a text instruction. For AI-901, don’t confuse the two and know how to implement each in Foundry. A memory aid: understanding is "image is the input," generation is "image is the output."
6.2.1Understanding images (visual input)
There are two routes to working with image content: (1) pass an image along with a prompt to a deployed multimodal model to flexibly interpret it—"what’s in this photo," "summarize this chart"; and (2) use the task-focused Azure AI Vision (Foundry Tools) for fixed analyses out of the box—OCR (text recognition, Read), object detection, image captioning, tagging, and face detection. Use a multimodal model for conversational/arbitrary questions, and Azure AI Vision for fixed extraction (especially text in images = OCR).
6.2.2Creating images (image generation)
To produce new images, use an image-generation model (a text-to-image generative model, e.g., DALL·E from the catalog). It generates an image from a text prompt (e.g., "a watercolor of a seaside at dusk") and is deployed and called like any other model. Because it "creates new content," it is a form of generative AI—the opposite goal of vision, which recognizes images. For AI-901: "create new visual output = a generative model" vs "interpret an existing image = a multimodal model / Azure AI Vision."
| Goal | Capability | How |
|---|---|---|
| Ask arbitrary questions about a photo | Understand image (visual input) | Deployed multimodal model |
| Read text inside an image | OCR (text recognition) | Azure AI Vision (Read) |
| Detect objects / get captions | Image analysis | Azure AI Vision |
| Make a new image from a prompt | Image generation (image is output) | Image-generation model (e.g., DALL·E) |
OCR is a computer-vision capability: reading text inside an image (OCR) is a core vision capability. Extracting structured fields from invoices/forms is instead the job of Azure Content Understanding (Chapter 5), which uses OCR internally but goes further to structure "which value is the amount." Don’t confuse "read text (OCR)" with "structured field extraction (Content Understanding)."
Watch the mix-ups: (1) understanding (image in) vs generation (image out) are opposite goals. (2) arbitrary questions = multimodal model; fixed analysis/OCR = Azure AI Vision. (3) image generation is a kind of generative AI (not computer vision). (4) Don’t confuse OCR (read text) with Content Understanding (structured field extraction).
Common: "freely describe a photo" = multimodal model; "read text in an image" = Azure AI Vision (OCR/Read); "make a new image from a prompt" = image-generation model (generative AI); "structured-extract the amount from an invoice" = Content Understanding. Decide by image-in vs image-out and fixed analysis vs free response.
6.2.3Section summary
- Understand images (image in): arbitrary questions = multimodal model; fixed analysis/OCR = Azure AI Vision
- Generate images (image out): text-to-image = image-generation model (e.g., DALL·E) = a kind of generative AI
- OCR is a vision capability; structured field extraction is Content Understanding (different)
- All are implemented by deploying models / connecting services in Foundry
Sign in to track progress — Log in.
Quick check
(just a quick review)Q1. What do you use to create a new image from a text prompt?
Q2. Which capability best reads text inside an image (e.g., on a receipt)?
Q3. Which correctly distinguishes "understanding an image" from "generating an image"?
Q4. What best answers free-form questions like "what is in this photo?" conversationally?
Q5. To structured-extract the "amount" field from an invoice PDF (not merely read the text), what fits best?
Q6. Image generation is closest to which broad category?
Keep track of your progress
The full study guide is free to read. Sign up free to practice with the question bank, track what you have read, review your mistakes, and highlight passages.