What's changed: Deepened AI-900 Chapter 3 to the AZ-900 baseline (how CV works = pixels/deep learning, segmentation/spatial analysis, resource creation with key+endpoint, Custom Vision project types, FAQ)
3.1Computer Vision Tasks
Understand the common computer vision tasks—image classification, object detection, OCR, and face detection—and how they differ.
Computer vision is the AI field that analyzes images and video to recognize and understand their content (objects, text, faces). Even within "working with images," the task type depends on the goal. For AI-900, map a goal → the right task; the classification vs. object detection distinction is especially common.
3.1.1Common tasks
- Image classification: assign one label to the whole image (e.g., this photo is a "dog").
- Object detection: return the location (bounding box) and label of objects (e.g., dog here, car there).
- OCR (optical character recognition): read text in images into machine text (printed or handwritten).
- Face detection: locate faces in an image and analyze attributes.
- Image analysis/description: generate tags and captions and summarize the content.
| Task | Output | Goal (example) |
|---|---|---|
| Image classification | One label per image | Cat or dog? |
| Object detection | Boxes + labels (multiple) | Locate each product on a shelf |
| OCR | Text | Read text on signs/docs |
| Face detection | Face location/attributes | Detect faces in a photo |
| Image analysis | Tags, captions | Auto-describe an image |
3.1.2How computer vision works
Computer vision takes an image as an array of pixel values and interprets the patterns within it using a trained model. Today’s high accuracy rests on the deep learning from the previous chapter—multi-layer neural networks developed for images in particular. The model trains on large sets of labeled images ("dog," "car") and finds the same features in new images. AI-900 does not test the inner algorithms in detail; it is enough to know what each task takes as input and produces as output.
Beyond the basic tasks, a few help round out the picture. Segmentation outlines objects as pixel-level regions rather than boxes—finer than object detection. Spatial analysis analyzes people’s movement, counts, and dwell time in video. Remember the order "classification < object detection < segmentation" by increasing location detail.
Scenario: retail shelf analysis. To know "which products are where and how many," you need positions → object detection. To read price tags → OCR. If you only need "is this a beverage shelf or a food shelf," image classification suffices. Split classification vs. detection by whether you need location.
Watch the mix-ups: (1) image classification (one label, no location) vs object detection (location = box + label, multiple)—use detection when you need "where." (2) OCR (text in images → machine text) differs from document intelligence (structured field extraction from forms). (3) Face detection has some restricted features for Responsible AI.
Q. OCR vs document intelligence? OCR just reads text from an image into machine text; document intelligence goes further and structures the meaning of fields like "amount," "date," "bill-to." Use OCR to "read text," document intelligence to "extract fields from forms." Q. Object detection vs segmentation? Detection marks location with a box; segmentation outlines the shape as pixel-level regions.
A common distinction: image classification (one label) vs. object detection (location + label). "What is in it" = classification; "what and where" = object detection. Reading text = OCR, faces = face detection, tags/captions = image analysis.
3.1.3Section summary
- Computer vision = analyze images/video to recognize and understand
- Distinguish classification (label) / object detection (location+label) / OCR (text) / face detection / image analysis (tags, captions)
- Split classification vs. detection by whether you need location ("where")
Sign in to track progress — Log in.
Quick check
(just a quick review)Q1. Which task fits drawing a box showing "where the dog is" plus a label?
Q2. Which task extracts written text from a scanned invoice image as text?
Q3. Which task assigns a single label like "this is a cat photo" to a whole image?
Q4. What is the key difference between image classification and object detection?
Q5. Which task auto-generates tags/captions to summarize image content?