3.8Information Media (Multimedia, Compression, and VR/AR)

Handling photos, video, and music from a single smartphone feels ordinary today, but behind the scenes, technology converts images and audio into numeric data and compresses it as small as possible. This section covers the idea of multimedia, which integrates multiple media types, how compression works, and the basics of VR/AR/the metaverse, which extend or substitute for physical space.

3.8.1Multimedia, encoding, and compression

• Multimedia is a way of conveying information by combining multiple types of media—text, still images, video, audio. Websites and video streaming services, among many modern services, assume multimedia by default.
• Encoding converts analog information or raw data—images, audio—into a digital format suited to storage and transmission. Compression shrinks the size of encoded data, and comes in two kinds: lossless compression (the original data can be fully restored; e.g., ZIP, PNG) and lossy compression (some information is discarded and cannot be restored, but the size shrinks further; e.g., JPEG, MP3).
• JPEG is a lossy compression format for still images (photos, etc.), shrinking size by discarding information the human eye is unlikely to notice. MPEG is a general term for a family of video compression schemes (compressing by exploiting differences between frames). MP3 is a lossy compression format for audio, discarding sound information the human ear is unlikely to notice.

3.8.2VR, AR, the metaverse, and CG

• VR (virtual reality) immerses the user in a computer-generated virtual space (e.g., experiencing 360-degree video through dedicated goggles). AR (augmented reality) overlays information onto the real-world view (e.g., overlaying navigation arrows on a smartphone camera feed). Note the distinction: VR immerses you in a virtual space, while AR augments the real space.
• The metaverse is a persistent, three-dimensional virtual space built on the internet where multiple users interact and act through avatars. It is often experienced immersively using VR technology, but access from a PC or smartphone is also assumed.
• CG (computer graphics) is the broad set of techniques for generating and processing images using a computer. It is used widely—compositing with live-action footage, animation production, 3D representations in product design—and the visuals in VR/AR are also underpinned by CG technology.

Consider a real estate company revamping its property-listing service. To post exterior and interior photos on the website, posting every file uncompressed at full quality would slow page loads, so photos are saved as lossy JPEG, discarding information the eye is unlikely to notice to keep file size down. Floor plans, by contrast, have crisp lines and text where degradation is noticeable, so the company chooses lossless PNG, which can be restored perfectly even after compression. To stream a video tour of a viewing, an MPEG-family compression scheme is typically used, and the narration audio commonly uses lossy compression like MP3. To differentiate the service further, the company adds a VR viewing feature letting users walk through a room wearing goggles, plus an AR feature that overlays furniture-placement previews when a smartphone camera is pointed at a room. Looking ahead, it is also considering a metaverse showroom where multiple users can visit the same property simultaneously as avatars and negotiate with a staff member—and all of these three-dimensional visual representations are underpinned by CG technology. As this shows, choosing the right compression method depending on the goal—whether some degradation is acceptable, or perfect restoration is required—matters in practice.

3.8.3Section summary

• Lossless compression = fully restorable (ZIP/PNG); lossy compression = not restorable, but higher compression ratio (JPEG/MP3)
• VR = immersion in a virtual space; AR = overlaying information on the real space. The metaverse is a 3D virtual space tied to VR technology
• Multimedia integrates multiple media types; CG is the underlying technology behind VR/AR visuals