2.2Memory and the Storage Hierarchy

No matter how fast the CPU, it means nothing if the data it needs is not at hand. Computers combine several storage devices of differing speed, capacity, and cost into a hierarchy to achieve both perceived speed and practical capacity at once. Understanding this lets you diagnose why an application feels slow and where reinforcing it would actually help.

2.2.1RAM, ROM, and the storage hierarchy

• RAM (Random Access Memory) is volatile read-write memory whose contents are lost when power is removed. It serves as main memory, holding the running program and its data. ROM (Read Only Memory) is non-volatile memory whose contents persist without power, used to store firmware needed at boot time (BIOS/UEFI, etc.).
• The storage hierarchy is the structure running register -> cache memory -> main memory (RAM) -> secondary storage (SSD/HDD), where the slower the access speed, the larger the capacity and the lower the cost. Keeping the principle "closer to the CPU = faster, smaller, and more expensive; farther away = slower, larger, and cheaper" in mind makes performance-design decisions easier.

2.2.2Cache memory write strategies

• Cache memory sits between the CPU and main memory: it is far faster than main memory but much smaller in capacity. By holding frequently used data in the cache, it reduces the number of slow accesses to main memory. The proportion of times the data the CPU needs is already present in the cache is called the hit rate.
• Write-through writes to main memory at the same time as writing to the cache. Because the cache and main memory always stay consistent, it is simple and reliable, but write performance suffers since every write also touches main memory. Write-back writes only to the cache, and reflects the change to main memory in a batch only when that cache line is evicted. Write performance is higher, but there is a window where the cache and main memory are inconsistent, making control more complex.

Let us walk through calculating effective access time. Consider a system where cache access time is 10 nanoseconds, main memory access time is 100 nanoseconds, and the hit rate is 90% (0.9). When the CPU requests data, 90% of the time it is found in the cache (a hit) and takes 10 ns; the remaining 10% of the time it misses the cache and must access main memory, taking 100 ns. The effective access time = cache time x hit rate + main memory time x (1 - hit rate). Plugging in the numbers: 10 x 0.9 + 100 x 0.1 = 9 + 10 = 19 nanoseconds—much faster than main memory alone (100 ns), though slower than the cache alone (10 ns), landing in between as expected. If the hit rate improves to 95%, the result shrinks further to 10 x 0.95 + 100 x 0.05 = 9.5 + 5 = 14.5 nanoseconds. This calculation shows that even a small improvement in hit rate substantially improves effective access time, and leads to the practical judgment that "adding more main memory has limited effect if the cache hit rate stays low." The choice of write strategy is a similar tradeoff: applications like financial systems, where data consistency is always the top priority, favor write-through, while applications with frequent writes that prioritize speed above all favor write-back (which then requires separate measures against data loss on power failure).

2.2.3Virtual memory and paging

• Virtual memory is a mechanism that makes an address space larger than the actual physical main memory appear usable. With the paging approach, memory is divided into fixed-size pages; only the pages currently needed are kept in main memory, while unused pages are evicted to secondary storage (disk). The OS manages this.
• When a needed page is not in main memory, a page fault occurs, triggering a read from secondary storage that stalls processing. When the pages simultaneously needed exceed what installed memory can hold, page swapping can occur so frequently that processing efficiency collapses—a phenomenon called thrashing.

2.2.4Section summary

• RAM = volatile main memory; ROM = non-volatile, holds firmware. The storage hierarchy gets faster, smaller, and more expensive closer to the CPU
• Write-through = simultaneous writes, simple, reliable; write-back = writes only to the cache, faster but temporarily inconsistent. Effective access time = cache time x hit rate + main memory time x (1 - hit rate)
• Virtual memory (paging) provides an address space beyond main memory by also using secondary storage. Excessive page swapping leads to thrashing