← CSC 115 Computer Organization and Architecture
Memory Hierarchy and Cache Organization
Notes
Principles of the Memory Hierarchy
In modern computer systems, processor clock speeds have advanced far more rapidly than random access memory access times, creating a performance disparity often termed the memory wall. To address this mismatch, computer architects structure system storage as a hierarchical pyramid. At the top of the pyramid sit processor registers, offering single-cycle access but minimal capacity. Below registers are multiple levels of cache memory, followed by dynamic random access memory, solid-state drives, and secondary magnetic storage. As one moves down the hierarchy, capacity increases significantly, cost per bit drops, but latency grows by orders of magnitude.
Static versus Dynamic Random Access Memory
Main system memory is built from Dynamic Random Access Memory, where each bit is stored using a single transistor and a tiny capacitor. Because capacitors leak charge rapidly, dynamic memory requires periodic electrical refresh cycles, making it comparatively slow. In contrast, cache memory uses Static Random Access Memory, which implements four to six transistors arranged as a bistable flip-flop per bit. Static memory requires no refresh cycles and switches states rapidly, but its larger circuit footprint makes it expensive and limited in capacity.
Locality of Reference and Cache Operation
Cache memory exploits the principle of locality to keep frequently accessed data near the processor. Temporal locality states that a memory location accessed recently is likely to be accessed again soon, such as variables inside a loop. Spatial locality states that data physically adjacent to recently accessed addresses will likely be needed shortly, such as contiguous elements of an array. When the processor finds requested data in the cache, a cache hit occurs. If absent, a cache miss forces the system to retrieve an entire cache block from main memory.
Practical example
Consider the national Kenya eCitizen immigration portal database engine processing national identity card lookups. An application thread checks applicant verification statuses stored in a sequential memory array starting at base address 0x4000. Each applicant record is 8 bytes, and the Level 1 data cache uses 64-byte block sizes.
When the processor requests the first record at 0x4000, it experiences an initial compulsory cache miss because the cache is cold. The memory controller fetches a full 64-byte block from main Dynamic Random Access Memory into the cache line, which covers records from address 0x4000 through 0x403F. This single transfer brings eight consecutive citizen records into cache simultaneously.
When the verification script processes citizen records two through eight, each lookup results in an instantaneous Level 1 cache hit. The portal completes these seven checks in single clock cycles without accessing main memory, illustrating how spatial locality maximizes throughput on Kenyan public infrastructure servers.
Quiz
1. What underlying hardware characteristic necessitates periodic refresh cycles in DRAM?
2. Which form of locality is leveraged when a loop repeatedly accesses the exact same counter variable?
3. What occurs when a processor searches for a memory block in Level 1 cache and fails to find it?
Sign in to submit and securely grade your quiz.