Notesfoundations🌱 seedling
Caching
Also: Cache · Hit Ratio · Cache Hit Rate
A cache keeps a copy of a result somewhere cheaper to read than its source, so repeat reads skip the expensive work. Its value is set by the hit ratio, and its danger is that the system comes to depend on it.
Why it exists
Reads repeat: the same pages, profiles and sessions, thousands of times a second. Recomputing them from disk, joins or remote services wastes latency and capacity.
How it works
- Layers everywhere: CPU caches, the OS page cache, a database buffer pool, in-process maps, a shared tier (Redis, Memcached) and a Content Delivery Network.
- Hit ratio h sets backend load: the store sees (1 − h) of reads. From 99% to 99.9% cuts backend load tenfold.
- Average latency ≈ h × hit cost + (1 − h) × miss cost.
- The design questions are always the same: who fills it (Cache-Aside, Write-Through Cache, Write-Back Cache), what it throws out (Cache Eviction Policy), and how it learns about changes (Cache Invalidation).
Trade-offs
| Gains | Costs |
|---|---|
| lower latency, far less backend load | staleness; another system to run |
| absorbs read spikes | the backend becomes sized for misses, so the cache is load-bearing |
In practice
- Cold caches cause outages: if the database can only handle the misses, an emptied cache (restart, failover, deploy) overloads it. Warm gradually or throttle.
- Shared vs local: a shared tier holds one copy for all servers and survives deploys; an in-process cache is nanoseconds away but duplicated and hard to invalidate.
Pitfalls
- Watching only the average hit ratio: one hot key's Cache Stampede can melt a database at "99%".
- Caching under the wrong key (missing user, locale or header) serves one user's data to another.
Connections
Recall
These cards are also in your review deck, scheduled with spaced repetition.