Notesfoundations🌱 seedling

Caching

Also: Cache · Hit Ratio · Cache Hit Rate

A cache keeps a copy of a result somewhere cheaper to read than its source, so repeat reads skip the expensive work. Its value is set by the hit ratio, and its danger is that the system comes to depend on it.

Why it exists

Reads repeat: the same pages, profiles and sessions, thousands of times a second. Recomputing them from disk, joins or remote services wastes latency and capacity.

How it works

Trade-offs

Gains Costs
lower latency, far less backend load staleness; another system to run
absorbs read spikes the backend becomes sized for misses, so the cache is load-bearing

In practice

  • Cold caches cause outages: if the database can only handle the misses, an emptied cache (restart, failover, deploy) overloads it. Warm gradually or throttle.
  • Shared vs local: a shared tier holds one copy for all servers and survives deploys; an in-process cache is nanoseconds away but duplicated and hard to invalidate.

Pitfalls

  • Watching only the average hit ratio: one hot key's Cache Stampede can melt a database at "99%".
  • Caching under the wrong key (missing user, locale or header) serves one user's data to another.

Recall

These cards are also in your review deck, scheduled with spaced repetition.