What’s Cache

A cache is a high-speed data storage layer that stores a subset of data, typically transient in nature, so that future requests for that data are served up faster than is possible by accessing the data’s primary storage location.

Example

Cache is a general technique that uses any high-speed temporary storage to avoid accessing low-speed storage. It’s widely used in various situations:

  1. (CPUCache > RAM): CPU Cache saves some data to avoid accessing RAM.
  2. (RAM > DISK): RAM could be a cache when loading the data of disk.
  3. (Local Memory > remote memory): application saves data to the local cache to reduce communication with the remote server.
  4. (Close server > far server): CDN
  5. ….

The Memory Hierarchy in Computer Architecture

Pros & Note

  • Pros

    • Improve Application Performance(latency & throughput), especially the reading operation
    • Reduce the load of low-storage, e.g. disk, database
    • Eliminate Database Hotspots

  • Cons

    • more memory usage
    • should be useless in a write-intensive scenario

  • Note

    • Cache should be transparent to business, so a good cache design should never impact business logic.
    • It’s important to monitor cache hit/miss rate to make sure if cache really works
    • Be careful about the data consistency between cache and primary storage if
      • the data on primary storage could modify without notifying cache
      • application writes newest data to cache only

Typical Issues and Solutions

Here is a typical workflow to query data powered by cache. Normally, cache plays a role in protecting the database and improving performance. But sometimes it does not work well, let’s show what will possibly happen and how to solve them.

Cache Avalanche (缓存雪崩)

If massive cache entries are not available for some reason, the most query requests that were originally blocked by the cache will flock to the database like a mad dog. At this point, if the database can’t withstand this huge pressure, it will collapse.

  • Typical cause
    1. cache node crash
    2. network stuck
    3. many cache entry get expired at the same time
  • Solution
    1. short-term solution: close/downgrade the application layer if it’s not important to protect database
    2. apply a cache cluster to ensure the high availability of caches
    3. rewrite a scattered expiration time(aka random gap time) or even disable expired time

Cache Penetration （缓存穿透)

The data to be queried usually exists! However, when the data does not exist at all, the application would always hit cache miss and have to query the database again and again.

  • Typical cause
    • illegal or unexpected data query
    • normal request
  • Solution
    • filter illegal or unexpected request
    • cache empty key
    • setup bloom filter to skip most queries of unexisted key

      Note: bloom filter is a data structure to check if it may exist or must not exist. It needs a initialization based on full data. It’s usually used with scheduled initialization for dynamic primary data.

Hotspot Invalid (缓存击穿)

Few hot data with really high requests get expired or deleted, there will be a large number of requests falling on the database at this moment, which may cause the database to crash even if most queries are duplicated.

  • Typical cause
    • business bug
  • Solution
    • make a clear plan for hot data refresh
    • use mutex or other checking to avoid duplicate query

Reference