56.caching.md

Using Caching

Usually, the performance bottleneck of a Web application appears in the relational database. When concurrent access is large, if all requests need the relational database to finish data persistence operations, the database will surely be overloaded. One of the most important ways to optimize Web application performance is to use caching. Put those data sets that are not large but are accessed very often into a cache server in advance. This is again a typical way of trading space for time. Usually, the cache server puts data directly in memory and uses very efficient data access strategies, such as hash storage and key-value pairs, so its read and write performance is far better than that of a relational database. Because of this, we can connect a Web application to a cache server to optimize its performance, and one very good choice is Redis.

The cache architecture of a Web application is roughly shown in the picture below.

Connecting a Django Project to Redis

In previous lessons, we already introduced the installation and use of Redis, so we will not repeat it here. If you need to connect Redis in a Django project, you can use the third-party library django-redis. This third-party library depends on another third-party library named redis, which wraps many Redis operations.

Install django-redis.

pip install django-redis

Modify the cache-related configuration in the Django settings file.

CACHES = {
    'default': {
        # Specify connecting to Redis through django-redis
        'BACKEND': 'django_redis.cache.RedisCache',
        # URL of the Redis server
        'LOCATION': ['redis://1.2.3.4:6379/0', ],
        # Prefix of keys in Redis, to solve naming conflicts
        'KEY_PREFIX': 'vote',
        # Other configuration options
        'OPTIONS': {
            'CLIENT_CLASS': 'django_redis.client.DefaultClient',
            # Connection pool parameters, with several Redis connections prepared in advance
            'CONNECTION_POOL_KWARGS': {
                # Maximum connection count
                'max_connections': 512,
            },
            # Password for connecting to Redis
            'PASSWORD': 'foobared',
        }
    },
}

At this point, our Django project can already connect to Redis. Next, let us modify the project code and use Redis to provide cache service for the API for getting subject data.

Providing Cache Service for Views

Declarative Caching

Declarative caching means not changing the original code, but using decorators, or proxies, in Python to add cache ability to the original code. For FBV, the code is shown below.

from django.views.decorators.cache import cache_page


@api_view(('GET', ))
@cache_page(timeout=86400, cache='default')
def show_subjects(request):
    """Get subject data"""
    queryset = Subject.objects.all()
    data = SubjectSerializer(queryset, many=True).data
    return Response({'code': 20000, 'subjects': data})

The code above uses Django's wrapped cache_page decorator to cache the return value of the view function, that is, the response object. The original purpose of cache_page is to cache the page rendered by the view function. For a view function that returns JSON data, this is equal to caching the JSON data. When using the cache_page decorator, we can pass the timeout parameter to specify the cache expiration time, and we can also use the cache parameter to specify which group of cache services should be used to cache data. A Django project allows multiple groups of cache services to be configured in the settings file. The cache='default' above specifies using the default cache service, because in the settings before, we only configured the cache service named default. The return value of the view function will be serialized into a byte string and placed in Redis. The str type in Redis can receive byte strings. We also do not need to handle serialization and deserialization of cached data by ourselves, because the cache_page decorator calls RedisCache in the django-redis library to connect to Redis. This class uses DefaultClient to connect to Redis and uses pickle serialization. django_redis.serializers.pickle.PickleSerializer is the default serializer class.

If the cache does not contain subject data, then when the subject data API is accessed, our view function sends an SQL statement to the database by executing Subject.objects.all() to get the data. The return value of the view function will be cached, so when this view function is requested next time, if the cache has not expired, the return value of the view function can be got directly from the cache, and there is no need to query the database again. If you want to understand cache usage, you can configure database logs or use Django-Debug-Toolbar to check. When the subject data API is accessed for the first time, you will see the SQL statement used to query subject data. When the subject data is got again, no SQL statement is sent to the database, because data can be got directly from the cache.

For CBV, we can use Django's decorator named method_decorator to put the decorator function cache_page onto a method in the class. The effect is the same as the code above. One thing everyone should pay attention to is that the cache_page decorator cannot be put directly on the class, because it is a decorator for functions. This is why Django provides method_decorator to solve this problem. Clearly, method_decorator is a decorator for decorating class methods.

from django.utils.decorators import method_decorator
from django.views.decorators.cache import cache_page


@method_decorator(decorator=cache_page(timeout=86400, cache='default'), name='get')
class SubjectView(ListAPIView):
    """View class for getting subject data"""
    queryset = Subject.objects.all()
    serializer_class = SubjectSerializer

Programmatic Caching

Programmatic caching means using cache service through code written by ourselves. Although this way has a little more code, compared with declarative caching, it is more flexible in how it uses and operates the cache, so it is used more often in real development. The code below removes the cache_page decorator used before, and directly gets a Redis connection through the get_redis_connection function provided by django-redis to operate Redis.

def show_subjects(request):
    """Get subject data"""
    redis_cli = get_redis_connection()
    # First try to get subject data from the cache
    data = redis_cli.get('vote:polls:subjects')
    if data:
        # If subject data is got, do deserialization
        data = json.loads(data)
    else:
        # If subject data is not got from the cache, query the database
        queryset = Subject.objects.all()
        data = SubjectSerializer(queryset, many=True).data
        # Serialize the queried subject data and put it into the cache
        redis_cli.set('vote:polls:subjects', json.dumps(data), ex=86400)
    return Response({'code': 20000, 'subjects': data})

It should be explained that the Django framework provides two ready-made variables, cache and caches, to support cache operations. The former accesses the default cache, which is the cache named default, and the latter can get the specified cache service through indexing, for example caches['default']. Sending get and set messages to the cache object can implement cache reading and writing, but the operations this way can do are limited, and are not as flexible as the way used in the code above. There is another thing worth noting. Since the Redis connection object got through get_redis_connection can send many operations to Redis, including dangerous operations such as FLUSHDB and SHUTDOWN, in real commercial project development, people usually wrap django-redis one more time, for example, wrap a utility class that only provides the cache operation methods needed by the project, so the possible risk of directly using get_redis_connection is avoided. Of course, wrapping cache operations by ourselves can also use the "Read Through" and "Write Through" ways to update cache, and this will be introduced below.

Cache-Related Problems

Updating Cached Data

When using cache, one question that must be understood clearly is: when data changes, how should the data in the cache be updated? Usually, cache updating has the following patterns:

1. Cache Aside Pattern
2. Read/Write Through Pattern
3. Write Behind Caching Pattern

The specific way of the first pattern is: when data is updated, first update the database, and then delete the cache. Notice that we must not use the way of first updating the database and then updating the cache, and we also must not use the way of first deleting the cache and then updating the database. You can think about the reason by yourself, especially in the case where there are concurrent read and write operations. Of course, the way of first updating the database and then deleting the cache also has theoretical risk, but the chance of a problem happening is very low, so many projects use this way.

In the first way, the developer who writes the business code needs to take care of operations on two storage systems by themselves, the cache and the relational database, so the code becomes very troublesome to write. The main idea of the second way is to turn the backend storage system into one set of code, and cache maintenance is wrapped inside this set of code. Among them, Read Through means updating the cache during query operations. That is to say, when the cache becomes invalid, the cache service itself is responsible for loading the data, and this is transparent to the application side. Write Through means that when data is updated, if the cache is not hit, update the database directly and return. If the cache is hit, update the cache first, and then let the cache service update the database by itself, synchronously. As we said just now, if you wrap Redis operations in the project one more time, you can implement the Read Through and Write Through patterns. Although this adds some work, it is surely something that works once and benefits for a long time.

The third way is that when data is updated, only the cache is updated, and the database is not updated. The cache service then updates the database asynchronously in batches. This way can greatly improve performance, but the cost is giving up the strong consistency of data. The logic of implementing the third way is relatively complex, because it needs to track which data has been updated, and then refresh it to the persistent layer in batches.

Cache Penetration

Cache is added as a middle layer to reduce database pressure. If malicious visitors frequently access data that does not exist in the cache, then the cache loses the meaning of existing. Instantly, the pressure of all requests falls on the database. This makes the database bear huge pressure and can even cause connection problems. This is similar to a distributed denial of service attack, or DDoS. One way to solve cache penetration is to agree that if a query returns an empty value, this empty value is also cached, but a short timeout must be set for the cache of the empty value, because caching such a value is still a waste of cache space. Another way to solve cache penetration is to use a Bloom filter. Everyone can learn the specific way by themselves.

Cache Breakdown

In a real project, it is possible that some cache key expires at a certain time, and just at this time a large number of concurrent requests for this key arrive. These requests do not find the data for that key in the cache, so they directly get the data from the database and write it back to the cache. At this moment, these highly concurrent requests may instantly crush the database. This phenomenon is called cache breakdown. A more common way to solve cache breakdown is to use a mutex lock. Simply speaking, when the cache becomes invalid, do not immediately load data from the database. First set a mutex lock, such as setnx in Redis. Only the request that successfully sets the mutex lock can run the query, load data from the database, and write it into the cache. Other requests that fail to set the mutex lock can first sleep for a short time, then try again to get data from the cache. If the cache still has no data, repeat the mutex-lock operation mentioned just now. The rough reference code is shown below.

data = redis_cli.get(key)
while not data:
    if redis_cli.setnx('mutex', 'x'):
        redis.expire('mutex', timeout)
        data = db.query(...)
        redis.set(key, data)
        redis.delete('mutex')
    else:
        time.sleep(0.1)
        data = redis_cli.get(key)

Cache Avalanche

Cache avalanche means that when data is put into the cache, the same expiration time is used, so the cache becomes invalid at the same time at some moment, all requests are forwarded to the database, and the database gets too much pressure instantly and crashes. The way to solve cache avalanche is also fairly simple. We can add a random time on top of the fixed cache expiration time. In this way, to some extent, we can avoid different keys expiring together at the same time. Another way is to use multi-level caching. Each level of cache has a different expiration time. In this case, even if all cache at one level expires together, cache at other levels can still provide data, so all requests do not fall to the database.