In the previous lesson, we explored higher-order functions in Python, and I believe everyone now has a deeper understanding of the definition and use of functions. In this lesson, we will continue to talk about function-related knowledge. One is decorators, which are distinctive syntax in Python, and the other is recursive function calls.
In Python, a decorator is a syntax feature that uses one function to decorate another function and give it extra ability. A decorator itself is a function. Its parameter is the function being decorated, and its return value is a function with the decoration ability.
The concept of decorators is still a headache for beginners, so let us first use a simple example to explain what a decorator does. Suppose there are two functions named download and upload, used for file downloading and uploading, as shown below.
import random
import time
def download(filename):
"""Download a file."""
print(f'开始下载{filename}.')
time.sleep(random.random() * 6)
print(f'{filename}下载完成.')
def upload(filename):
"""Upload a file."""
print(f'开始上传{filename}.')
time.sleep(random.random() * 8)
print(f'{filename}上传完成.')
download('MySQL从删库到跑路.avi')
upload('Python从入门到住院.pdf')Note: The code above uses sleeping for a random amount of time to simulate that downloading and uploading files take some time. It does not really upload or download files over the network. In fact, it is also very easy to use Python to upload and download files over the network, and we will talk about that later.
Now there is a new requirement. We want to know exactly how much time calling download and upload takes. How should we do this? I believe many readers have already thought of it: we can record one time when the function begins executing, record another time after the function call ends, and then subtract the two to calculate the time spent on downloading or uploading, as shown below.
start = time.time()
download('MySQL_Full_Crash_Course.avi')
end = time.time()
print(f'花费时间: {end - start:.2f}秒')
start = time.time()
upload('Python从入门到住院.pdf')
end = time.time()
print(f'花费时间: {end - start:.2f}秒')Through the code above, we can record the time spent downloading and uploading files, but you may have noticed that the code for recording the time, calculating the time, and displaying it is all duplicate code. People with programming experience know that duplicate code is the source of all evil, so is there a simple and elegant way to record function execution time without writing duplicate code? In Python, decorators are the best solution to this kind of problem. Through decorator syntax, we can package the timing code, which has nothing to do with the original business of uploading and downloading, into a function. If the upload and download functions need timing, we only need to apply the decorator to these two functions.
The basic structure looks like this:
def record_time(func):
def wrapper(*args, **kwargs):
result = func(*args, **kwargs)
return result
return wrapperThe parameter func in the record_time function above means the decorated function. The wrapper function defined inside the function is the function with the decoration ability. It will execute the decorated function func, and it also needs to return the value produced at the end of the function call. You may have noticed that I left two blank places in the code above. This means we can add code in these places to provide extra ability. The record_time function finally returns this decorated function wrapper, and uses it to replace the original function func. After the original function func is decorated by record_time, when we call it, what we are really calling is the wrapper function, so we get the extra ability. The parameters of wrapper are special. Because we want to use wrapper to replace the original function func, but we do not know what parameters func will receive, we use variable positional arguments and keyword arguments to accept everything, and then pass them all to func unchanged. One more thing to emphasize is that Python allows nested function definitions. Just like above, we can define wrapper inside record_time. Many programming languages do not support this.
Now let us add the timing behavior:
import time
def record_time(func):
def wrapper(*args, **kwargs):
start = time.time()
result = func(*args, **kwargs)
end = time.time()
print(f'{func.__name__}执行时间: {end - start:.2f}秒')
return result
return wrapperWriting a decorator takes some effort, but once it is written, it can be reused again and again. If we later need to record function execution time again, we only need to add this decorator. There are two ways to use the decorator function above. The first way is to directly call the decorator function, pass in the decorated function, and get the return value. We can use this return value to directly replace the original function. Then, when calling it, we already get the extra ability provided by the decorator.
download = record_time(download)
upload = record_time(upload)
download('MySQL从删库到跑路.avi')
upload('Python从入门到住院.pdf')In Python, there is an even more convenient syntax sugar for using decorators. We can use @decorator_function to put the decorator function directly on the decorated function, and the effect is the same as the code above.
import random
import time
def record_time(func):
def wrapper(*args, **kwargs):
start = time.time()
result = func(*args, **kwargs)
end = time.time()
print(f'{func.__name__}执行时间: {end - start:.2f}秒')
return result
return wrapper
@record_time
def download(filename):
print(f'开始下载{filename}.')
time.sleep(random.random() * 6)
print(f'{filename}下载完成.')
@record_time
def upload(filename):
print(f'开始上传{filename}.')
time.sleep(random.random() * 8)
print(f'{filename}上传完成.')
download('MySQL从删库到跑路.avi')
upload('Python从入门到住院.pdf')In the code above, we used decorator syntax sugar to add the decorator to the download and upload functions. The decorated download and upload functions are actually the wrapper function returned in the decorator. Calling them is really calling wrapper, so they gain the function-execution-time recording ability.
If in some places in the code we want to remove the effect of the decorator and execute the original function, then when defining the decorator function, we need to do a little extra work. The wraps function in Python's standard-library functools module is also a decorator. If we put it on the wrapper function, this decorator can help us keep the function before decoration. In this way, when we need to cancel the decorator, we can get the function before decoration through the __wrapped__ attribute of the decorated function.
import random
import time
from functools import wraps
def record_time(func):
@wraps(func)
def wrapper(*args, **kwargs):
start = time.time()
result = func(*args, **kwargs)
end = time.time()
print(f'{func.__name__}执行时间: {end - start:.2f}秒')
return result
return wrapper
@record_time
def download(filename):
print(f'开始下载{filename}.')
time.sleep(random.random() * 6)
print(f'{filename}下载完成.')
@record_time
def upload(filename):
print(f'开始上传{filename}.')
time.sleep(random.random() * 8)
print(f'{filename}上传完成.')
# Calling the decorated function records execution time.
download('MySQL从删库到跑路.avi')
upload('Python从入门到住院.pdf')
# Cancel the decorator effect and do not record execution time.
download.__wrapped__('MySQL必知必会.pdf')
upload.__wrapped__('Python从新手到大师.pdf')Decorator functions themselves can also be parameterized. Simply speaking, decorators can also be customized by parameters passed in by the caller. We will explain this point later when we use it.
Python allows nested function definitions, and also allows functions to call each other. One function can even directly or indirectly call itself. When a function calls itself, this is called recursion.
A classic recursive example is the factorial. The factorial of a non-negative integer N satisfies:
Since that is the case, we can use recursive function calls to write a factorial function, as shown below.
def fac(num):
if num in (0, 1):
return 1
return num * fac(num - 1)In the code above, the condition if num in (0, 1) is called the convergence condition of recursion. Simply speaking, it means when the recursive call of the function should stop. The expression num * fac(num - 1) is the recursive formula, that is, the recursive definition of factorial.
Below, let us simply analyze what the whole process looks like if we use fac(5) to calculate the factorial of 5.
# 5 * fac(4)
# 5 * (4 * fac(3))
# 5 * (4 * (3 * fac(2)))
# 5 * (4 * (3 * (2 * fac(1))))
# 5 * (4 * (3 * (2 * 1)))
# 5 * (4 * (3 * 2))
# 5 * (4 * 6)
# 5 * 24
# 120
print(fac(5)) # 120Please note that function calls save the current execution context and restore the previous execution context through a data structure in memory called the stack. The stack is a last-in, first-out data structure. This means the function that enters the stack earliest returns last, while the function that enters the stack latest returns first. For example, if we call a function named a, and inside function a we call function b, and inside function b we call function c, then the first function to enter the stack is a, but the first function to leave the stack is c. Every time we enter one function call, the stack adds one stack frame. The stack frame is the structure we just mentioned for saving the current execution context. Every time a function call ends, the stack removes one stack frame. Usually, the stack space in memory is very small, so if the number of recursive calls is too large, it will cause stack overflow. Therefore, recursive calls must always make sure they can converge quickly. We can try executing fac(5000) and see whether it raises a RecursionError. The error message is maximum recursion depth exceeded in comparison, which is really stack overflow.
If we use the official Python interpreter, CPython, the default maximum stack depth for function calls is usually set to 1000. If that depth is exceeded, the RecursionError mentioned above occurs. Of course, we can use the setrecursionlimit function from the sys module to change the maximum recursion depth, but this is generally not recommended. Making recursion converge quickly is what we should do; otherwise, it is better to consider iterative looping rather than recursion.
Here is another example we talked about before: generating the Fibonacci sequence. Because the first two numbers in the Fibonacci sequence are both 1, and from the third number on each number is the sum of the previous two numbers, it can be written as f(n) = f(n - 1) + f(n - 2). Clearly, this is again a recursive definition, so we can use the recursive function below to calculate the nth Fibonacci number.
def fib1(n):
if n in (1, 2):
return 1
return fib1(n - 1) + fib1(n - 2)
for i in range(1, 21):
print(fib1(i))What needs to be reminded is that although the code above for calculating Fibonacci numbers looks very simple and clear, its execution performance is rather bad. You can try it yourself: change the second argument of the range function in the for loop above to 51, which means output the first 50 Fibonacci numbers, and see how long it takes. As for why it is so slow, think about it yourself. Clearly, directly using loop recurrence to get the Fibonacci sequence is a better choice, as shown below.
def fib2(n):
a, b = 0, 1
for _ in range(n):
a, b = b, a + b
return aBesides that, we can also use the lru_cache function in Python's standard-library functools module to optimize the recursive code above. The lru_cache function is a decorator. If we put it above the fib1 function, it can cache the execution results of this function, and avoid a large amount of repeated work during recursion. In this way, the execution performance improves dramatically. You can try outputting the first 50 Fibonacci numbers again and see how long it takes after adding the decorator.
from functools import lru_cache
@lru_cache()
def fib1(n):
if n in (1, 2):
return 1
return fib1(n - 1) + fib1(n - 2)
for i in range(1, 51):
print(i, fib1(i))Tip:
lru_cacheis a decorator with parameters, which is why it is written as@lru_cache()with parentheses. One of its most important parameters ismaxsize, which controls the size of the cache.
Decorators are a distinctive syntax feature in Python. We can use decorators to enhance existing functions, and this is a very useful programming skill. On the other hand, through recursive function calls, we can simplify some complex problems at the code level, but recursive calls must pay attention to the convergence condition and the recursive formula. Only by finding the recursive formula do we have a chance to use recursion, and the convergence condition makes sure the recursion can stop. Function calls save and restore execution context through the stack space in memory, and stack space is usually very small, so if recursion cannot converge quickly, it is very likely to cause a stack overflow error and crash the program.