Function decorators enable the addition of new functionality to a function without altering the function’s original functionality. Prior to reading this post, it is important that you have read and understood the first installment on python functions. The major take away from that tutorial is that python functions are first class objects; a result of this is that:
- Python functions can be passed as arguments to other functions.
- Python functions can be returned from other function calls.
- Python functions can be defined inside other functions resulting in closures.
The above listed properties of python functions provide the foundation needed to explain function decorators. Put simply, function decorators are “wrappers” that let you execute code before and after the function they decorate without modifying the function itself. The structure of this tutorial follows an excellent stack overflow answer to a question on explaining python decorators.
Function Decorators
Function decorators are not unique to python so to explain them, we ignore python function decorator syntax for the moment and instead focus on the essence of function decorators. To understand what decorators do, we implement a very trivial function that is decorated with another trivial function that logs calls to the decorated function. The function decoration is achieved using function composition as shown below (follow the comments):
import datetime
# decorator expects another function as argument
def logger(func_to_decorate):
# A wrapper function is defined on the fly
def func_wrapper():
# add any pre original function execution functionality
print("Calling function: {} at {}".format(func_to_decorate.__name__, datetime.datetime.now()))
# execute original function
func_to_decorate()
# add any post original function execution functionality
print("Finished calling : {}".format(func_to_decorate.__name__))
# return the wrapper function defined on the fly. Body of the
# wrapper function has not been executed yet but a closure
# over the func_to_decorate has been created.
return func_wrapper
def print_full_name():
print("My name is John Doe")
>>>decorated_func = logger(print_full_name)
>>>decorated_func
# the returned value, decorated_func, is a reference to a func_wrapper
<function func_wrapper at 0x101ed2578>
>>>decorated_func()
# decorated_func call output
Calling function: print_full_name at 2015-01-24 13:48:05.261413
# the original functionality is preserved
My name is John Doe
Finished calling : print_full_name
In the trivial example defined above, the decorator adds a new feature, printing some information before and after the original function call, to the original function without altering it. The decorator, logger takes a function to be decorated, print_full_name and returns a function, func_wrapper that calls the decorated function, print_full_name, when it is executed. The function returned, func_wrapper is closed over the reference to the decorated function, print_full_name and thus can invoke the decorated function when it is executing. In the above, calling decorated_func results in print_full_name being executed in addition to some other code snippet that implement new functionality. This ability to add new functionality to a function without modifying the original function is the essence of function decorators. Once this concept is understood, the concept of decorators is understood.
Python Decorators
Now that we hopefully understand the essence of function decorators, we move on to deconstructing python constructs that enable us to define decorators more easily. The previous section describes the essence of decorators but having to use decorators via function compositions as described is cumbersome. Python introduces the @ symbol for decorating functions. Decorating a function using python decorator syntax is achieved as shown below:
@decorator
def a_stand_alone_function():
pass
Calling stand_alone_function now is equivalent to calling decorated_func function from the previous section but we no longer have to define the intermediate decorated_func.
Note that decorators can be applied not just to python functions but also to python classes and class methods but we discuss class and method decorators in a later tutorial.
It is important to understand what the @ symbol does with respect to decorators in python. The @decorator line does not define a python decorator rather one can think of it as syntactic sugar for decorating a function.
I like to define decorating a function as the process of applying an existing decorator to a function. The decorator is the actual function, decorator that adds the new functionality to the original function. According to PEP 318, the following decorator snippet
@dec2
@dec1
def func(arg1, arg2, ...):
pass
is equivalent to
def func(arg1, arg2, ...):
pass
func = dec2(dec1(func))
without the intermediate func argument. In the above, @dec1 and @dec2 are the decorator invocations. Stop, think carefully and ensure you understand this. dec1 and dec2 are function object references and these are the actual decorators. These values can even be replaced by any function call or a value that when evaluated returns a function that takes another function. What is of paramount importance is that the name reference following the @ symbol is a reference to a function object (for this tutorial we assume this should be a function object but in reality it should be a callable object) that takes a function as argument. Understanding this profound fact will help in understanding python decorators and more involved decorator topics such as decorators that take arguments.
Function Arguments For Decorated Functions
Arguments can be passed to functions that are being decorated by simply passing this function into the function that wraps, i.e the inner function returned when the decorator is invoked, the decorated function. We illustrate this with an example below:
import datetime
# decorator expects another function as argument
def logger(func_to_decorate):
# A wrapper function is defined on the fly
def func_wrapper(*args, **kwargs):
# add any pre original function execution functionality
print("Calling function: {} at {}".format(func_to_decorate.__name__, datetime.datetime.now()))
# execute original function
func_to_decorate(*args, **kwargs)
# add any post original function execution functionality
print("Finished calling : {}".format(func_to_decorate.__name__))
# return the wrapper function defined on the fly. Body of the
# wrapper function has not been executed yet but a closure over
# the func_to_decorate has been created.
return func_wrapper
@logger
def print_full_name(first_name, last_name):
print("My name is {} {}".format(first_name, last_name))
print_full_name("John", "Doe")
Calling function: print_full_name at 2015-01-24 14:36:36.691557
My name is John Doe
Finished calling : print_full_name
Note how we use *args and **kwargs in defining the inner wrapper function; this is for the simple reason that we cannot know before hand what arguments are being passed to a function being decorated.
Decorator Function with Function Arguments
We can also pass arguments to the actual decorator function but this is more involved than the case of passing functions to decorated functions. We illustrate this with quite an example below:
# this function takes arguments and returns a function.
# the returned functions is our actual decorator
def decorator_maker_with_arguments(decorator_arg1):
# this is our actual decorator that accepts a function
def decorator(func_to_decorate):
# wrapper function takes arguments for the decorated
# function
def wrapped(function_arg1, function_arg2) :
# add any pre original function execution
# functionality
print("Calling function: {} at {} with decorator arguments: {} and function arguments:{} {}".\
format(func_to_decorate.__name__, datetime.datetime.now(), decorator_arg1, function_arg1, function_arg2))
func_to_decorate(function_arg1, function_arg2)
# add any post original function execution
# functionality
print("Finished calling : {}".format(func_to_decorate.__name__))
return wrapped
return decorator
@decorator_maker_with_arguments("Apollo 11 Landing")
def print_name(function_arg1, function_arg2):
print ("My full name is -- {} {} --".format(function_arg1, function_arg2))
>>> print_name("Tranquility base ", "To Houston")
Calling function: print_name at 2015-01-24 15:03:23.696982 with decorator arguments: Apollo 11 Landing and function arguments:Tranquility base To Houston
My full name is -- Tranquility base To Houston --
Finished calling : print_name
As mentioned previously, the key to understanding what is going on with this is to note that we can replace the reference value following the @ in a function decoration with any value that evaluates to a function object that takes another function as argument. In the above, the value returned by the function call, decorator_maker_with_arguments("Apollo 11 Landing") , is the decorator. The call evaluates to a function, decorator that accepts a function as argument. Thus the decoration ‘@decorator_maker_with_arguments(“Apollo 11 Landing”)’ is equivalent to @decorator but with the decorator, decorator , closed over the argument, Apollo 11 Landing by the decorator_maker_with_arguments function call. Note that the arguments supplied to a decorator can not be dynamically changed at run time as they are executed on script import.
Functools.wrap
Using decorators involves swapping out one function for another. A result of this is that meta information such as docstrings in the swapped out function are lost when using a decorator with such function. This is illustrated below:
import datetime
# decorator expects another function as argument
def logger(func_to_decorate):
# A wrapper function is defined on the fly
def func_wrapper():
# add any pre original function execution functionality
print("Calling function: {} at {}".format(func_to_decorate.__name__, datetime.datetime.now()))
# execute original function
func_to_decorate()
# add any post original function execution functionality
print("Finished calling : {}".format(func_to_decorate.__name__))
# return the wrapper function defined on the fly. Body of the
# wrapper function has not been executed yet but a closure
# over the func_to_decorate has been created.
return func_wrapper
@logger
def print_full_name():
"""return john doe's full name"""
print("My name is John Doe")
>>> print(print_full_name.__doc__)
None
>>> print(print_full_name.__name__)
func_wrapper
In the above example an attempt to print the documentation string returns None because the decorator has swapped out the print_full_name function with the func_wrapper function that has no documentation string.
Even the function name now references the name of the wrapper function rathe than the actual function. This, most times, is not what we want when using decorators. To work around this python functools module provides the wraps function that also happens to be a decorator. This decorator is applied to the wrapper function and takes the function to be decorated as argument. The usage is illustrated below:
import datetime
from functools import wraps
# decorator expects another function as argument
def logger(func_to_decorate):
@wraps(func_to_decorate)
# A wrapper function is defined on the fly
def func_wrapper(*args, **kwargs):
# add any pre original function execution functionality
print("Calling function: {} at {}".format(func_to_decorate.__name__, datetime.datetime.now()))
# execute original function
func_to_decorate(*args, **kwargs)
# add any post original function execution functionality
print("Finished calling : {}".format(func_to_decorate.__name__))
# return the wrapper function defined on the fly. Body of the
# wrapper function has not been executed yet but a closure over
# the func_to_decorate has been created.
return func_wrapper
@logger
def print_full_name(first_name, last_name):
"""return john doe's full name"""
print("My name is {} {}".format(first_name, last_name))
>>> print(print_full_name.__doc__)
return john doe's full name
>>>print(print_full_name.__name__)
print_full_name
Applications of Decorators
Decorators have a wide variety of applications in python and these can not all be covered in this article. Some examples of applications of decorators include:
- Memoization which is the caching of values to prevent recomputing such values if the computation is expensive; A memoization decorator can be used to decorate a function that performs the actual calculation and the added feature is that for a given argument if the result has been computed previously then the stored value is returned but if it has not then it is computed and stored before returned to the caller.
- In web applications, decorators can be used to protect end points that require authentication; an endpoint is protected with a decorator that checks that a user is authenticated when a request is made to the endpoint. Django a popular web application framework makes use of decorators for managing caching and views permissions.
- Decorators can also provide a clean way for carrying out household tasks such as logging function calls, timing functions etc.
The use of decorators is a very large playing field that is unique to different situations. The python decorator library provides a wealth of use cases of python decorators. Browsing through this collection will provide insight into practical uses of python decorators.