Classes and Objects II: Descriptors
Descriptors are an esoteric but integral part of the python programming language. They are used widely in the core of the python language and a good grasp of descriptors provides a python programmer with an extra trick in his or her toolbox. To set the stage for the discussion of descriptors I describe some scenarios that a programmer may encounter in his or her daily programming activities; I then go ahead to explain what descriptors are and how they provide elegant solutions to these scenarios. For this writeup, I refer to python version using new style classes.
Consider a program in which we need to enforce strict type checking for object attributes. Python is a dynamic languages and thus does not support type checking but this does not prevent us from implementing our own version of type checking regardless of how rudimentary it may be. The conventional way to go about type checking object attributes may take the form shown below:
def __init__(self, name, age): if isinstance(str, name): self.name = name else: raise TypeError("Must be a string") if isinstance(int, age): self.age = age else: raise TypeError("Must be an int")
The above method is one method of enforcing such type checking but as the arguments increase in number it gets cumbersome. Alternatively, we could create a
type_check(type, val)function that is called in the
__init__method before assignment but then how would we easily implement such checking when we want to set the attribute value somewhere else. A quick solution that comes to mind is the
setterspresent in Java but that is un-pythonic and cumbersome.
Consider a program in which we want to create attributes that are initialized once at run-time and then become read-only. One could also think of ways of implementing this using python special methods but once again such implementation would be unwieldy and cumbersome.
Finally, imagine a program in which we wanted to somehow customize object attribute access. This maybe to log such attribute access for example. Once again, it is not too difficult to come up with a solution to this although such solution maybe unwieldy and not re-useable.
All the above mentioned issues are all linked together by the fact that they are all related to attribute references; we are trying to customize attribute access.
Descriptors provides solutions to the above listed issues that are elegant, simple, robust and re-useable. Simply put, a descriptor is an object that represents the value of an attribute. This means that if an account object has an attribute
name, a descriptor is another object that can be used to represent the value held by that attribute,
name. A descriptor is any object that implements any of the
__delete__ special methods of the descriptor protocol. The signature for each of these methods is shown below:
descr.get(self, obj, type=None) –> value
descr.__set__(self, obj, value) --> None descr.__delete__(self, obj) --> None ```
Objects implementing the
__get__ method are non-data descriptors meaning they can only be read from after initialization while objects implementing the
__set__ are data descriptors which means such attribute are writable.
To get a better understanding of descriptors we provide descriptor based solution to the issues mentioned. Implementing type checking on an object attribute using python descriptors is then a very simple task. A decorator implementing this type checking is shown below:
class TypedProperty(object): def __init__(self, name, type, default=None): self.name = "_" + name self.type = type self.default = default if default else type() def __get__(self, instance, cls): return getattr(instance, self.name, self.default) def __set__(self,instance,value): if not isinstance(value,self.type): raise TypeError("Must be a %s" % self.type) setattr(instance,self.name,value) def __delete__(self,instance): raise AttributeError("Can't delete attribute") class Foo(object): name = TypedProperty("name",str) num = TypedProperty("num",int,42) >> acct = Foo() >> acct.name = "obi" >> acct.num = 1234 >> print acct.num 1234 >> print acct.name obi # trying to assign a string to number fails >> acct.num = '1234' TypeError: Must be a <type 'int'>
In the example, we implement a descriptor,
TypedProperty and this descriptor class enforces type checking for any attribute of a class which it is used to represent. It is important to note that descriptors can only be legally defined at the class level rather than instance level i.e. in
__init__ method as shown in the example above.
When the any attribute of a
Foo class instance is accessed, the descriptor calls its
__get__ method. Notice that the first argument to the
__get__ method is the object from which the attribute the descriptor represents is being referenced. When the attribute is assigned to, the descriptor calls its
__set__ method. To understand why descriptors can be used to represent object attributes, we need to understand the way attribute reference resolution is carried out in python. For objects, the mechanics for attribute resolution is in
object.__getattribute__(). This method transforms
b.x into type(b).__dict__['x'].__get__(b, type(b)). The resolutions then searches for the attribute using a precedence chain that gives data descriptors found in class dict priority over instance variables, instance variables priority over non-data descriptors, and assigns lowest priority to getattr() if provided. This precedence chain can be overridden by defining custom
__getattribute__ methods for a given object class.
With a firm understanding of the mechanics of descriptors, it is easy to imagine elegant solutions to the second and third issues raised in the previous section. Implementing a read only attribute with descriptors becomes a simple case of implementing a data descriptor i.e descriptor with no
__set__ method`. The issue of customizing access, though trivial in this instance, would just involve adding the required functionality in the
Having to define descriptor classes each time we want to use them is cumbersome. Python properties provide a concise way of adding data descriptors to attributes in python. A property signature is given below:
property(fget=None, fset=None, fdel=None, doc=None) -> property attribute
fdel are the getter, setter and deleter methods for such class. We illustrate creating properties with an example below:
class Accout(object): def __init__(self): self._acct_num = None def get_acct_num(self): return self._acct_num def set_acct_num(self, value): self._acct_num = value def del_acct_num(self): del self._acct_num acct_num = property(get_acct_num, set_acct_num, del_acct_num, "Account number property.")
acct is an instance of
acct.acct_num will invoke the getter,
acct.acct_num = value will invoke the setter and
del acct_num.acct_num will invoke the deleter.
The property object and functionality can be implemented in python as illustrated in Descriptor How-To Guide using the descriptor protocol as shown below :
class Property(object): "Emulate PyProperty_Type() in Objects/descrobject.c" def __init__(self, fget=None, fset=None, fdel=None, doc=None): self.fget = fget self.fset = fset self.fdel = fdel if doc is None and fget is not None: doc = fget.__doc__ self.__doc__ = doc def __get__(self, obj, objtype=None): if obj is None: return self if self.fget is None: raise AttributeError("unreadable attribute") return self.fget(obj) def __set__(self, obj, value): if self.fset is None: raise AttributeError("can't set attribute") self.fset(obj, value) def __delete__(self, obj): if self.fdel is None: raise AttributeError("can't delete attribute") self.fdel(obj) def getter(self, fget): return type(self)(fget, self.fset, self.fdel, self.__doc__) def setter(self, fset): return type(self)(self.fget, fset, self.fdel, self.__doc__) def deleter(self, fdel): return type(self)(self.fget, self.fset, fdel, self.__doc__)
Python also provides the
@property decorator that can be used to create read only attributes. A property object has getter, setter, and deleter decorator methods that can be used to create a copy of the property with the corresponding accessor function set to the decorated function. This is best explained with an example:
class C(object): def __init__(self): self._x = None @property # the x property. the decorator creates a read-only property def x(self): return self._x @x.setter # the x property setter makes the property writeable def x(self, value): self._x = value @x.deleter def x(self): del self._x
If we wanted to make the property read-only then we would leave out the
Descriptors see wide application in the python language itself. Python functions, class methods, static methods are all examples of non-data descriptors. Descriptor How-To Guide provides a basic description of how the listed python objects are implemented using descriptors.
Descriptor How-To Guide
Python Essential Reference 4th Edition; David Beazley
Caching in python with a descriptor and a decorator
Inside story on new style classes