In this lesson we learn about dunder methods, decorators, and context managers in Python. These tools, when used properly, can help us to write concise, readable, and expressive code when working with complex objects, higher-order functions, or scoped side-effects.
Review the introduction to the Python data model, section 3.1.
Read through the “Special method names” section of the Python Data Model. Specifically, focus on sections 3.3.1, 3.3.2, 3.3.6, 3.3.7, and 3.3.8.
Read through the definition of a decorator in the Python glossary.
Read through the Python documentation for with
-statements,
the context manager type, and the contextlib
module.
Read this informational article on understanding decorators.
Finally, read this blog post for additional information on decorators and context managers.
To learn as much as possible from these exercises, we recommend that you write your response before revealing the provided answers.
What is operator overloading, and how is it achievable in Python? When is it appropriate to use operator overloading?
From Wikipedia:
In computer programming, operator overloading, sometimes termed operator ad hoc polymorphism, is a specific case of polymorphism, where different operators have different implementations depending on their arguments
More specifically to Python, operator overloading allows the programmer to
hook custom classes into fundamental operators of the language like “+” or
“==”. Some specific examples of this are accessing NumPy arrays using slice
notation (via the dunder method __getitem__
), or the fact that string
equality means an element-wise comparison of the code points in a unicode
string while object equality means comparison of each object’s id()
(via
the dunder method __eq__
). Generally speaking, operator overloading in
Python is performed by implementing dunder methods for a class, which define
how the class interacts with various operators.
Dunder methods are a powerful way to make certain operations more “pythonic”; however, they should be used with care. A warning from the Python data model:
When implementing a class that emulates any built-in type, it is important that the emulation only be implemented to the degree that it makes sense for the object being modelled. For example, some sequences may work well with retrieval of individual elements, but extracting a slice may not make sense.
A good rule of thumb for when to use operator overloading: someone who does
not have access to the class definition should be able to guess what is
happening when they see the class used with an operator. If it is not clear
what “+” means for class MyAvatar
, then it is probably not a good use of
__add__
. However, it may be obvious what “+” means for class
ComplexNumber
. Operator overloading should always be used to make
operations simpler to understand, not just more concise.
(As a side note, don’t forget that Python has built-in support for complex
numbers, so there’s no need to write a class ComplexNumber
!)
In a Python shell, observe the output of dir(list())
. Explain what the
output of this call is, and explain the purpose of each printed “dunder”
method.
dir
, when called with an object as its argument, “returns an alphabetized
list of names comprising (some of) the attributes of the given objects, and
of attributes reachble from it.” A more in-depth description of dir
can
be found in the function’s docstring, accessible via dir.__doc__
.
__add__
: Implements the behavior of this object for the +
operator.__class__
: Returns the class of which this object is an instance.__contains__
: Implements the boolean check somevar in self
(i.e. is some value contained in the collection)__delattr__
: Implements the behavior for the delattr()
function, which as the name implies, deletes an attribute from an object.__delitem__
: Implements behavior for del self[x]
, which deletes the item at position x
within self.__dir__
: Describes what is listed out when dir()
is called. Note that we used dir
to get this list of dunder methods in the first place; objects can override what is shown to dir
, so some attributes may not be listed by using this method.__doc__
: Returns the docstring for the object.__eq__
: Implements the boolean check self == some_other
.__format__
:__ge__
: Implements “greater than or equal” comparison__getattribute__
: Implements behavior when getattribute(self, 'someattribute')
is called__getitem__
: Implements indexing behavior, i.e. self[x]
.__gt__
: Implementa “greater than” comparison.__hash__
: Returns a unique hash of the object. Some objects cannot, by nature, have a unique hash that does not change through the lifetime of the object. list
is one of these; you’ll notice that [].__hash__
is actually None
.__iadd__
: Implements +=
, which performs in-place addition.__imul__
Implements *=
, in-place multiplication__init__
: The method called when an object is first created.__init_subclass__
: A method called when a class is subclassed.__iter__
: Implements iterator behavior for self
, allowing the object to be used in for
loops.__le__
: Implementation of “less than or equal”__len__
: Returns the object length when accessed via len(self)
.__lt__
: Implements “less than”__mul__
: Implements multiplication *
.__ne__
: Implements the not-equal check self != someval
. Usually can just be the logical not of __eq__
.__new__
: A class method called when creating a new instance of that class.__repr__
: Returns the official string representation of the object (i.e. for printing with print(self)
)__reversed__
: Returns a reverse iterator.__rmul__
: “Reflected” multiplication, where the right-hand operator is first. This is a fallback method called if the left-hand operator cannot perform normal multiplication.__setattr__
: Implements behavior of setattr(self, 'attr_name', 'attr_value')
__setitem__
: Implements assigning to an index (i.e. self[x] = 5
)__sizeof__
: Specific to lists; returns the size of the list in memory (bytes).__str__
: Returns a string form of the object when called as str(self)
.__subclasshook__
: A method that can be used to alter the result of issubclass(self, SomeClass)
.Dunder methods allow customization of essentially every aspect of an object in Python. They should be used carefully in order to preserve readability and minimize unnecessary complexity.
Write the definition of the repeatme
decorator in the following example:
@repeatme(5)
def some_function():
print("Function has been called")
some_function()
# prints "Function has been called" 5 times
def repeatme(number_of_repeats):
def wrapper_function(f, *args, **kwargs):
def inner():
for i in range(number_of_repeats):
f(*args, **kwargs)
return inner
return wrapper_function
What is the purpose of the context manager, and why is it useful in Python’s data model?
The context manager allows the explicit opening and closing of external resources. This is particularly important in Python since the timing of garbage collection of a reference is implementation-specific and not guaranteed. From the Python data model:
Some objects contain references to “external” resources such as open files or windows. It is understood that these resources are freed when the object is garbage-collected, but since garbage collection is not guaranteed to happen, such objects also provide an explicit way to release the external resource, usually a close() method. Programs are strongly recommended to explicitly close such objects. The ‘try…finally’ statement and the ‘with’ statement provide convenient ways to do this.
Context managers are also more generally useful for any kind of operation that requires defined setup and teardown steps.
What are two different ways you can create a context manager?
A context manager can be created by defining a class with __enter__
and
__exit__
methods. Alternatively, it can be created with a generator
function decorated with the @contextmanager
decorator. The latter
approach is usually preferred since it is visually simpler to understand.
Write the definition of the context manager in this example:
GLOBAL_CONFIG = {
'timeout': 42,
}
# TODO: Write definition of temporary_override
assert GLOBAL_CONFIG['timeout'] == 42
with temporary_override('timeout', 23):
assert GLOBAL_CONFIG['timeout'] == 23
assert GLOBAL_CONFIG['timeout'] == 42
from contextlib import contextmanager
@contextmanager
def temporary_override(key, newValue):
oldValue = GLOBAL_CONFIG[key]
GLOBAL_CONFIG[key] = newValue
yield
GLOBAL_CONFIG[key] = oldValue
Extend the previous context manager so that it restores the global
configuration value in the event of an exception inside the with
block.
# ...continued from previous question
try:
assert GLOBAL_CONFIG['timeout'] == 42
with temporary_override('timeout', 27):
assert GLOBAL_CONFIG['timeout'] == 27
raise Exception('Testing exception handling in context managers.')
except Exception:
assert GLOBAL_CONFIG['timeout'] == 42
The new definition of temporary_override
should be:
@contextmanager
def temporary_override(key, newValue):
oldValue = GLOBAL_CONFIG[key]
GLOBAL_CONFIG[key] = newValue
try:
yield
except:
pass
GLOBAL_CONFIG[key] = oldValue
Open the lesson page in the GitHub editor.
Remove any exercises or learning material that are not useful to the intended audience. Find ways to shorten and clarify the writing. Add generally useful exercises, responses, or learning material. Your improvements will make our training program great!
Create a new branch and pull request and assign it to your lesson mentor. The available lesson mentors are included in the YAML front matter of the lesson. They will set up a time to review your suggested changes and to talk through your exercises for the lesson.
After the review add your self to the "completed" property in the lesson's YAML front matter and merge in your changes!