Functions

The def keyword defines a function. After def come the function name, the parameter list, and a colon that begins the function body:

# a_function.py

def a_function(response):
    val = 0
    if response == "yes":
        print("affirmative")
        val = 1
    print("continuing...")
    return val

print(a_function("no"))
#: continuing...
#: 0
print(a_function("yes"))
#: affirmative
#: continuing...
#: 1

response is a parameter, the name the function gives to what it receives. The value a call passes, "no" or "yes", is an argument.

A string literal that is the first statement under def becomes the function’s docstring, stored on __doc__:

# documented_function.py

def greet(name):
    """Return a greeting for name."""
    return f"Hello, {name}!"

print(greet("Ann"))
#: Hello, Ann!
print(greet.__doc__)
#: Return a greeting for name.

A docstring documents the function for a reader or a tool, not for the interpreter, which stores the text and otherwise ignores it. Metaprogramming reads it back with inspect.getdoc().

The signatures so far give only the function name and the parameter names, with no argument types or return types. Python is dynamically typed, so type errors surface at runtime rather than at compile time:

# add.py
from exceptions import expect

def add(arg1, arg2):
    return arg1 + arg2

print(add(42, 47))
#: 89
print(add("spam ", "eggs"))
#: spam eggs
expect(TypeError, add, 42, "spam")
#: [TypeError] unsupported operand type(s) for +: 'int' and
#: 'str'

A function argument works as long as the function can apply its operations to it. The failure comes from + inside the function body, not from the call that passed the arguments. Nothing checks the arguments on the way in.

expect() is a helper that many of this book’s listings import. It calls its second argument, passing along the arguments that follow, and prints the exception that call raises.

A function can also return a different type from one call to the next:

# flexible_args_and_returns.py

def flexible_args_and_returns(arg):
    if arg == 1:
        return "Hello"
    if arg == "one":
        return 2

print(flexible_args_and_returns(1))
#: Hello
print(flexible_args_and_returns("one"))
#: 2
print(flexible_args_and_returns(2))
#: None

Every Python function returns a value. The third call matches neither test, so the function runs off its end, and a function that runs off its end returns None. A bare return returns None too.

A return with several expressions produces a tuple, which the caller usually unpacks:

def minmax(values):
    return min(values), max(values)

low, high = minmax([3, 1, 4])

The commas build the tuple. The function still returns one object.

Default Arguments

Parameters can have default values, and keyword arguments let callers pass them by name, in any order. A call that names its arguments documents itself:

# default_args.py

def connect(host, port=5432, timeout=30):
    return f"{host}:{port} (timeout {timeout}s)"

print(connect("db.example.com"))  # Uses both defaults
#: db.example.com:5432 (timeout 30s)
# Skip to a keyword
print(connect("db.example.com", timeout=5))
#: db.example.com:5432 (timeout 5s)
# Any order by name
print(connect(port=80, host="web.example.com"))
#: web.example.com:80 (timeout 30s)

Passing by name does not require a default. host has none, and the last call still names it.

At the call site, write every keyword argument after the positional ones. connect(port=80, "web.example.com") is a SyntaxError: positional argument follows keyword argument. The grammar allows a few rarer arrangements that this chapter leaves alone, and keeping the keyword arguments last avoids every one of them.

A parameter with a default cannot come before one with no default. def f(a=1, b): is a SyntaxError: parameter without a default follows parameter with a default. Keyword-only parameters are exempt, because the caller names them.

The Mutable Default Trap

Python evaluates a default value once, when it executes the def, so every call shares that one object. A mutable default therefore carries changes from one call to the next:

# mutable_default.py

def bad_append(item, target=[]):  # The same list every call
    target.append(item)
    return target

print(bad_append(1))
#: [1]
print(bad_append(2))  # Surprise, the default kept the 1
#: [1, 2]
print(bad_append.__defaults__)
#: ([1, 2],)

def good_append(item, target=None):
    if target is None:
        target = []  # A fresh list each call
    target.append(item)
    return target

print(good_append(1))
#: [1]
print(good_append(2))
#: [2]

A mutable default persists because it lives on the function object. No call rebuilds it. __defaults__ holds the tuple of default values, and both calls append to the same list inside it. The default looks like an expression each call evaluates, but Python evaluates it once, at the def.

Underneath, a parameter is another name bound to the caller’s object, the binding that Variables and References describes. When that object is mutable, the caller sees the changes the function makes. bad_append() combines that binding with a default Python builds once, so each call mutates the object the next call uses.

# mutating_arguments.py

def append_all(target, extras):
    target.extend(extras)

mine = [1, 2]
append_all(mine, [3, 4])
print(mine)  # The caller's list changed
#: [1, 2, 3, 4]

def rebind(target):
    target = ["replaced"]  # Rebinds the local name only
    print(target)

rebind(mine)
#: ['replaced']
print(mine)
#: [1, 2, 3, 4]

append_all() calls a method on the object the caller passed, so the caller sees the change. rebind() assigns to the parameter, so the local name points at a new list and the caller’s list stays as it was. Mutating an argument reaches outside the function. Rebinding one does not.

Safe Defaults

good_append() builds a fresh list on every call that omits target, and any function that mutates a parameter with a default must do the same. If the function reads the parameter and leaves it as it was, use an immutable default such as an empty tuple. Calls still share that tuple, and the sharing is harmless because a tuple cannot change:

# immutable_default.py

# An empty tuple is safe: it can't be mutated
def show(items=()):
    for item in items:
        print(item)
    print(f"({len(items)} items)")

show()
#: (0 items)
show(["a", "b"])
#: a
#: b
#: (2 items)

With type hints, such a parameter reads:

items: Sequence[str] = ()

Sentinel Values

The None default in good_append() is a sentinel: a value chosen to mean “the caller passed nothing” rather than to serve as data. Test the parameter with is None rather than truthiness. if not target: also discards an empty list the caller passed on purpose.

None works there because no caller would pass None as a real target. When None is a valid argument, you need a distinct marker. Python 3.15 (PEP 661) adds a sentinel built-in that creates a unique self-describing value for this purpose:

# sentinel_default.py

MISSING = sentinel("MISSING")

def get(data, key, default=MISSING):
    try:
        return data[key]
    except KeyError:
        if default is MISSING:
            return MISSING  # Normally re-raises here
        return default

prefs = {"volume": 3, "mute": None}
print(get(prefs, "volume"))
#: 3
print(get(prefs, "mute"))  # None is a real stored value
#: None
print(get(prefs, "theme"))
#: MISSING
print(get(prefs, "theme", "dark"))
#: dark

Here prefs stores mute as None, so None cannot also mean “not supplied,” and the MISSING sentinel keeps the two cases apart. A stored None comes back untouched, and a missing key with no default comes back as MISSING (a real get() re-raises the KeyError there, as the comment says).

Create a sentinel once and share that name. Two sentinel() calls with the same name build two different objects, so default is sentinel("MISSING") compares against a second object and is always false.

Code written before 3.15 builds the same marker as MISSING = object(): a bare object() is unique, so is tells it apart from every other value, but it prints as <object object at 0x...> where sentinel()’s value prints its name.

Names Inside a Function

A function can read a module-level name, but assigning to that name anywhere in the function makes it local for the whole function. Python decides which names are local when it compiles the function body, so where the assignment sits makes no difference. global tells Python to rebind the module-level name instead:

# function_scope.py

count = 0

def read_only():
    print(count)

def rebinds():
    # A local, unrelated to the module-level count
    count = 99
    print(count)

def writes_global():
    global count
    count += 1

read_only()
#: 0
rebinds()
#: 99
writes_global()
print(count)
#: 1

rebinds() leaves the module-level count alone. Python decides which names are local when it compiles the function, before any of it runs: a name assigned anywhere in the body is local throughout the body. += is an assignment, so if you drop the global from writes_global(), the compiler marks count local for the whole function, and the read half of count += 1 looks up a local that has no value yet. The call raises an UnboundLocalError. global governs rebinding, not reading, so read_only() needs no declaration. Closures covers nonlocal, which rebinds a name in an enclosing function the way global rebinds a module-level name.

A function that rebinds a global couples every caller to that shared, mutable state. Pure Functions shows the cost. To understand one call, you must trace every call before it. Effect Management classifies rebinding a global as a side effect and reading one as a side cause.

Variable Argument Lists

A *args parameter collects extra positional arguments into a tuple, and **kwargs collects extra keyword arguments into a dictionary:

# var_args.py

def report(label, *values, **options):
    print(label, values, options)

report("nums", 1, 2, 3)
#: nums (1, 2, 3) {}
report("point", 3, 4, color="red", size=10)
#: point (3, 4) {'color': 'red', 'size': 10}

The names args and kwargs are convention. The * and ** do the collecting, so *values and **options behave identically.

Unpacking Arguments

* and ** also work in the other direction. At a call site, * unpacks any iterable into separate positional arguments, and ** unpacks a dictionary into keyword arguments.

# unpacking_arguments.py

def f(a, b, c):
    print(a, b, c)

x = [1, 2, 3]
f(*x)
#: 1 2 3
f(*(4, 5, 6))
#: 4 5 6
d = {"a": 3.14, "b": 1.62, "c": 2.72}
f(**d)
#: 3.14 1.62 2.72

def report(label, *values, **options):
    print(label, values, options)

nums = (1, 2, 3)
opts = {"color": "red", "size": 10}
report("point", *nums, **opts)
#: point (1, 2, 3) {'color': 'red', 'size': 10}

def trace(func, *args, **kwargs):
    print("calling", func.__name__)
    return func(*args, **kwargs)

trace(report, "point", *nums, **opts)
#: calling report
#: point (1, 2, 3) {'color': 'red', 'size': 10}

Because collecting and unpacking are inverses, a function can gather arguments it knows nothing about and pass them on unchanged. trace() accepts any call and forwards it, and that is the standard shape of a wrapper. A function is an object like any other, so you can pass report to trace() as an argument, and func.__name__ reads the name of whatever function arrived. expect() has the same shape. It receives a function and the arguments for it, and makes the call. Decorators builds on that forwarding.

A forwarded **kwargs can collide with an argument the wrapped function receives another way. When a key in the unpacked dictionary names a parameter that a positional argument fills, Python raises a TypeError:

# forwarding_collision.py
from exceptions import expect

def report(label, *values, **options):
    print(label, values, options)

def trace(func, *args, **kwargs):
    print("calling", func.__name__)
    return func(*args, **kwargs)

nums = (1, 2, 3)
opts = {"label": "oops", "color": "red"}
expect(TypeError, trace, report, *nums, **opts)
#: calling report
#: [TypeError] report() got multiple values for argument
#: 'label'

opts carries a "label" key, and trace() forwards it as label=. The same func(*args, **kwargs) call spreads nums positionally, so report()’s first parameter, label, also receives 1, and no parameter can take two values. trace() knows nothing about the signature of the function it calls, so it cannot detect the clash. The error arrives one level down, when trace() calls report().

Positional-Only and Keyword-Only Parameters

Two markers in a parameter list control how callers may pass arguments. The markers also decide how much of a signature you commit to keeping. A parameter’s name is part of the contract when a caller can write it, and stays outside the contract when the caller must pass by position.

A / ends the positional-only parameters. You must pass every parameter before it by position, not by name. A * begins the keyword-only parameters. You must pass every parameter after it by name. A *args parameter has the same effect as a bare *. *args absorbs every remaining positional argument, so a parameter declared after it can arrive by name alone.

# param_markers.py
from exceptions import expect

def divide(a, b, /):
    return a / b

print(divide(10, 2))
#: 5.0

def make_user(name, *, admin=False):
    return f"{name} (admin={admin})"

print(make_user("Bob"))
#: Bob (admin=False)
print(make_user("Sue", admin=True))
#: Sue (admin=True)

def tally(label, *values, total=False):
    print(label, values, total)

tally("nums", 1, 2, True)  # [1]
#: nums (1, 2, True) False
tally("nums", 1, 2, total=True)
#: nums (1, 2) True

expect(TypeError, divide, a=10, b=2)  # type: ignore
#: [TypeError] divide() got some positional-only arguments
#: passed as keyword arguments: 'a, b'
expect(TypeError, make_user, "Sue", True)  # type: ignore
#: [TypeError] make_user() takes 1 positional argument but 2
#: were given

The True in the tally() call at [1] joins values like any other positional argument. Only the named form, total=True, reaches total.

Calling divide(a=10, b=2) is an error, because a and b are positional-only. Calling make_user("Sue", True) is an error, because admin is keyword-only. The type checker catches both mistakes before the code runs, so each line carries a # type: ignore saying the misuse is deliberate.

A signature can use every form at once, in one set order: positional-only, positional-or-keyword, *args, keyword-only, **kwargs:

# all_markers.py

def f(a, /, b, *args, c, **kwargs):
    print(a, b, args, c, kwargs)

f(1, 2, 3, 4, c=5, d=6)
#: 1 2 (3, 4) 5 {'d': 6}

a must arrive positionally, c must arrive by name, and b can arrive either way.

In the standard library, many built-in functions and methods take positional-only parameters, such as dict.get(key, default=None, /). Marking a parameter positional-only also keeps its name out of the method’s contract. That matters when a subclass overrides a method. The subclass can rename the parameter, and the type checker accepts the rename.

Lambdas

A lambda is a small anonymous function you write as a single expression. Use one to pass behavior to functions such as sorted(), which accepts a key function, calls it on each element, and orders by the results. When an existing function computes the key, pass that function. key=len needs no lambda. Write a lambda when no existing function computes the key you want, such as ordering by a word’s last letter:

# lambdas.py

words = ["banana", "kiwi", "apple", "fig"]
print(sorted(words, key=len))
#: ['fig', 'kiwi', 'apple', 'banana']
print(sorted(words, key=lambda w: w[-1]))
#: ['banana', 'apple', 'fig', 'kiwi']
square = lambda n: n * n  # Usually prefer def
print(square(9))
#: 81

square = lambda n: n * n gives up the anonymity that is a lambda’s point, and the function’s name in a traceback stays <lambda> where a def shows square. Unlike the body of an anonymous function in many other languages, a lambda body must be a single expression. For anything more complicated, write a separate function.

For a key that reads an index or an attribute, operator.itemgetter() and operator.attrgetter() name the same operation without a lambda. sorted(words, key=operator.itemgetter(-1)) replaces key=lambda w: w[-1] in lambdas.py. For a key that calls a method on each element, operator.methodcaller("lower") replaces lambda w: w.lower(). Write a lambda when the key needs an expression that none of the three builds.

Exercises

This chapter’s solutions give a hint, usually the shape of the code, and a full answer for each exercise.

1. In mutable_default.py, call bad_append(3) a third time and predict the result before checking it. Then change bad_append()’s default from [] to () and explain why that change alone does not fix bad_append() (hint: target.append(item) on a tuple).
2. In sentinel_default.py, replace return MISSING with a bare raise, so a missing key with no default re-raises the KeyError. Confirm that get(prefs, "theme") raises a KeyError and that get(prefs, "theme", None) returns None. Explain why default=None could not serve as the sentinel in this function.
3. In param_markers.py, add a parameter label="result" to divide(), keyword-only, so print(divide(10, 2, label="half")) shows half: 5.0. Confirm that divide(10, 2, "half"), passing label positionally, is now a TypeError.
4. Rewrite report() from var_args.py so it accepts a total=False keyword-only flag that, when true, also prints sum(values). Confirm report("nums", 1, 2, 3, total=True) prints the sum.
5. Write apply_twice(func, value) that returns func(func(value)), then call it with a lambda that appends "!" to a string. Predict the result of apply_twice(lambda s: s + "!", "hi") before running it.
6. Given args = ("point", 3, 4) and opts = {"color": "red"}, call report() from var_args.py so it prints point (3, 4) {'color': 'red'}, passing both containers without naming their contents.
7. Write describe(name, /, **facts) that prints name followed by each keyword argument as key=value, one per line. Confirm that describe(name="Bob") is a TypeError, and explain which marker caused it.
8. In function_scope.py, delete the global count line from writes_global() and predict what a call raises before running it. Then restore it, and instead add print(count) as the first line of rebinds(). Explain why that also raises an UnboundLocalError, although the assignment to count comes after the print.
9. Write clear_by_assignment(target), which assigns target = [], and clear_by_method(target), which calls target.clear(). Pass the same list to each, and predict which call empties the caller’s list before running them.