Chapter 17

Metaprogramming

Ordinary code makes objects. Metaprogramming works one level up, on the classes that make them.

Every object comes from another, special object. These special objects are classes, and you configure them to produce the objects you want.

Classes are also objects, and you can modify objects.

# modify_class.py
from display import display_object

class Clay:
    pass

display_object(Clay)
#: [Attributes]
#:   None
#: [Methods]
#:   None

x = Clay()
display_object(x)
#: [Attributes]
#:   None
#: [Methods]
#:   None

Clay.n = 42  # type: ignore
display_object(Clay)
#: [Attributes]
#:   • n = 42 [CV]
#: [Methods]
#:   None

Clay.m = lambda self: f"{self.n = }"  # type: ignore
display_object(Clay)
#: [Attributes]
#:   • n = 42 [CV]
#: [Methods]
#:   • m(self)

print(x.m())  # type: ignore
#: self.n = 42

display_object(x)
#: [Attributes]
#:   • n = 42 [CV]
#: [Methods]
#:   • m(self)

print(vars(x))
#: {}

x sees the changes you make to the class after x’s creation. The instance stays as it was. The last line shows its instance dictionary still empty. Attribute lookup on an instance falls through to its class, so a change to a class reaches every object of that class, including those created before the change.

What creates these “class” objects? Other special objects, called metaclasses. The default metaclass is type, and it almost always does the right thing. You can customize how Python produces classes by running extra code or injecting members as it builds each class. That is metaclass programming.

You have used metaclasses without writing one. abc.ABCMeta builds abc.ABC, and makes a class with an unimplemented abstract method refuse instantiation. enum.EnumType builds each Enum subclass, turning every class-body assignment into a member and making for c in Color walk them. Iterating over a class is behavior on the class object, and a metaclass puts behavior there. An ordinary class cannot.

You rarely need a metaclass. It is a fascinating tool and tempting to use, but simpler hooks cover almost every case a metaclass handled before those hooks existed:

__init_subclass__() runs at subclass creation. It replaces most “do something at each class definition” metaclasses.
__set_name__() lets a class attribute learn its own name, at class-creation time.
Class decorators transform a class after Python builds it.

Use a metaclass only when these cannot do the job.

Building classes by hand shows what a class statement does. Then come the simpler hooks, and metaclasses for the jobs that still need them. The inspect module closes the chapter from the other side, reading class structure instead of changing it.

Generating Classes with `type()`

Since metaclasses create classes, you can call the metaclass yourself. type() with one argument gives the type of an existing object. type() with three arguments creates a new class. These arguments are the name, a tuple of base classes, and a namespace dictionary of fields and methods. A class definition is shorthand for calling type():

# class_via_type.py
class C:
    pass

D = type("D", (), {})  # The same construction, by hand

print(type(C), type(D))
#: <class 'type'> <class 'type'>
# Both inherit object:
print(C.__bases__, D.__bases__)
#: (<class 'object'>,) (<class 'object'>,)
# Both make ordinary instances:
print(isinstance(C(), C), isinstance(D(), D))
#: True True

You can add bases, fields, and methods the same way:

# my_list.py
from display import display_object

def howdy(self, you: str) -> None:
    print(f"Howdy, {you}")

MyList = type("MyList", (list,), dict(x=42, howdy=howdy))

display_object(MyList)
#: [Attributes]
#:   • x = 42 [CV]
#: [Methods]
#:   • append(self, object, /)
#:   • clear(self, /)
#:   • copy(self, /)
#:   • count(self, value, /)
#:   • extend(self, iterable, /)
#:   • howdy(self, you: str) -> None
#:   • index(self, value, start=0, stop=92233720368547758...
#:   • insert(self, index, object, /)
#:   • pop(self, index=-1, /)
#:   • remove(self, value, /)
#:   • reverse(self, /)
#:   • sort(self, /, *, key=None, reverse=False)

ml = MyList()
ml.append("Camembert")
print(ml)
#: ['Camembert']
print(ml.x)
#: 42
ml.howdy("John")
#: Howdy, John

print(ml.__class__.__class__)
#: <class 'type'>

Because MyList inherits list, it gets all the methods from list.

Printing the class of the class produces the metaclass.

A Family of Generated Classes

Generating classes programmatically with type() pays off when a family of classes differs only by name. A greenhouse controller runs scheduled events, one class per kind of event, and a dict comprehension builds all of them:

# eager_event_classes.py
from collections.abc import Callable
from dataclasses import dataclass
from typing import Final

@dataclass
class Event:
    action: str
    hour: int
    minute: int

type EventMaker = Callable[[int, int], Event]
NAMES: Final[tuple[str, ...]] = (
    "ThermostatDay", "ThermostatNight", "LightOn",
    "LightOff", "WaterOn", "WaterOff", "RingBell",
)

def make(name: str) -> EventMaker:
    def init(self: Event, hour: int, minute: int) -> None:
        Event.__init__(self, name, hour, minute)
    new_cls = type(name, (Event,), {"__init__": init})
    return new_cls

makers = {name: make(name) for name in NAMES}
print(len(makers))
#: 7
light = makers["LightOn"](1, 0)
water = makers["WaterOff"](2, 0)
print(light)
#: LightOn(action='LightOn', hour=1, minute=0)
print(isinstance(light, Event))
#: True
print(type(light) is type(water))
#: False

Each generated class is a real type, not a label. light and water are both Event instances, so isinstance(light, Event) is True, but type(light) is type(water) is False. They belong to distinct subclasses, and isinstance() tells them apart.

Type checkers follow a type() call to different depths. ty reads the bases and a namespace written as a dict literal. It takes the constructor from the "__init__" entry, so it knows each generated class takes an hour and a minute, and it checks return new_cls against EventMaker, the two-argument signature make() declares. If init() gains a third parameter, ty reports an invalid-return-type that names the extra parameter. Pyright builds the class from its bases alone, so it gives the class Event’s three-argument constructor and rejects return new_cls. Under Pyright the listing needs cast(EventMaker, new_cls). Under mypy the result is an unknown class, so mypy checks nothing and accepts either form. Under ty a cast would replace a check with a claim, the distinction Where Enforcement Lives draws. The cast states the signature, and nothing confirms it.

make() exists so that each init() closes over its own name. A lambda written inline in the comprehension closes over the comprehension’s variable instead, so every generated class records the final name, RingBell, as its action: the late-binding trap that late_binding.py demonstrates.

init() calls Event.__init__(self, ...) by name instead of super().__init__(...). It is a nested function, not a method defined inside a class statement, so the compiler gives it no __class__ cell for zero-argument super() to use.

Building Each Class on First Lookup

The dict comprehension in eager_event_classes.py builds all seven classes whether the schedule uses them or not. Seven classes are few enough to build up front. With hundreds, the schedule uses few of the classes the comprehension builds. So the next version delays building each class until the first lookup asks for it. Delaying the build takes a dict subclass and a placeholder for the classes not yet built:

# greenhouse.py
from collections.abc import Callable
from dataclasses import dataclass
from pathlib import Path
from typing import ClassVar, cast

type EventMaker = Callable[[int, int], Event]
NOT_CREATED = sentinel("NOT_CREATED")

class EventMakers(dict[str, EventMaker | NOT_CREATED]):
    def __getitem__(self, class_name: str) -> EventMaker:
        if class_name not in self:
            raise KeyError(
                f"Unknown event class: {class_name!r}")
        maker = super().__getitem__(class_name)
        if maker is NOT_CREATED:
            print(f"Creating {class_name}")
            # Local function to pass to type constructor:
            def init(self: Event,
                     hour: int, minute: int) -> None:
                Event.__init__(
                    self, class_name, hour, minute)
            new_cls = type(class_name, (Event,),
                           {"__init__": init})
            maker = new_cls
            self[class_name] = maker
        return maker

@dataclass
class Event:
    action: str
    hour: int
    minute: int
    # Registry of all Events
    events: ClassVar[list[Event]] = []
    _event_maker: ClassVar[EventMakers] = EventMakers({
        name : NOT_CREATED  # Dict key : value
        for name in (
            "ThermostatDay", "ThermostatNight",
            "LightOn", "LightOff",
            "WaterOn", "WaterOff",
            "RingBell",
        )
    })

    def __post_init__(self) -> None:
        Event.events.append(self)

    @staticmethod
    def load_schedule(path: Path) -> None:
        lines = [
            line for line in path.read_text().splitlines()
            if line.strip() and not line.startswith("#")
        ]
        for line in lines:
            class_name, hour, minute = (
                line.replace(":", " ").split())
            Event._event_maker[class_name](
                int(hour), int(minute))

    @staticmethod
    def run_events() -> None:
        bell = cast(
            type, Event._event_maker["RingBell"])
        for e in sorted(
                Event.events,
                key=lambda e: (e.hour, e.minute)):
            prefix = "* " if isinstance(e, bell) else ""
            line = f"{e.hour}:{e.minute:02d}: {e.action}"
            print(prefix + line)

EventMakers subclasses dict so the laziness is invisible at the call site. Event._event_maker[class_name] reads as an ordinary lookup, and the overridden __getitem__() decides whether that lookup returns a class or builds one first. The alternative, a make_event() function, pushes that decision into every caller.

Event._event_maker starts out holding the seven legitimate event names, each paired with the NOT_CREATED sentinel as a placeholder. Populating that dict reserves the names and builds nothing yet, so EventMakers.__getitem__() can check a class_name against those names before building anything. The dict’s value type is EventMaker | NOT_CREATED, naming the sentinel value rather than the generic sentinel class, so ruling out one member with maker is NOT_CREATED leaves EventMaker in the other branch.

Now the end user needs only to write and maintain the schedule file:

# schedule.txt
LightOff 2:00
LightOn 1:00
RingBell 7:00
LightOn 8:00

load_schedule() reads that file, filtering out blank lines and comments, then builds an Event from each resulting line. line.replace(":", " ").split() turns "LightOn 1:00" into three strings in a single step, replacing the colon with a second space before splitting on whitespace. Event._event_maker[class_name] gets the class object that builds that Event. The first time a lookup asks for an event type, EventMakers builds the class and registers it under its name. An unknown name raises a KeyError, as a missing key in any dict does, so a try: ... except KeyError around a lookup catches it.

Running the schedule prints a line for each class as EventMakers builds it, then the events in time order:

# greenhouse_demo.py
from pathlib import Path
from greenhouse import Event

Event.load_schedule(Path("schedule.txt"))
#: Creating LightOff
#: Creating LightOn
#: Creating RingBell
Event.run_events()
#: 1:00: LightOn
#: 2:00: LightOff
#: * 7:00: RingBell
#: 8:00: LightOn

The schedule names three of the seven declared event types. EventMakers builds those three: seven classes declared, three built.

run_events() puts the type distinction to work. It fetches the RingBell class through _event_maker, the same lookup load_schedule() uses, and marks every event isinstance() recognizes as a RingBell with a leading *. That is what a distinct subclass adds. Each event carries its kind as its type, where isinstance() or a match class pattern can test it.

Calling the base class, Event(class_name, hour, minute), still produces the right field values, but every entry shares one type, and run_events() has no RingBell class for isinstance() to test. The * marker depends on RingBell being a distinct class, not just a distinct name.

Generating Classes with `exec()`

The type() approach in greenhouse.py builds a class from a name, a tuple of bases, and a namespace dict. A second way is to write an ordinary class statement in an f-string, then exec() that string as code. That class body, held in klass below, is easier to read and modify than a namespace dict:

# commander.py
from collections.abc import Callable
from dataclasses import dataclass
from typing import Any, ClassVar, cast
from exceptions import expected

@dataclass
class Command:
    label: str
    KNOWN_COMMANDS: ClassVar[set[str]] = {
        "Start", "Stop", "Pause"}

    def run(self) -> str:
        return f"Running {self.label}"

    @classmethod
    def make_class(
        cls, class_name: str) -> Callable[[], Command]:
        if class_name not in cls.KNOWN_COMMANDS:
            raise ValueError(
                f"Unknown command: {class_name!r}")
        klass = f"""
class {class_name}(Command):
    def __init__(self) -> None:
        super().__init__("{class_name}")
"""
        namespace: dict[str, Any] = {"Command": Command}
        exec(klass, namespace)
        return cast(Callable[[], Command],
                    namespace[class_name])

if __name__ == "__main__":
    for name in ("Start", "Stop", "Pause"):
        command_class = Command.make_class(name)
        print(command_class().run())
    with expected(ValueError):
        Command.make_class("Reset")
#: Running Start
#: Running Stop
#: Running Pause
#: [ValueError] Unknown command: 'Reset'

make_class() execs klass into a private namespace dict, not the module’s namespace, and seeds it with {"Command": Command} so the generated class can find its base. The type checker can’t see into the string, so namespace[class_name] is Any to it. exec() also drops a __builtins__ entry into any globals mapping that lacks one, and that entry is the second reason namespace carries the annotation dict[str, Any]. cast(Callable[[], Command], ...) records the actual no-argument signature at the one place that creates the class.

Unlike EventMakers, make_class() caches nothing. Calling make_class("Start") twice builds two distinct classes.

__init__()’s definition sits textually inside a class block. The compiler treats a block that arrived as a string the same as one read from a file. Sitting inside a class body is the difference from greenhouse.py, whose init() is a nested function rather than a method in a class body, so init() gets no __class__ cell and cannot use zero-argument super().

The Injection Risk

The klass string is the danger in this approach. exec() runs its argument with the full power of the language, and klass splices class_name into source text as-is. An unvalidated name containing a newline and a second statement could then break out of the class block and run anything, the same way an unescaped value breaks out of a hand-built SQL query. The KNOWN_COMMANDS check closes that hole. Only three known names reach the template. EventMakers carries no such risk, because type(class_name, (Event,), ...) treats class_name as a string value, not as source code. Treat exec() and eval() like string-built SQL: safe on values you’ve validated, dangerous on anything that reaches the program from outside, unchecked.

Generated Classes Cannot Be Pickled

Both generators, type() and exec(), have a second limitation, unrelated to injection. exec()’s private namespace has no __name__ key, so the class it creates gets __module__ set to "builtins". pickle.dumps() on an instance then raises a PicklingError, because pickle looks the class up as builtins.Start and does not find it, and inspect.getsource() raises a TypeError, because a built-in class carries no source. type()-built classes fail differently, but just as completely. LightOn gets __module__ set to eager_event_classes correctly, but it lives in the makers dict, not as a module attribute. Pickle therefore looks for eager_event_classes.LightOn and does not find that either, and inspect.getsource() raises an OSError instead. Neither generator’s classes survive a round trip through pickle, and neither yields source to inspect.getsource(). A class built this way serves the process that built it. It is not for storage or introspection.

Self-Registration of Subclasses

Often a base class needs to keep track of its subclasses. Tracking subclasses is the textbook justification for a metaclass. Python calls __init_subclass__() automatically for every new subclass, so a base class can register its own subclasses in a few lines. This example tracks the “leaf” subclasses (those with no subclasses of their own), using __init_subclass__() instead of a metaclass:

# init_subclass.py
from typing import ClassVar

class Color:
    registry: ClassVar[set[type[Color]]] = set()

    def __init_subclass__(cls, **kwargs: object) -> None:
        super().__init_subclass__(**kwargs)
        Color.registry.add(cls)
        # Keep only the leaves
        Color.registry -= set(cls.__bases__)

class Blue(Color):
    pass
class Red(Color):
    pass
class Green(Color):
    pass
print(sorted(c.__name__ for c in Color.registry))
#: ['Blue', 'Green', 'Red']

class PhthaloBlue(Blue):
    pass
class CeruleanBlue(Blue):
    pass
print(sorted(c.__name__ for c in Color.registry))  # [1]
#: ['CeruleanBlue', 'Green', 'PhthaloBlue', 'Red']

# A second, independent hierarchy keeps its own registry:
class Shape:
    registry: ClassVar[set[type[Shape]]] = set()

    def __init_subclass__(cls, **kwargs: object) -> None:
        super().__init_subclass__(**kwargs)
        Shape.registry.add(cls)
        Shape.registry -= set(cls.__bases__)

class Round(Shape):
    pass
class Square(Shape):
    pass
class Circle(Round):
    pass
print(sorted(c.__name__ for c in Shape.registry))
#: ['Circle', 'Square']

For each new subclass, __init_subclass__() adds it to the registry and removes its base classes, so only the current leaves remain. That is why Blue is absent from the Color print at [1]. Creating PhthaloBlue and CeruleanBlue removes their base Blue, leaving those two leaves beside Green and Red. For the same reason, Round is missing from the Shape registry. Creating Circle, a subclass of Round, removes Round, leaving Circle and Square. None of this needs a metaclass.

__init_subclass__() is implicitly a class method. Its first argument is the new subclass. __init_subclass__() runs for classes derived from the class whose body defines it, and not for that class, so neither Color nor Shape appears in its own registry.

The keyword arguments come from the subclass header. Writing class Blue(Color, shade="cool"): delivers shade="cool" to __init_subclass__(), so a subclass can configure its own registration. The base that declares the keyword names it as a parameter, def __init_subclass__(cls, shade: str = "", **kwargs: object), so it never reaches kwargs. super().__init_subclass__(**kwargs) passes the rest up the chain, so a base further up can take the keywords it declared, and a keyword no base declares reaches object.__init_subclass__(), which accepts none and raises a TypeError.

Testing shows that each registry holds only its current leaf classes, and that the two registries share no class:

# test_init_subclass.py
import init_subclass

def test_color_registry_holds_only_leaves() -> None:
    leaves = {c.__name__
              for c in init_subclass.Color.registry}
    assert leaves == {"Red", "Green", "PhthaloBlue",
                      "CeruleanBlue"}

def test_independent_hierarchies_have_separate_registries(
) -> None:
    shapes = {c.__name__
              for c in init_subclass.Shape.registry}
    assert "Round" not in shapes
    assert shapes == {"Square", "Circle"}
    # Neither registry leaks into the other
    assert init_subclass.Shape.registry.isdisjoint(
        init_subclass.Color.registry)

The mechanism is reliable. The registries built on it fail in two ways that have nothing to do with __init_subclass__(). Factory covers both: a class in a module nobody imports stays out of the registry, and keying on cls.__name__ lets two same-named classes overwrite each other.

Attributes on a Function

A function is an object, and like most objects it carries a __dict__. Assigning f.x = 1 stores 1 in f.__dict__ under the key "x", and the entry stays for as long as the function exists. A decorator can use that storage to mark a method. A method decorator runs as Python executes the class body, before the class object exists, so it cannot register the method with its class. It can leave a mark on the function instead, and __init_subclass__() collects the marks once Python builds the class:

# marked_methods.py
from collections.abc import Callable
from typing import Any, ClassVar

def on[F: Callable[..., object]](
        event: str
) -> Callable[[F], F]:
    def mark(func: F) -> F:
        func.__dict__["event"] = event
        return func
    return mark

class Widget:
    handlers: ClassVar[dict[str, Callable[..., Any]]] = {}

    def __init_subclass__(cls, **kwargs: object) -> None:
        super().__init_subclass__(**kwargs)
        marked = {
            attr.__dict__["event"]: attr
            for attr in vars(cls).values()
            if "event" in getattr(attr, "__dict__", {})
        }
        cls.handlers = {**cls.handlers, **marked}

    def dispatch(self, event: str) -> None:
        self.handlers[event](self)

class Button(Widget):
    @on("click")
    def press(self) -> None:
        print("pressed")

    @on("hover")
    def glow(self) -> None:
        print("glowing")

    def label(self) -> str:
        return "OK"

class SubmitButton(Button):
    @on("submit")
    def send(self) -> None:
        print("sent")

print(sorted(Button.handlers))
#: ['click', 'hover']
print(sorted(SubmitButton.handlers))
#: ['click', 'hover', 'submit']
Button().dispatch("click")
#: pressed
print(Button.press.__dict__)
#: {'event': 'click'}
# ty: Function `press` has no attribute `event`:
# print(Button.press.event)

on() takes the event name and returns mark(), the decorator that does the work. on()’s type parameter F has the bound Callable[..., object], which any function satisfies, and mark() returns the same F it receives, so press() keeps its own signature through the decoration. @on("click") stores "click" under the key "event" in the __dict__ of press(), then returns press() unchanged. When the class Button statement finishes, __init_subclass__() walks vars(cls), the new class’s own namespace, and files each function that carries an "event" key under its event. label() carries no mark, so handlers leaves it out. On the right side of the assignment, cls.handlers still resolves to the base class’s dictionary, so each new table starts from the inherited handlers: SubmitButton keeps click and hover and adds submit. The namespace also holds the bookkeeping entries every class carries, such as __module__, a string with no __dict__, and the {} default in getattr() passes over them. handlers holds the functions the class body defined, not bound methods, so dispatch() passes self to the function it finds.

The listing writes func.__dict__["event"] rather than func.event, and the difference matters to the type checker. A function’s type declares a closed set of attributes, so the type checker reports Button.press.event as an unresolved attribute, and it reports the assignment func.event = event the same way. Every object’s type declares __dict__ as a dict[str, Any], so indexing it type-checks.

The standard library marks functions the same way. @abstractmethod sets __isabstractmethod__ = True on the function, and ABCMeta collects the marked names into the class’s __abstractmethods__, which is how an abstract class knows to refuse instantiation. @final sets __final__, and @override sets __override__. Template Method’s near_miss.py reads __final__ through getattr() to refuse an override at runtime. functools.wraps() copies the wrapped function’s __dict__ onto the wrapper, so a mark applied beneath a decorator written with @wraps survives that decorator.

A mark suits data that describes the function and that other code reads later, as handlers does here. State that changes with each call, such as a call count, belongs in a closure variable or on a callable object, where the type checker can see it.

Making a Class Final

Sometimes you need to forbid inheritance. The modern way to say so is the typing.final decorator:

# final.py
from typing import final

@final
class B:
    pass

# ty: Class `C` cannot inherit from final class `B`:
# class C(B): pass
b = B()
print(type(b).__name__)
#: B

Type checkers such as ty, mypy, and Pyright check @final statically. At runtime the decorator marks the class, setting __final__ = True (as test_final.py below confirms), so the interpreter still runs class C(B): pass.

Older literature claims that making the interpreter refuse subclassing requires a metaclass. __init_subclass__() does the job at each subclass creation:

# final_runtime.py
from exceptions import expected

class A:
    pass

class B(A):
    def __init_subclass__(cls, **kwargs: object) -> None:
        raise TypeError(
            f"{B.__name__} is final; "
            f"you cannot subclass it")

with expected(TypeError):
    class C(B):
        pass
#: [TypeError] B is final; you cannot subclass it

The check runs at class-creation time. Python builds B normally. A class’s own __init_subclass__() does not run for that class, and the version that does run at B’s creation is the one B inherits from A, which is object’s do-nothing default. Use the runtime version when @final is not enough, a rare case.

Testing confirms that the @final marker is present, the runtime-final class refuses subclassing, and its non-final base still allows it:

# test_final.py
import final
import final_runtime
import pytest

def test_final_decorator_marks_class() -> None:
    assert final.B.__final__ is True  # type: ignore

def test_runtime_final_cannot_be_subclassed() -> None:
    with pytest.raises(TypeError):
        class Sub(final_runtime.B):
            pass

def test_runtime_non_final_base_can_be_subclassed() -> None:
    class Ok(final_runtime.A):
        pass
    assert issubclass(Ok, final_runtime.A)

Where Enforcement Lives

Each device for constraining a class answers one question: who enforces a rule about a class, and when? The language devices you have met divide into four families.

The Four Families

@final and @override are markers. At runtime each sets a single attribute, __final__ or __override__, that the interpreter ignores and your own code can read, as near_miss.py does. The type checker carries the entire meaning, and Making a Class Final builds the runtime half by hand with __init_subclass__().

@dataclass is mirrored machinery. At runtime it is a code generator, synthesizing __init__() and its siblings at class-creation time. The type checker does not run the decorator. It recognizes the name and re-implements the generator’s rules statically: the synthesized signature, the frozen write-ban, the field-ordering rule. The typing specification mandates that model, so every conformant checker derives the same class, and some rules end up enforced twice, independently. If you declare a field without a default after one with a default, the checker reports that field before anything runs, while the interpreter raises its own TypeError at class creation.

A third family carries two real semantics. @abstractmethod makes the checker report an abstract instantiation, and separately makes the runtime refuse one. assert_never() proves exhaustiveness statically and raises an AssertionError at runtime when a lying value reaches it.

The fourth family runs in the other direction. Annotations survive into the running program, as The inspect Module shows, so a library can read the checker’s types and enforce them live.

`@dataclass_transform` Is a Claim

How does the checker know what @dataclass does? For the standard library, the checker has that knowledge built in. For a class-building decorator of your own, the checker knows only what you declare. @dataclass_transform marks a decorator as dataclass-like, and the declaration alone changes what the checker believes:

# claimed_transform.py
from typing import dataclass_transform
from exceptions import expect

@dataclass_transform()
def model[T](cls: type[T]) -> type[T]:
    return cls  # The claim, with nothing behind it

@model
class User:
    name: str
    age: int = 0

# The checker accepts this call:
expect(TypeError, User, "Guido", 30)
#: [TypeError] User() takes no arguments

The checker synthesizes a User.__init__() from the field declarations, with name required and age defaulted, as it does for @dataclass. It believes the declaration without running model(), so the call checks clean and fails at runtime. This model() generates nothing, and object’s constructor takes no arguments. The declaration is a claim, and this one is false.

Libraries like attrs and pydantic make the claim true by generating the methods their @dataclass_transform declares, and that is how their classes get first-class checking without any checker hard-coding them. The shortest model() that makes its claim true delegates the generation:

# kept_transform.py
from dataclasses import dataclass
from typing import dataclass_transform

@dataclass_transform(frozen_default=True)
def model[T](cls: type[T]) -> type[T]:
    return dataclass(frozen=True)(cls)

@model
class User:
    name: str
    age: int = 0

u = User("Guido", 30)
print(u)
#: User(name='Guido', age=30)
# ty: Property `age` defined in `User` is read-only:
# u.age = 9

Now both sides hold. The runtime User is a real frozen data class, and frozen_default=True tells the checker that classes built by model() reject assignment statically too.

@dataclass_transform generalizes exactly one shape: a decorator that builds a class’s methods from its field declarations. Anything stranger stays invisible to the checker, which neither imports nor executes your code. That is why commander.py in Generating Classes with exec(), whose class exists only as text in a string, uses a cast() to state the real signature. The checker emulates what it recognizes, believes what you declare, and sees nothing else.

Learning a Name with `__set_name__()`

A descriptor is any object whose class defines at least one of __get__(), __set__(), or __delete__(). Most descriptors define __get__() and add the others as needed. When a class attribute holds a descriptor, that descriptor takes over access to the attribute. Instead of going to the instance’s __dict__, a read calls __get__() and a write calls __set__(). Decorators depends on this, naming the descriptor without showing the protocol. A function is an object like any other, and its class defines __get__(), so every function is a descriptor:

# function_is_descriptor.py
from dataclasses import dataclass

@dataclass
class Person:
    name: str

    def greet(self) -> str:
        return f"Hello, {self.name}"

# def created a plain function in the class namespace:
plain = Person.__dict__["greet"]
print(type(plain).__name__, hasattr(plain, "__get__"))
#: function True

# Reading it through an instance triggers __get__(),
# which returns a bound method:
p = Person("Ann")
print(p.greet())
#: Hello, Ann

print(plain.__get__(p, Person)())
#: Hello, Ann

The last line performs by hand what p.greet() does automatically. Method binding is the descriptor protocol at work.

A Descriptor That Learns Its Name

A class attribute learning its own name is another job that once needed a metaclass. In x = Field() below, Field() runs before the assignment, so the new instance cannot know it is about to get the name x. Python delivers that name automatically. When a class body finishes executing, Python calls __set_name__(owner, name) on every class attribute that defines it, not only descriptors, passing the freshly created class and the name that holds the attribute. Field pairs __set_name__() with __get__() and __set__(), the descriptor protocol, and uses the delivered name to build its storage key. A print() at the top of each method traces the descriptor’s whole life, from naming at class creation through every read and write:

# set_name.py
from typing import Any

class Field:
    def __set_name__(self, owner: type, name: str) -> None:
        print(f"{name}.__set_name__ on {owner.__name__}")
        self.name = name
        self.storage = f"_{name}"

    def __get__(self, obj: Any,
                owner: type | None = None) -> Any:
        via = "class" if obj is None else "instance"
        print(f"{self.name}.__get__ via {via}")
        if obj is None:
            return self
        return getattr(obj, self.storage)

    def __set__(self, obj: Any, value: Any) -> None:
        print(f"{self.name}.__set__ = {value}")
        setattr(obj, self.storage, value)

class Point:
    x = Field()
    y = Field()
#: x.__set_name__ on Point
#: y.__set_name__ on Point

p = Point()
p.x = 3
#: x.__set__ = 3
p.y = 4
#: y.__set__ = 4
print(p.x, p.y)
#: x.__get__ via instance
#: y.__get__ via instance
#: 3 4
print(isinstance(Point.x, Field))
#: x.__get__ via class
#: True

The first two trace lines appear before any instance exists. Python calls __set_name__() as it finishes executing the class Point statement, once for each Field, handing each one the new class and its own attribute name. From then on, every read and write routes through the descriptor instead of going to the instance’s __dict__. p.x = 3 prints x.__set__ = 3 before storing anything. In print(p.x, p.y), Python evaluates both arguments before calling print(), so both __get__ lines appear ahead of 3 4. The final access, Point.x, goes through the class rather than an instance, so __get__() receives obj=None and reports via class. That branch returns self, the descriptor object, which is why isinstance(Point.x, Field) is True.

Each Field stores values under _x or _y in the instance’s __dict__. The underscore prefix does real work. A descriptor that defines __set__() is a data descriptor, and on every lookup a data descriptor outranks the instance’s __dict__. If __get__() asks obj for "x", the attribute’s own name, that lookup routes back to the descriptor and calls __get__() again, until Python raises a RecursionError. Point has no descriptor named _x, so storing the value there breaks the loop.

A descriptor that defines __get__() alone is a non-data descriptor, and the ranking reverses. The instance’s __dict__ wins. That is why assigning to greet on a Person instance shadows the method on that one instance, while p.x = 3 cannot shadow Field, because Field defines __set__().

__set_name__() is metaprogramming with no metaclass in sight.

Testing confirms the descriptor learns its name, stores each value under the storage key built from that name, and returns itself when you read it through the class:

# test_set_name.py
import set_name

def test_descriptor_learns_its_name() -> None:
    field = vars(set_name.Point)["x"]
    assert field.name == "x"
    assert field.storage == "_x"

def test_values_stored_under_storage_keys() -> None:
    p = set_name.Point()
    p.x = 3
    p.y = 4
    assert (p.x, p.y) == (3, 4)
    assert p.__dict__ == {"_x": 3, "_y": 4}

def test_descriptor_on_class_returns_itself() -> None:
    assert isinstance(set_name.Point.x, set_name.Field)
    assert set_name.Point.x is vars(set_name.Point)["x"]

A Descriptor That Validates

Field shows the protocol and stores whatever you hand it. A descriptor pays for itself when the write must do more than store. Positive checks the value on its way in, so every attribute declared with one enforces the same rule:

# validating_descriptor.py
from exceptions import expected

class Positive:
    def __set_name__(self, owner: type, name: str) -> None:
        self.storage = f"_{name}"

    def __get__(self, obj: object,
                owner: type | None = None) -> float:
        return getattr(obj, self.storage)

    def __set__(self, obj: object, value: float) -> None:
        if value <= 0:
            raise ValueError(f"{value} is not positive")
        setattr(obj, self.storage, value)

class Rectangle:
    width = Positive()
    height = Positive()

    def __init__(
        self, width: float, height: float
    ) -> None:
        self.width = width
        self.height = height

    def area(self) -> float:
        return self.width * self.height

r = Rectangle(3.0, 4.0)
print(r.area())
#: 12.0

with expected(ValueError):
    r.width = -1.0
#: [ValueError] -1.0 is not positive

print(r.area())
#: 12.0

Rectangle names the rule twice and writes no checking code of its own. self.width = width inside __init__() routes through Positive.__set__() like any other write, so the constructor validates its arguments without a line devoted to it. The rejected assignment leaves _width as it was, so r.area() still uses the width the constructor stored. A property protects one attribute the same way, but Rectangle then carries the check twice, once per attribute. With a descriptor, the rule lives in one class, and each attribute that needs it says Positive().

Positive.__get__() omits the obj is None branch that Field has, so it works through an instance and fails through the class. Reading Rectangle.width passes None as obj and raises an AttributeError. A descriptor meant for wider use returns self there, as Field does.

Writing a Metaclass

A metaclass is a subclass of type, and you write one when the simpler hooks are not enough. You attach it with the metaclass= keyword in the class header. Python then uses your metaclass, instead of type, to build the class.

# simple_meta.py
from typing import Any
from display import display_object

class SimpleMeta(type):
    def __init__(cls, name: str, bases: tuple[type, ...],
                 nmspc: dict[str, Any]) -> None:
        super().__init__(name, bases, nmspc)
        setattr(cls, "uses_metaclass", lambda self: "Yes!")

class Simple(metaclass=SimpleMeta):
    def ping(self) -> None: pass

    @staticmethod
    def pong() -> None: pass

display_object(Simple)
#: [Attributes]
#:   None
#: [Methods]
#:   • ping(self) -> None
#:   • pong() -> None
#:   • uses_metaclass(self)
print(Simple().uses_metaclass())  # type: ignore
#: Yes!

SimpleMeta.__init__() runs once, as the class Simple statement finishes, and patches a new method onto the freshly built class. In the display_object() output, uses_metaclass(self) sits alongside ping() and pong(), indistinguishable from the methods in the class body. The injected value is a lambda, but a function is a descriptor, so Simple().uses_metaclass() binds it like any other method.

Since a metaclass is a subclass of type, writing class Simple(SimpleMeta): means something else. That syntax makes SimpleMeta an ordinary base class, so Simple inherits type and becomes a second metaclass, not a class built by SimpleMeta. metaclass= is the mechanism for naming what builds a class, independent of its base classes. Python computes a new class’s metaclass from all of its bases, so a subclass of Simple inherits SimpleMeta without repeating metaclass=.

By convention the first argument of a metaclass method is cls rather than self, except for __new__(), whose first argument is the metaclass and usually takes the name mcls or mcs. Here cls is the class object under construction, Simple. As with any subclass, call the base-class version first through super().

Metaprogramming and static typing pull against each other. A type describes a known set of attributes and signatures, but a metaclass changes that structure at runtime, adding attributes the class does not declare and replacing methods like __new__(). The type checker cannot follow those changes, so it reports the dynamic lines as errors. Three ways quiet it, from narrowest to broadest: setattr(cls, "name", value) adds an attribute through a string the type checker does not track; a localized # type: ignore silences one line, as on Simple().uses_metaclass() above; and copying the class into an Any-typed name, klass: Any = cls, stops attribute checking for everything reached through that name. Prefer the narrowest escape that fits, because a broad Any also hides genuine mistakes.

`init()` versus `new()` in a Metaclass

Metaclass examples appear to use __new__() and __init__() interchangeably. The difference is timing. __new__() runs before the class object exists, so it can change the name, bases, and namespace that Python uses to build it. __init__() runs after the class exists, so changing those arguments has no effect, though you can still modify the finished class object:

# new_vs_init.py
from typing import Any
from display import display_object

class Tag:
    pass

class Meta(type):
    def __new__(mcls, name: str, bases: tuple[type, ...],
                nmspc: dict[str, Any]) -> type:
        # Before creation: these changes take effect
        nmspc["added_in_new"] = 42
        bases += (Tag,)
        return super().__new__(mcls, name, bases, nmspc)

    def __init__(cls, name: str, bases: tuple[type, ...],
                 nmspc: dict[str, Any]) -> None:
        super().__init__(name, bases, nmspc)
        # No effect: the class is already built
        nmspc["added_in_init"] = 99
        # Effect: this modifies the finished class
        setattr(cls, "patched_in_init", 3.14)

class Built(metaclass=Meta):
    pass

display_object(Built(), dunder=["__new__", "__init__"])
#: [Attributes]
#:   • added_in_new = 42 [CV]
#:   • patched_in_init = 3.14 [CV]
#: [Methods]
#:   • __init__(self, /, *args, **kwargs)
#:   • __new__(*args, **kwargs)

print("has Tag base:", Tag in Built.__bases__)
#: has Tag base: True

added_in_init is missing from the output because type.__new__() copies nmspc into the new class’s own __dict__ as it builds the class. By the time __init__() runs, the two mappings are independent, so mutating the original dict changes nothing the class can see. setattr(cls, ...) still works because it modifies the class object.

Override __new__() when you must change name, bases, or the namespace (including special members like __slots__) before Python builds the class. Otherwise, prefer __init__(), which is simpler, and reserve __new__() for a genuine need.

Intercepting Instance Creation

A method defined on the metaclass becomes a method of the class object, callable on the class but not on its instances. Such methods are sometimes called metamethods. They differ from classmethods. A classmethod stays callable on both the class and its instances, while a metamethod works only through the class. The class is an instance of the metaclass. The class’s own instances are not.

One useful metamethod is __call__(). It is the same method that makes any object callable. obj() invokes type(obj).__call__(obj, ...). A class is an object, an instance of its metaclass, so ClassName() invokes __call__() on the metaclass the same way. That __call__() runs first when you create an instance of the class. __new__() and __init__() normally run only because the default type.__call__() calls them. A metaclass that overrides __call__() sits above that step and decides whether to call them, so it can skip building a new instance and return a cached one. Caching there is one way to build a Singleton:

# singleton.py
from typing import Any, ClassVar

class Singleton(type):
    # A shared dict of class objects : instances
    _instances: ClassVar[dict[type, Any]] = {}

    def __call__[T](
            cls: type[T], *args: Any, **kwargs: Any) -> T:
        if cls not in Singleton._instances:
            print(f"building {cls.__name__}")
            Singleton._instances[cls] = type.__call__(
                cls, *args, **kwargs)
        else:
            print(f"reusing {cls.__name__}")
        return Singleton._instances[cls]

class ASingleton(metaclass=Singleton):
    pass

class BSingleton(metaclass=Singleton):
    pass

a = ASingleton()
#: building ASingleton
b = ASingleton()  # [1]
#: reusing ASingleton
assert a is b

c = BSingleton()
#: building BSingleton
d = BSingleton()
#: reusing BSingleton
assert c is d
assert a is not c

The ASingleton() call at [1] does not reach __new__() or __init__(). __call__() finds the cached instance and returns it without building anything. Each class gets its own entry in the _instances dictionary, so the singletons are independent.

The [T] on __call__() ties its return type to cls, so ty sees ASingleton() as an ASingleton instead of Any. Without the [T], every singleton comes back as Any under ty and Pyright, and a misspelled attribute access on the result passes the check. Under mypy the [T] changes nothing, because mypy ignores a metaclass __call__() return type and keeps ASingleton either way.

That same [T] is why the body calls type.__call__(cls, ...) instead of the more usual super().__call__(...). Annotating the first parameter as type[T] hides that cls is a Singleton, and ty and mypy must confirm that before they accept a zero-argument super(). Pyright accepts it without the check. Both forms do the same work at run time.

You might expect to parameterize¹ the class, with class Singleton[T](type) and _instances: ClassVar[dict[type, T]]. That fails twice. A ClassVar means one shared value for the whole class, so it cannot depend on a type parameter that varies per instantiation. And each class must write class ASingleton(metaclass=Singleton[ASingleton]):, naming ASingleton before its class body finishes defining it.² The method-level [T] on __call__() avoids both problems. It binds T from cls at the call site, ASingleton(), and that call runs after ASingleton exists.

The metaclass version works, but it is heavier than the problem usually requires. Singleton covers the lighter alternatives, from a class decorator down to a module. Choose the lightest tool that solves your problem.

Multiple Inheritance and Metaclasses

Singleton stores its cache in _instances, a dict attribute, rather than inheriting from dict. Can a metaclass inherit from more than one class, the way an ordinary class can?

Trying the obvious version fails. type and dict are both built-in types with their own C-level instance layout, and CPython allows multiple inheritance only when the bases’ layouts are compatible, one extending the other:

# metaclass_layout_conflict.py
from typing import Any
from exceptions import expected

with expected(TypeError):
    class Singleton(type, dict[type, Any]):  # type: ignore
        pass
#: [TypeError] multiple bases have instance lay-out conflict

The failure has nothing to do with metaclasses. class X(list, dict): pass fails the same way, and neither base is a metaclass. type, list, and dict each bring a layout of their own, so no class can combine two of them.

The # type: ignore comment appears because ty knows this rule statically. At check time, its instance-layout-conflict check reports the very TypeError this example exists to demonstrate at run time. A type checker that predicts a crash before the program runs is static typing at its best. The comment suppresses the diagnostic only because provoking that crash is educational.

A metaclass can multiply inherit like any other class, as long as the extra class is a mixin with no competing layout:

# mixin.py
from exceptions import expected

class Mixin:
    def helper(self) -> str:
        return "hi"

class Base(type, Mixin):
    pass

class Sub(metaclass=Base):
    pass

print(Sub.helper())
#: hi

with expected(AttributeError):  # A metamethod: class only
    Sub().helper()  # type: ignore
#: [AttributeError] 'Sub' object has no attribute 'helper'

helper() arrives through the metaclass, so Sub has it and a Sub instance does not. The AttributeError comes from the metamethod rule at the start of Intercepting Instance Creation. An instance of Sub is not an instance of Base, so nothing in its lookup chain reaches Mixin. A classmethod answers on both.

The constraint here is the ordinary layout-compatibility rule that governs every Python class, not something specific to metaclasses. Composing a dict, the way Singleton._instances does, sidesteps the conflict.

Multiple inheritance fails a second way, from the other direction. A class has a single metaclass, so inheriting from two classes built by different metaclasses has no answer:

# multiple_metaclass_inheritance.py
from exceptions import expected

class MetaA(type):
    pass

class MetaB(type):
    pass

class A(metaclass=MetaA):
    pass

class B(metaclass=MetaB):
    pass

with expected(TypeError):
    class C(A, B):  # type: ignore
        pass
#: [TypeError] metaclass conflict: the metaclass of a
#: derived class must be a (non-strict) subclass of the
#: metaclasses of all its bases

class MetaC(MetaA, MetaB):
    pass

class D(A, B, metaclass=MetaC):
    pass

print(type(D).__name__)
#: MetaC

As with the layout conflict just shown, ty reports conflicting-metaclass and names both MetaA and MetaB, so the line carries a # type: ignore. The expected() helper wraps the message across three lines to fit the page. Python reports it as a single line. The message names the fix. D’s metaclass, MetaC, must be a subclass of every base’s metaclass, MetaA and MetaB both. Once MetaC exists, class D(A, B, metaclass=MetaC) builds cleanly.

Both failures have the same shape: an inheritance graph that looks legal until you notice what the bases carry with them. That shape is one more reason to avoid metaclasses (and, arguably, multiple inheritance) unless you truly need them.

When You Still Need a Metaclass

Use a metaclass when you need to change the class object rather than react to its creation:

Adding methods to the class (metamethods such as the __call__() in singleton.py, or the __iter__() that lets EnumType make for c in Color work).
Replacing the namespace mapping with __prepare__() so the class body populates a custom dictionary.
Enforcing an invariant across a family of classes that shares no base class. __init_subclass__() needs a common base to live on, and the shared metaclass is what such a family has instead.

A metaclass can give the class object an __iter__(), the same method that lets EnumType make for c in Color work:

# iterable_class.py
from collections.abc import Iterator
from typing import Any

class IterableMeta(type):
    def __iter__(cls) -> Iterator[Any]:
        return (
            v for k, v in vars(cls).items()
            if not k.startswith("_")
        )

class Color(metaclass=IterableMeta):
    red = "red"
    green = "green"
    blue = "blue"

for c in Color:
    print(c)
#: red
#: green
#: blue

IterableMeta.__iter__() fires when you write for c in Color, iterating over the class object, not an instance of it. It walks vars(cls), the class’s own namespace, and skips every underscore-prefixed name. For Color those names are the dunder bookkeeping every class carries, so __iter__() yields the three values the body assigned: "red", "green", "blue".

A class decorator cannot make for c in Color work. It can add methods that instances see, but not a protocol method the class object must answer, which is why Color needs a metaclass, not a decorator. Python looks up a special method such as __iter__() on the object’s type and skips the object’s own namespace, so for c in Color consults type(Color). An __iter__ that a decorator assigns onto Color serves instances of Color and leaves the class without one.

__prepare__() is the one with no simpler substitute:

# prepare_namespace.py
from typing import Any
from exceptions import expected

class NoDuplicates(dict[str, Any]):
    def __setitem__(self, key: str, value: Any) -> None:
        if key in self:
            raise TypeError(f"{key} defined twice")
        super().__setitem__(key, value)

class Strict(type):
    @classmethod
    def __prepare__(cls, name: str, bases: tuple[type, ...],
                    **kwargs: Any) -> NoDuplicates:
        return NoDuplicates()

with expected(TypeError):
    class Handlers(metaclass=Strict):
        def on_open(self) -> None: ...  # [1]
        def on_close(self) -> None: ...
        def on_open(self) -> None: ...  # noqa: F811  # [2]
#: [TypeError] on_open defined twice

__prepare__() runs before the class body does, and whatever mapping it returns becomes the namespace for that body. Every def and every assignment in the body becomes a __setitem__() call on that mapping, so NoDuplicates sees the on_open at [2] assigned to a name it already holds. Python then hands the finished mapping to type.__new__().

__prepare__() must carry @classmethod. Python calls it on the metaclass before any class object exists, so an ordinary method receives the class name as its self and leaves bases unfilled, producing a TypeError that says nothing about the real mistake.

No other hook can catch the on_open at [2]. __init_subclass__(), __set_name__(), and a class decorator all run after the body has finished, by which time [2] has overwritten [1]. The static half of the check is ruff’s report of the same mistake, and the # noqa: F811 suppresses it so the listing can run. __prepare__() catches the mistake at run time, including on names the body computes.

These needs are real but uncommon. For everything else, __init_subclass__(), __set_name__(), and class decorators are simpler and easier to read. A class decorator receives the finished class, so it can add, replace, or inspect members. It cannot change the name, the bases, or the namespace, and it cannot give the class object behavior of its own. Setting __call__ from a decorator makes instances callable. Only a metaclass makes the class callable in a new way. The case for a metaclass rests on that limit. The class object needs behavior, and nothing that runs after the class exists can give it any.

The `inspect` Module

Up to now, you’ve modified classes. type() builds them, and metaclasses and __init_subclass__() run code during their creation. The inspect module is the other half of metaprogramming. It reads the structure of live objects. It answers questions like which members an object has, what a function’s signature is, and what its docstring says.

The Core Functions

inspect works on any live object: modules, classes, functions, methods, and instances.

inspect.signature(callable) returns a Signature object describing the parameters, their annotations, and their defaults.
inspect.getdoc(obj) returns the cleaned-up docstring.
inspect.getmembers(obj) and inspect.getmembers_static(obj) return an object’s (name, value) pairs. The _static variant reads them without running properties or other descriptors.
Predicates such as inspect.isclass(), inspect.isfunction(), and inspect.ismethod() classify what you find.

# inspect_tour.py
import inspect

def greet(name: str, loud: bool = False) -> str:
    "Return a greeting."
    text = f"Hello, {name}"
    return text.upper() if loud else text

print(inspect.signature(greet))
#: (name: str, loud: bool = False) -> str
print(inspect.getdoc(greet))
#: Return a greeting.
print(inspect.isfunction(greet), inspect.isclass(greet))
#: True False
print(list(inspect.signature(greet).parameters))
#: ['name', 'loud']

signature() recovers the full call interface, annotations and defaults included, as a structured object rather than a string. Python keeps type annotations (a.k.a. type hints) at runtime, attached to the function and evaluated on demand, the deferred evaluation of PEP 649, although it does not check them. signature() reads that stored data (not the original source text) to build the Signature object. The ALL_DUNDERS listing in The Tool in Use shows that machinery on a class. __annotate_func__ is the code that computes the annotations, and __annotations_cache__ holds the result after the first request. __annotate__, the name PEP 649 gives the function, is a descriptor on type that reads __annotate_func__, so it lives on the metaclass rather than in the class’s own namespace.

display_object() relies on inspect functions. getmembers_static() finds the members, signature() renders each method, and get_annotations(), which returns a class’s declared annotations in a dict, supplies the declared types. Its source and its display options are reference material, collected in display_object() Reference at the end of this chapter.

Sorting Members into Attributes and Methods

display_object() walks every member that inspect.getmembers_static() returns. The static variant reads members from the __dict__ of the object and of each of its classes, without invoking descriptors, properties, or __getattr__(). Inspecting an object therefore runs none of its code and triggers no side effect, and that safety matters when you point this tool at something unfamiliar.

The tool sorts each member into one of two lists. Callables become methods, printed with the signature that inspect.signature() reports, or (...) when a built-in has no inspectable signature. Everything else becomes an attribute, printed as name: type = value. The declared type comes from the class annotations, gathered across the whole inheritance chain with inspect.get_annotations() in FORWARDREF format, so a name the module never defined prints as a ForwardRef instead of raising a NameError (Reading Annotations Before Their Names Exist explains the formats). An attribute with no annotation, such as one assigned dynamically, prints as name = value. The value is the member’s repr(), truncated to keep the line within max_width.

An attribute tagged [CV], for class variable, lives on the class or a base class rather than in obj’s own __dict__. When obj is a class, every attribute lives on a class, so all of them carry the tag. In Comparing Ordinary Classes and Data Classes, classvar_dataclass.py’s show(D) tags both D.x and D.s, although D declares them in its own body, because neither belongs to an instance. For an instance, the tag says whether the value lives on the class or on the object, the same rule Stars.rating demonstrates in Class Attributes. class_with_defaults.py’s show(B()), from that same chapter 12 comparison, tags B.x and B.s, while display_object(Messenger("iris", 12, 3.14)) tags none, since @dataclass assigns every field straight onto the new instance. The tag comes from where the value lives, so it applies whether or not the attribute’s declaration uses typing.ClassVar.

Reading Annotations Before Their Names Exist

Deferred evaluation lets an annotation name a class that does not exist yet. Nothing evaluates the annotation until something asks for it, so a class can declare a field whose type appears further down the module. A metaprogram asks early. A decorator, an __init_subclass__() hook, or a metaclass runs while Python creates the class, and if it reads the annotations then, a name they mention may still be undefined. Evaluating them at that moment raises a NameError.

The standard library’s annotationlib module handles this case. Its get_annotations() takes a format argument that chooses how to evaluate the annotations, and inspect.get_annotations() accepts the same argument.

Format.VALUE, the default, evaluates each annotation to a real object, and raises a NameError when a name is missing.
Format.FORWARDREF evaluates what it can, and substitutes an annotationlib.ForwardRef for each name it cannot resolve.
Format.STRING evaluates nothing, and returns each annotation as a string that matches the source as closely as it can.

The listing reads one forward-referencing annotation in all three formats:

# forward_formats.py
from annotationlib import Format, get_annotations
from exceptions import expect

class Order:
    item: str
    buyer: Customer

expect(NameError, get_annotations, Order,
       format=Format.VALUE)
#: [NameError] name 'Customer' is not defined
fwd = get_annotations(Order, format=Format.FORWARDREF)
print(fwd["item"])
#: <class 'str'>
ref = fwd["buyer"]
print(type(ref).__name__, ref.__forward_arg__)
#: ForwardRef Customer
print(get_annotations(Order, format=Format.STRING))
#: {'item': 'str', 'buyer': 'Customer'}

class Customer:
    pass

print(get_annotations(Order)["buyer"] is Customer)
#: True

Order names Customer before Customer exists. The VALUE request fails the same way it would inside a decorator applied to Order. FORWARDREF resolves item to the str class and puts a ForwardRef in place of buyer. The reference’s __forward_arg__ holds the missing name. STRING evaluates nothing, so it cannot fail, but it gives up the objects. You get the text 'str' rather than the class str. Once Customer exists, the default VALUE request succeeds, because the failed attempt cached nothing.

A tool that reads annotations at definition time therefore asks for FORWARDREF. It keeps the real objects it gets back, and holds on to each ForwardRef until the name exists, when the reference’s evaluate() method resolves it to the class.

Which Hook for Which Job

Every hook in this chapter is an ordinary function that Python calls at a known moment during class construction.

# hook_order.py
from typing import Any

class Watched:
    def __set_name__(self, owner: type, name: str) -> None:
        print(f"__set_name__({owner.__name__}, {name})")

class Meta(type):
    @classmethod
    def __prepare__(cls, name: str, bases: tuple[type, ...],
                    **kwargs: Any) -> dict[str, Any]:
        print(f"__prepare__ {name}")
        return {}

    def __new__(mcls, name: str, bases: tuple[type, ...],
                nmspc: dict[str, Any]) -> type:
        print(f"__new__ {name} enter")
        cls = super().__new__(mcls, name, bases, nmspc)
        print(f"__new__ {name} exit")
        return cls

    def __init__(cls, name: str, bases: tuple[type, ...],
                 nmspc: dict[str, Any]) -> None:
        super().__init__(name, bases, nmspc)
        print(f"__init__ {name}")

def tag[T](cls: type[T]) -> type[T]:
    print(f"decorator {cls.__name__}")
    return cls

class Base(metaclass=Meta):
    def __init_subclass__(cls, **kwargs: object) -> None:
        super().__init_subclass__(**kwargs)
        print(f"__init_subclass__ {cls.__name__}")
#: __prepare__ Base
#: __new__ Base enter
#: __new__ Base exit
#: __init__ Base

@tag
class Derived(Base):
    field = Watched()
    print("class body")
#: __prepare__ Derived
#: class body
#: __new__ Derived enter
#: __set_name__(Derived, field)
#: __init_subclass__ Derived
#: __new__ Derived exit
#: __init__ Derived
#: decorator Derived

Base’s four lines are the bare sequence, and they also show that Base.__init_subclass__() does not run for Base, the rule Making a Class Final needs. Derived adds the rest. __prepare__() runs before the body, so its line comes first. The body then executes, printing class body. __set_name__() and __init_subclass__() both run between __new__ Derived enter and __new__ Derived exit, because type.__new__() calls them as it assembles the class, so they are not merely “after the body” but inside the metaclass’s own construction step. The decorator is last, because it receives a finished class.

React to each new subclass: __init_subclass__().
Let a class attribute learn its own name: __set_name__().
Rewrite a finished class: a class decorator.
Build a family of classes from data: type() with three arguments, or an exec()ed class body when the definition is easier to read as source.
Change the name, the bases, or the namespace before Python builds the class: a metaclass __new__().
Control the namespace into which the body executes: __prepare__().
Decide whether an instance gets built: a metaclass __call__().
Read a class you did not write: inspect.

None of this is a special facility bolted onto the language. A class is an object that Python builds at run time by executing its body, and hook_order.py displays each step of that construction as it runs. The one hook missing from its trace is __call__(), which runs later still, each time someone calls the finished class.

`display_object()` Reference

The rest of display_object() is presentation: how it formats what inspect reports, and which members it shows. Read it when a listing’s output raises a question, and skip it otherwise.

Building `display_object()`

Throughout the book you’ve seen display_object() show the layout of an object. The utils/ prefix on the file marker below puts it in the shared utils/ directory at the top of the Examples tree, where any chapter can import it:

# utils/display.py
import inspect
from annotationlib import Format
from collections.abc import Callable, Sequence
from typing import Final

ALL_DUNDERS = sentinel("ALL_DUNDERS")
REDEFINED_DUNDERS = sentinel("REDEFINED_DUNDERS")
INTERESTING_DUNDERS: Final[tuple[str, ...]] = (
    "__init__", "__repr__", "__eq__", "__hash__",
)

def _annotations(cls: type) -> dict[str, object]:
    # Annotations declared on the class or any of its bases:
    return {**inspect.get_annotations(
                base, format=Format.FORWARDREF)
            for base in reversed(cls.__mro__)}

def _type_name(annotation: object) -> str:
    # A readable annotation name, keeping [parameters]:
    if isinstance(annotation, type):
        return annotation.__name__
    return str(annotation)

def _redefined(name: str, value: object) -> bool:
    # Restricted to INTERESTING_DUNDERS: every class has
    # __module__, __dict__, and other bookkeeping dunders
    # that always differ from object's, so comparing those
    # never filters anything out.
    if name not in INTERESTING_DUNDERS:
        return False
    return getattr(object, name, None) is not value

def _show_dunder(
    dunder: Sequence[str] | ALL_DUNDERS | REDEFINED_DUNDERS,
    name: str,
    value: object,
) -> bool:
    if dunder is ALL_DUNDERS:
        return True
    if dunder is REDEFINED_DUNDERS:
        return _redefined(name, value)
    return name in dunder

def _shared(obj: object, name: str) -> bool:
    # A class has no instance-level storage to compare
    # against, so every attribute it shows is class-level
    # storage by construction. For an instance, only a name
    # missing from its own __dict__ is:
    if inspect.isclass(obj):
        return True
    return name not in getattr(obj, "__dict__", {})

def _truncate(text: str, budget: int) -> str:
    # Fit the budget, marking a cut with an ellipsis:
    if len(text) <= budget:
        return text
    if budget < 4:  # No room for text plus the ellipsis
        return "..."[:max(budget, 0)]
    return text[:budget - 3] + "..."

def _format_method(
    name: str, value: Callable[..., object], max_width: int
) -> str:
    try:
        sig = str(inspect.signature(value))
    except (ValueError, TypeError):
        sig = "(...)"
    sig = _truncate(sig, max_width - len(name) - 4)
    return f"  • {name}{sig}"

def _format_attribute(
    obj: object,
    name: str,
    value: object,
    annotations: dict[str, object],
    max_width: int,
) -> str:
    label = name
    if name in annotations:
        label = f"{name}: {_type_name(annotations[name])}"
    tag = " [CV]" if _shared(obj, name) else ""
    budget = max_width - len(label) - len(tag) - 7
    val_str = _truncate(repr(value), budget)
    return f"  • {label} = {val_str}{tag}"

def display_object(
    obj: object,
    dunder: (Sequence[str] | ALL_DUNDERS
             | REDEFINED_DUNDERS) = (),
    max_width: int = 57,
    exclude: Sequence[str] = (),
) -> None:
    # For a class, the class; for an instance, its class:
    cls = obj if inspect.isclass(obj) else type(obj)
    annotations = _annotations(cls)
    attributes: list[str] = []
    methods: list[str] = []
    # Read members statically, without triggering
    # dynamic descriptors:
    for name, value in inspect.getmembers_static(obj):
        if name in exclude:
            continue
        is_dunder = (name.startswith("__")
                     and name.endswith("__"))
        if is_dunder and not _show_dunder(
                dunder, name, value):
            continue  # Skip standard dunder clutter
        if callable(value):
            methods.append(
                _format_method(name, value, max_width))
        else:
            attributes.append(_format_attribute(
                obj, name, value, annotations, max_width
            ))
    print("[Attributes]")
    print("\n".join(attributes) or "  None")
    print("[Methods]")
    print("\n".join(methods) or "  None")

Importing into any example works because the example tooling puts utils/ on the import path, not because Python searches other directories automatically. tools/run_examples.py sets PYTHONPATH to the tree’s utils/ directory before running each script. The same directory reaches pytest through pythonpath in pyproject.toml. Without either, from display import display_object fails with ModuleNotFoundError.

Choosing Which Dunders to Show

display_object() hides standard dunder members by default. Pass their names in dunder to keep specific ones, as new_vs_init.py does to show __new__ and __init__. Pass the ALL_DUNDERS sentinel instead to keep every dunder member, including the interpreter’s own machinery. dunder’s type is Sequence[str] | ALL_DUNDERS | REDEFINED_DUNDERS, naming each sentinel value rather than the generic sentinel class, so a type checker narrows dunder to Sequence[str] once it rules out both sentinels, and name in dunder needs no further guard. ALL_DUNDERS is useful for exploring an unfamiliar object, but it buries a class’s own choices under everything object and the interpreter add. INTERESTING_DUNDERS names the four a reader typically customizes when defining a class: __init__, __repr__, __eq__, and __hash__. Pass it as dunder to see those four without the surrounding noise.

A class that overrides none of the four still shows all four, because it inherits object’s versions, and the report cannot tell those from ones the class wrote. REDEFINED_DUNDERS filters harder. Among those same four, it keeps only the ones whose value differs from object’s own, so a class that overrides none of them shows no dunders. _redefined() checks membership in INTERESTING_DUNDERS before comparing, deliberately narrowing the comparison to those four.

Every class, even an empty one, has its own __module__, __dict__, and a handful of other bookkeeping dunders that always differ from object’s, so comparing every dunder this way shows that bookkeeping instead of filtering it out. The comparison uses is, not ==, since a dunder inherited unchanged from object is the same function object, not merely an equal one.

INTERESTING_DUNDERS and REDEFINED_DUNDERS side by side, on a class that redefines nothing and one that redefines almost everything:

# dunder_modes.py
from dataclasses import dataclass
from display import (INTERESTING_DUNDERS, REDEFINED_DUNDERS,
                     display_object)

class Plain:
    pass

@dataclass
class Point:
    x: int
    y: int

display_object(Plain, INTERESTING_DUNDERS)
#: [Attributes]
#:   None
#: [Methods]
#:   • __eq__(self, value, /)
#:   • __hash__(self, /)
#:   • __init__(self, /, *args, **kwargs)
#:   • __repr__(self, /)

display_object(Plain, REDEFINED_DUNDERS)
#: [Attributes]
#:   None
#: [Methods]
#:   None

display_object(Point, REDEFINED_DUNDERS)
#: [Attributes]
#:   • __hash__ = None [CV]
#: [Methods]
#:   • __eq__(self, other)
#:   • __init__(self, x: int, y: int) -> None
#:   • __repr__(self)

display_object(Point, REDEFINED_DUNDERS,
               exclude=("__hash__",))
#: [Attributes]
#:   None
#: [Methods]
#:   • __eq__(self, other)
#:   • __init__(self, x: int, y: int) -> None
#:   • __repr__(self)

Plain writes none of the four, so INTERESTING_DUNDERS shows object’s versions and REDEFINED_DUNDERS shows nothing. @dataclass writes three of them and sets __hash__ to None, so Point reports a __hash__ attribute rather than a method. The last call drops that row, for the same reason comparison.py in Data Classes as Types passes exclude=("__hash__",).

exclude drops specific names regardless of what dunder otherwise shows, and it applies to any member, not just dunders. display_object(obj, REDEFINED_DUNDERS, exclude=("__hash__",)) shows whatever REDEFINED_DUNDERS finds redefined, minus __hash__, useful when an earlier listing has made that point and repeating it adds noise. The check runs first, before the dunder logic sees the name, so an excluded name stays out of [Attributes] and [Methods] no matter which mode selects it.

The Tool in Use

# demo_display_object.py
from dataclasses import dataclass
from display import ALL_DUNDERS, display_object

@dataclass
class Fraggle:
    """A small dataclass for the demo."""
    x: int
    y: float = 1.14659
    z: str = "blivet"

    def f(self) -> None: ...
    def g(self, x: int) -> float:
        return 0.001
    def h(self, s: str) -> str:
        return f"h({s})"

display_object(Fraggle)  # [1]
#: [Attributes]
#:   • y: float = 1.14659 [CV]
#:   • z: str = 'blivet' [CV]
#: [Methods]
#:   • f(self) -> None
#:   • g(self, x: int) -> float
#:   • h(self, s: str) -> str

display_object(Fraggle(9, 2.3))  # [2]
#: [Attributes]
#:   • x: int = 9
#:   • y: float = 2.3
#:   • z: str = 'blivet'
#: [Methods]
#:   • f(self) -> None
#:   • g(self, x: int) -> float
#:   • h(self, s: str) -> str

# ALL_DUNDERS also reveals what @dataclass generated:
display_object(Fraggle(9, 2.3), dunder=ALL_DUNDERS)  # [3]
#: [Attributes]
#:   • __annotations_cache__ = {'x': <class 'int'>, ... [CV]
#:   • __class__ = <attribute '__class__'> [CV]
#:   • __dataclass_fields__ = {'x': Field(name='x',t... [CV]
#:   • __dataclass_params__ = _DataclassParams(init=... [CV]
#:   • __dict__ = <attribute '__dict__'> [CV]
#:   • __doc__ = 'A small dataclass for the demo.' [CV]
#:   • __firstlineno__ = 5 [CV]
#:   • __hash__ = None [CV]
#:   • __match_args__ = ('x', 'y', 'z') [CV]
#:   • __module__ = '__main__' [CV]
#:   • __static_attributes__ = () [CV]
#:   • __weakref__ = <attribute '__weakref__'> [CV]
#:   • x: int = 9
#:   • y: float = 2.3
#:   • z: str = 'blivet'
#: [Methods]
#:   • __annotate_func__(format, /)
#:   • __delattr__(self, name, /)
#:   • __dir__(self, /)
#:   • __eq__(self, other)
#:   • __format__(self, format_spec, /)
#:   • __ge__(self, value, /)
#:   • __getattribute__(self, name, /)
#:   • __getstate__(self, /)
#:   • __gt__(self, value, /)
#:   • __init__(self, x: int, y: float = 1.14659, z: str ...
#:   • __init_subclass__(type, /)
#:   • __le__(self, value, /)
#:   • __lt__(self, value, /)
#:   • __ne__(self, value, /)
#:   • __new__(*args, **kwargs)
#:   • __reduce__(self, /)
#:   • __reduce_ex__(self, protocol, /)
#:   • __replace__(self, /, **changes)
#:   • __repr__(self)
#:   • __setattr__(self, name, value, /)
#:   • __sizeof__(self, /)
#:   • __str__(self, /)
#:   • __subclasshook__(type, object, /)
#:   • f(self) -> None
#:   • g(self, x: int) -> float
#:   • h(self, s: str) -> str

The calls at [1] and [2] show the same class from two angles. display_object(Fraggle) inspects the class object. It lists y and z, the fields with defaults. x’s declaration is x: int with no default, so on the class it is an annotation with no bound attribute, and getmembers_static() skips it.

display_object(Fraggle(9, 2.3)) inspects an instance, whose attributes hold its field values, so x now appears beside y and z. The method list is the same either way, because methods live on the class.

The call at [3] passes ALL_DUNDERS. A @dataclass produces many of these:

__dataclass_fields__
__dataclass_params__
__match_args__
__replace__
__hash__, set to None
__init__, __eq__, and __repr__

The generated __init__(), __eq__(), and __repr__() give Fraggle a constructor, equality, and a repr() that you did not write.

The rest is the bookkeeping every class carries.

Exercises

This chapter’s solutions give a hint, usually the shape of the code, and a full answer for each exercise.

In init_subclass.py, add a class Yellow(Color) and then Gold(Yellow). Predict Color.registry after each new class, then confirm.
In set_name.py, add a third Field() attribute, z, to Point, set p.z = 9, and confirm p.__dict__ now also holds _z.
In singleton.py, add a third class CSingleton(metaclass=Singleton) and confirm c1 = CSingleton(); c2 = CSingleton(); c1 is c2 is True, while c1 is a (comparing across the different singleton classes) is False.
Extend final_runtime.py so a class declares itself final with a keyword in its header, class B(A, final=True):, using the **kwargs that __init_subclass__() receives. Confirm that a non-final sibling of B still subclasses freely.
Using inspect_tour.py as a model, write a function describe(func) that prints a function’s name, its inspect.signature(), and its docstring (or "(no docstring)" if inspect.getdoc() returns None), then call it on greet and on a lambda.
Delete the # type: ignore comment from metaclass_layout_conflict.py and run ty over the file. Compare the instance-layout-conflict diagnostic it reports with the TypeError the program prints. The static report and the runtime failure describe the same collision.
Using type() directly, build a class Celsius with a base of float, an attribute unit = "C", and a method describe(self) returning f"{self} degrees {self.unit}". Confirm Celsius(21.5).describe() works and that type(Celsius) is type.
In new_vs_init.py, move the bases += (Tag,) line from __new__() into __init__() and predict the outcome before running it. Explain the result in terms of when the class object comes into existence.
commander.py validates class_name against KNOWN_COMMANDS before splicing it into source text. Remove that check, call Command.make_class() with a name containing a newline and a second statement, and confirm that the injected statement runs. make_class() splices the name in twice, the second time inside a string literal, so a bare newline ends the payload as an unterminated string. The payload’s last line must close or swallow that second splice. Restore the check.
Change prepare_namespace.py’s NoDuplicates so that instead of raising an exception, it keeps the first definition of a duplicated name and discards the later one. Give the two on_open bodies different print() calls so you can tell them apart, then confirm that Handlers().on_open() runs the first one. Explain why no class decorator could achieve the same thing.

pytest’s own spelling for @pytest.mark.parametrize, and “parameterize” everywhere else. Testing explains the four possible spellings.↩︎
C++ templates can do this via the Curiously Recurring Template Pattern (CRTP):
```
template <typename T>
class Singleton {
public:
    static T& instance() {
        static T inst;
        return inst;
    }
};

class ASingleton : public Singleton<ASingleton> {};
```
The C++ compiler instantiates templates on demand, rather than executing them the way Python executes a class statement. A C++ class name is a valid incomplete type the moment the compiler sees class ASingleton, before it reads a single member. Singleton<ASingleton> can use that name as its template argument while the class remains incomplete. The compiler does not compile its member functions until something calls them, by which point ASingleton is complete.

Python evaluates Singleton[ASingleton] eagerly, before it binds the name ASingleton, and offers no equivalent incomplete-type stage to exploit.↩︎