A value can be wrong, and every function that receives it checks again. A data class that refuses illegal values moves that check into the type.
A type is a set of values. The type int
is the set of whole numbers. A type you define, like a rating
from one to ten, is a smaller set: the allowed values.
Programmers have historically been bad at keeping objects inside
that set. It’s too easy to construct or mutate an object in an
illegal state. Defensive checks then spread through your
code.
This chapter shows a better approach, built from frozen data classes. You validate the value once, at construction, and freeze it so it cannot change. The type then guarantees that every object is a legal value. Code that receives one need not check it again. This material comes from my PyCon 2022 talk, Making Data Classes Work for You.
check() and
TypeFailurecheck() raises TypeFailure when its
condition is false. TypeFailure is a custom
exception meaning a value falls outside the type’s allowed
set:
# validation.py
from dataclasses import dataclass
@dataclass(eq=False)
class TypeFailure(ValueError):
subject: str
reason: str = ""
def __str__(self) -> str:
return f"{self.subject} {self.reason}".rstrip()
def check(condition: bool, subject: str,
reason: str = "") -> None:
if not condition:
raise TypeFailure(subject, reason)An exception is a value like any other, and the values it
carries deserve names. @dataclass generates the
constructor that stores them. subject is the
rejected value as the caller rendered it, such as
Stars(11). reason explains the
rejection when the name alone does not, such as
needs an @. A handler can read
e.subject and e.reason rather than
parsing them from the exception text.
check() uses an explicit raise
instead of assert 1 <= stars <= 10. Python
strips every assert when you run with
-O or -OO, which silently disables
every validation this chapter builds, the failure this chapter
exists to prevent. No flag removes a raise.
assert is a debugging aid for an invariant the
program’s own logic guarantees, which is why the optimizer may
remove it; a value that arrives from outside the program needs a
check that stays.
eq=False turns off the generated
__eq__(), for two reasons. A mutable data class
that defines __eq__() sets
__hash__ to None, and putting an
unhashable exception in a set raises a TypeError.
Identity is the correct comparison for an exception. Two
failures carrying the same text are still two separate
failures.
Suppose a “stars” rating is an integer from one to ten. If
you represent it as an int, nothing stops a caller
from passing eleven, or minus one. To prevent that, every
function that takes a rating must check it:
# stars_unchecked.py
from validation import check
def f1(stars: int) -> int:
# Check the argument here...
check(1 <= stars <= 10, f"f1({stars})")
return stars + 5
def f2(stars: int) -> int:
# ...and again in every other function
check(1 <= stars <= 10, f"f2({stars})")
return stars * 5
def f3(stars: int) -> int: # The check is missing here
return stars * 100
rating = 6
print(f1(rating))
#: 11
print(f2(rating))
#: 30
print(f3(11))
#: 1100Each function duplicates the check, the check is easy to
forget, and the type system is no help. The int
annotation says “any integer,” which is not what you mean.
Checking the argument also says nothing about the result.
f1(6) returns 11, which no rating may be.
f3() shows a forgotten check. 11 is
not a legal rating, and nothing objects: not the annotation, not
the type checker, not the running program. f3(11)
returns 1100, a number that no rating can produce, passed along
as if it were fine.
The object-oriented answer is to wrap the value in a class and validate it in the constructor. That is better, but it does not finish the job. The value is still mutable, so every method that changes it must re-validate, and a method can leave the object in an illegal state between steps:
# stars_class.py
from exceptions import expect
from validation import TypeFailure, check
class Stars:
def __init__(self, number: int) -> None:
self._number = number # Private by convention
self._validate()
def _validate(self) -> None:
check(1 <= self._number <= 10,
f"Stars({self._number})")
@property
# No setter: blocks external mutation
def number(self) -> int:
return self._number
def __str__(self) -> str:
return f"Stars({self._number})"
def f1(self) -> int:
self._number = self._number + 5
self._validate() # Postcondition
return self._number
if __name__ == "__main__":
rating = Stars(4)
print(rating)
print(rating.f1())
damaged = Stars(8)
expect(TypeFailure, damaged.f1)
print(damaged)
#: Stars(4)
#: 9
#: [TypeFailure] Stars(13)
#: Stars(13)A read-only @property keeps users from assigning
to number, but the object still mutates
_number, so f1() must re-check the
result before returning it. That check runs after the mutation,
not instead of it. Stars(8).f1() sets
_number to 13, then raises
TypeFailure, and the object goes on holding that
illegal value. Catching the exception does not undo the
damage.
The order is a choice, not something mutation forces.
f1() could check the sum before assigning it, and
then damaged stays Stars(8) instead of
holding a corrupted 13. What mutation forces is
that every method must choose that order correctly, every time
it changes the value. f1() here gets that order
wrong.
Design by Contract (DbC) is the practice of checking
arguments on entry and results on exit, with a class invariant
that must hold between calls. f1() takes no
argument, so only the postcondition appears here. A method that
accepts a second rating needs a precondition for it as well.
The problem with DbC is that the contract spreads across
every method that touches the value. The invariant is the part
this chapter replaces. _validate() states it, and
every mutating method must remember to call it. That is the same
scattering of checks as before, but moved inside the class. The
class encapsulates the value without constraining it to a set of
legal values.
Some values must change in place over their lifetime: a
counter, a connection’s open-or-closed state, a running total.
You cannot always replace one with a fresh instance on every
change. The accepted answer for those is a validating setter
that checks before assigning, the fix that stars_class.py’s
f1() skips. Pay DbC’s scattering cost, because the
value must stay mutable. Immutability covers the case the rest
of this chapter prefers, where a fresh, validated instance
replacing the old one is cheap enough.
A data class removes the boilerplate from a class
that holds data. The @dataclass decorator generates
__init__(), __repr__(), and
__eq__() from the fields you declare:
# messenger.py
from dataclasses import dataclass
@dataclass
class Messenger:
name: str
number: int
depth: float = 0.0display_object(),
the inspection helper, shows what @dataclass
generates:
# display_messenger_class.py
from display import INTERESTING_DUNDERS, display_object
from messenger import Messenger
display_object(Messenger, INTERESTING_DUNDERS)
#: [Attributes]
#: • __hash__ = None [CV]
#: • depth: float = 0.0 [CV]
#: [Methods]
#: • __eq__(self, other)
#: • __init__(self, name: str, number: int, depth: floa...
#: • __repr__(self)@dataclass generates the dunder methods, and the
constructor arguments cover all the fields in
Messenger. The trailing ... is
display_object() trimming that line to its report
width. __hash__ is None. A
@dataclass compares by value with
__eq__(), so it gives up hashability rather than
let you put a mutable instance in a set or use it
as a dict key. As Class
Attributes explains, depth is the one field of
the three that appears as an attribute, because it has an
initialization value. name and number
declare a type alone.
# demo_messenger.py
from dataclasses import replace
from messenger import Messenger
m = Messenger("iris", 12, 3.14)
print(m)
#: Messenger(name='iris', number=12, depth=3.14)
print(m.name, m.number, m.depth)
#: iris 12 3.14
# The generated __eq__ compares by field value:
print(Messenger("iris", 1) == Messenger("iris", 1))
#: True
print(Messenger("iris", 1) == Messenger("iris", 2))
#: False
mc = replace(m, depth=9.9) # Copy with one field changed
print(m)
#: Messenger(name='iris', number=12, depth=3.14)
print(mc)
#: Messenger(name='iris', number=12, depth=9.9)
m.name = "hermes" # Data classes are mutable by default
print(m)
#: Messenger(name='hermes', number=12, depth=3.14)print(m) uses the generated
__repr__(), which produces the class name and the
named argument values.
replace() returns a copy with some fields
changed, leaving the original alone. Copying instead of mutating
reduces errors. copy.replace()
does the same for anything immutable, not only for data
classes.
display_object() shows the attributes with their
declared types:
# display_messenger.py
from display import display_object
from messenger import Messenger
display_object(Messenger("iris", 12, 3.14))
#: [Attributes]
#: • depth: float = 3.14
#: • name: str = 'iris'
#: • number: int = 12
#: [Methods]
#: NoneThe default display_object() omits the generated
__init__(), __repr__(), and
__eq__().
Messenger shows what @dataclass
produces, but not what adding the decorator changes.
A is an ordinary class with bare
annotations.B adds default values but no constructor.C is a @dataclass.D adds a ClassVar field alongside
an ordinary one.The same helper inspects each one:
# comparison.py
from display import REDEFINED_DUNDERS, display_object
def show(obj: object) -> None:
display_object(obj, REDEFINED_DUNDERS,
exclude=("__hash__",))show() calls display_object() with
REDEFINED_DUNDERS, so each report lists only the
dunders a class customizes, not the standard machinery every
object inherits from object. For clarity,
show() also excludes __hash__ from these
reports.
A: Annotations
OnlyA is the simplest case, with no defaults and no
constructor, but with field declarations that look like class
variables:
# ordinary_class.py
from comparison import show
class A:
x: int
s: str
show(A())
#: [Attributes]
#: None
#: [Methods]
#: None
print(A.__annotations__)
#: {'x': <class 'int'>, 's': <class 'str'>}A does not override __init__(),
__repr__(), __eq__(), or
__hash__(), so every one of them is
object’s generic version, and
show(A()) reports none as redefined.
x and s in A are
bare annotations: declared, but not assigned a value. A
bare
annotation is a declaration rather than a placeholder. It
records, in A.__annotations__, that some future
A will carry an x and an
s, but stores nothing until code assigns a value.
A has no __init__() to assign them, so
the declaration goes unfulfilled. That is why
show(A()) finds nothing. No x and no
s exist to report, on the class or on the
instance.
B: Class-Level
DefaultsB adds default values to x and
s. The assignments allocate storage, so the two
turn from bare annotations into class variables:
# class_with_defaults.py
from comparison import show
class B:
x: int = 42
s: str = "Answer"
show(B())
#: [Attributes]
#: • s: str = 'Answer' [CV]
#: • x: int = 42 [CV]
#: [Methods]
#: None
print(B.__annotations__)
#: {'x': <class 'int'>, 's': <class 'str'>}The [CV] tags in show(B()) mark
both as class variables. B has no
__init__() to copy them onto each instance, so
every B object reads the same two values straight
from the class attributes.
C: The
Same Annotations, DecoratedC is A decorated with
@dataclass:
# plain_dataclass.py
from dataclasses import dataclass
from comparison import show
@dataclass
class C:
x: int
s: str
show(C(11, "this is C"))
#: [Attributes]
#: • s: str = 'this is C'
#: • x: int = 11
#: [Methods]
#: • __eq__(self, other)
#: • __init__(self, x: int, s: str) -> None
#: • __repr__(self)
print(C.__annotations__)
#: {'x': <class 'int'>, 's': <class 'str'>}show(C(11, "this is C")) finds the same two
names as show(B()). Neither x nor
s carries [CV] this time. Because
C is a @dataclass, its generated
__init__(self, x: int, s: str) -> None runs
self.x = x and self.s = s for every
new C. Each C instance owns its own
copies from the moment of construction. With no
__init__(), show(B()) keeps finding
x and s on the class, tagged
[CV], no matter how many B instances
exist.
C starts from the same bare annotations as
A. @dataclass reads them to learn what
fields exist and in what order, then uses that to write the
parameter list of __init__() and the assignments
inside it. dataclasses.fields() reports the field
list @dataclass recorded. @dataclass
stores no value for a field on the class. x is
still absent from C.__dict__ after decoration, as
it was before. The generated __init__() fulfills
the declaration when it runs, once per instance. That is the
difference from A: not that @dataclass
changes the annotations, but that it builds an
__init__() to act on them.
D: A Real
ClassVarD adds a real ClassVar alongside an
ordinary field:
# classvar_dataclass.py
from dataclasses import dataclass
from typing import ClassVar
from comparison import show
@dataclass
class D:
x: int = 99
s: ClassVar[str] = "Initializer"
f: ClassVar[float] # No initializer
show(D)
#: [Attributes]
#: • s: typing.ClassVar[str] = 'Initializer' [CV]
#: • x: int = 99 [CV]
#: [Methods]
#: • __eq__(self, other)
#: • __init__(self, x: int = 99) -> None
#: • __repr__(self)
show(D())
#: [Attributes]
#: • s: typing.ClassVar[str] = 'Initializer' [CV]
#: • x: int = 99
#: [Methods]
#: • __eq__(self, other)
#: • __init__(self, x: int = 99) -> None
#: • __repr__(self)
for k, v in D.__annotations__.items():
print(f"'{k}': {v}")
#: 'x': <class 'int'>
#: 's': typing.ClassVar[str]
#: 'f': typing.ClassVar[float]show(D) tags both attributes [CV],
since no instance owns either of them yet.
show(D()) tags only one, and what
@dataclass generates for each field decides which
one keeps the tag.
x is an ordinary field. __init__()
takes it as a parameter and runs self.x = x, so
each new D gets its own copy at construction. That
is why show(D())’s x: int = 99 carries
no tag. It now lives in that instance’s own
__dict__, not on the class.
s, declared ClassVar[str], behaves
differently. @dataclass treats a
ClassVar field as belonging to the class, not to
any instance, and leaves it out of __init__().
__init__(self, x: int = 99) -> None has no
s parameter, so no constructor call can assign one.
s stays on D and keeps its
[CV] tag no matter how many D objects
exist.
f: ClassVar[float] appears in neither report. It
has no initializer, so it is a bare annotation, as
x and s are back in A: a
declaration recorded in D.__annotations__, with no
value stored anywhere to report. D.f raises
AttributeError, for the same reason
A().x does. Assigning a value creates the
attribute. Declaring it ClassVar does not.
Passing frozen=True makes the data class
immutable. Assigning to a field raises
FrozenInstanceError. A frozen instance is also
hashable, so you can use it as a dictionary key or put it in a
set. frozen=True removes the mutability that cost
Messenger its __hash__, so
@dataclass generates one from the fields:
# frozen_messenger.py
from dataclasses import dataclass
from exceptions import expect
@dataclass(frozen=True)
class Messenger:
name: str
number: int
depth: float = 0.0
m = Messenger("iris", 12, 3.14)
print(m)
#: Messenger(name='iris', number=12, depth=3.14)
expect(Exception, setattr, m, "name", "hermes")
#: [FrozenInstanceError] cannot assign to field 'name'
cache = {m: "Ni!"} # Frozen instances are hashable
print(cache[m])
#: Ni!Two defenses guard a frozen field. The type checker rejects
m.name = "hermes" before the program runs, which is
why the listing goes through setattr() to reach the
runtime defense. frozen=True holds at runtime,
against code the type checker did not see.
frozen=True guards the binding, not the object
behind it. You can still mutate a field’s list in
place, and a frozen instance is hashable only when every field
it holds is. Rethinking
Objects demonstrates that leak. The types this chapter
validates hold ints and strs, so for
them the guarantee is total. Months and
Line, both later in this chapter, each hold a
list. Neither is hashable, and neither is safe from
a change made through that list.
A frozen data class still carries a per-instance
__dict__. Adding slots=True drops it,
for less memory and faster attribute access. Performance
measures the memory difference.
The standard library has a second immutable record,
typing.NamedTuple, which also rejects assignment
and hashes under the same rule. Every field it holds must be
hashable. The two differ in equality. A frozen data class equals
another instance of its own class, while a
NamedTuple equals
any tuple holding the same values. They also differ in validation.
If nothing about an object can change after construction, then validating it at construction makes it valid for its lifetime.
If you make Stars a frozen data class, you can
guarantee that every Stars object is legal.
__post_init__() validates a Stars
after its fields receive their values. The generated
__init__() calls __post_init__()
automatically:
# stars.py
from dataclasses import dataclass
from validation import check
@dataclass(frozen=True)
class Stars:
number: int
def __post_init__(self) -> None:
check(1 <= self.number <= 10,
f"Stars({self.number})")
def f1(s: Stars) -> Stars:
return Stars(s.number + 5)
def f2(s: Stars) -> Stars:
return Stars(s.number * 5)
if __name__ == "__main__":
print(Stars(4))
print(f1(Stars(2)))
print(f2(Stars(2)))
#: Stars(number=4)
#: Stars(number=7)
#: Stars(number=10)Stars now constrains number to a
set of values: the integers one through ten. The only way to
make a Stars is through the constructor, and the
constructor refuses anything outside that set. If you hold a
Stars, it is legal. You know it without
checking.
That guarantee changes how you write the functions.
f1() and f2() take a
Stars and return a Stars. They do not
check their argument, because every Stars is legal.
They do not test their result, because building the returned
Stars runs the check.
The validation lives in one place, the constructor, so it is easy to change. Immutability guarantees no one can rebind the fields after construction, and when the fields are immutable too, no one can damage the value.
Two checks define the set, and they run at different times.
The annotation number: int belongs to the type
checker, which rejects Stars(5.5) and
Stars("five") before the program runs.
__post_init__() holds the range and runs at
construction. @dataclass uses an annotation to find
a field and does not enforce it at runtime, so a
float the type checker did not see meets the range
check alone:
# stars_float.py
from stars import Stars
print(Stars(5.5)) # type: ignore
#: Stars(number=5.5)The # type: ignore silences the type checker so
the listing can reach the runtime result. Inside annotated code
the type checker closes this gap, and
__post_init__() need not repeat the annotation. A
value from outside the program is different. Parsed JSON arrives
as Any, which the type checker accepts for every
parameter, so a type built at that boundary tests the field’s
type in the check as well:
check(isinstance(self.number, int) and 1 <= self.number <= 10, ...).
__post_init__() can check a field but cannot
change one. frozen=True works by installing a
__setattr__() that rejects every assignment,
including the ones arriving from inside the class, so
normalizing a field in __post_init__() raises
FrozenInstanceError:
# post_init_normalize.py
from dataclasses import dataclass
from exceptions import expect
@dataclass(frozen=True)
class Email:
text: str
def __post_init__(self) -> None:
self.text = self.text.lower() # type: ignore
expect(Exception, Email, "Grace@Example.com")
#: [FrozenInstanceError] cannot assign to field 'text'
@dataclass(frozen=True)
class Normalized:
text: str
def __post_init__(self) -> None:
object.__setattr__(self, "text", self.text.lower())
print(Normalized("Grace@Example.com"))
#: Normalized(text='grace@example.com')Both defenses fire here, as they do for frozen_messenger.py. The
type checker reports the assignment, and the
# type: ignore silences it so the listing can reach
the runtime failure.
object.__setattr__() skips the rejecting
__setattr__() and writes the field into the
instance’s __dict__. It works, and it says what it
does. The same call works from outside the class, so
frozen=True stops an accidental assignment, not a
caller determined to change the value.
The alternative is to refuse the unnormalized value and
normalize before construction. Which to choose depends on the
type. Normalizing inside makes
Normalized("A@b.com") and
Normalized("a@b.com") the same value, which is
usually what an email address should mean. Refusing instead
keeps the constructor a gate and leaves the cleanup to the
caller.
Validating once, at construction, often goes by the name parse, don’t validate.1 Instead of checking a changeable value everywhere and hoping you cover every spot, you parse it once into a precise type. After that, holding the type is proof the check passed. No other code repeats the check, because it cannot fail. Illegal values are unrepresentable.
Parsing once into a precise type is one aspect of functional
programming. Instead of mutating an object and re-guarding
it, you transform one legal value into a new legal value. A
parameter annotated
Stars states more than a type: the value passed the
check.
test_stars.py demonstrates
that illegal values cannot exist. The test file confirms
that:
Stars(1) and Stars(10), the ends
of the range, are legal.f1() and f2() return legal
Stars.TypeFailure
when f2() builds it.# test_stars.py
import pytest
from stars import Stars, f1, f2
from validation import TypeFailure
def test_legal_stars() -> None:
assert Stars(1).number == 1
assert Stars(10).number == 10
@pytest.mark.parametrize("n", [0, 11, -1, 100])
def test_illegal_stars_rejected(n: int) -> None:
with pytest.raises(TypeFailure):
Stars(n)
def test_transformations_return_legal_values() -> None:
assert f1(Stars(2)) == Stars(7)
assert f2(Stars(2)) == Stars(10)
def test_illegal_result_is_rejected() -> None:
with pytest.raises(TypeFailure, match=r"Stars\(20\)"):
f2(Stars(4))Once each small type guarantees its own values, you can
safely build larger types out of them. A Person
made of a valid FullName and a valid
EmailAddress is valid by construction:
# person.py
from dataclasses import dataclass
from validation import check
@dataclass(frozen=True)
class FullName:
text: str
def __post_init__(self) -> None:
check(len(self.text.split()) >= 2,
f"FullName({self.text!r})",
"needs a first and last name")
@dataclass(frozen=True)
class EmailAddress:
text: str
def __post_init__(self) -> None:
check("@" in self.text,
f"EmailAddress({self.text!r})",
"needs an @")
@dataclass(frozen=True)
class Person:
name: FullName
email: EmailAddress
if __name__ == "__main__":
person = Person(
FullName("Grace Hopper"),
EmailAddress("grace@example.com"),
)
print(person.name)
print(person.email)
#: FullName(text='Grace Hopper')
#: EmailAddress(text='grace@example.com')Person declares no checks of its own. Its
annotations require a FullName and an
EmailAddress, which the type checker enforces, and
neither can exist holding an illegal value.
test_person_composes_validated_parts() builds a
Person from legal parts and reads them back.
test_full_name_needs_first_and_last() and
test_email_needs_at_sign() show that an illegal
name or email cannot exist, so no Person can hold
one:
# test_person.py
import pytest
from person import EmailAddress, FullName, Person
from validation import TypeFailure
def test_person_composes_validated_parts() -> None:
person = Person(FullName("Grace Hopper"),
EmailAddress("grace@example.com"))
assert person.name.text == "Grace Hopper"
assert person.email.text == "grace@example.com"
@pytest.mark.parametrize("bad", ["Grace", "", " "])
def test_full_name_needs_first_and_last(bad: str) -> None:
with pytest.raises(TypeFailure):
FullName(bad)
@pytest.mark.parametrize(
"bad", ["grace", "example.com", ""])
def test_email_needs_at_sign(bad: str) -> None:
with pytest.raises(TypeFailure):
EmailAddress(bad)When validation grows complicated, libraries make it lighter. The attrs library predates and inspired data classes and offers richer validators and converters. Pydantic builds validation and parsing into the type, which is especially useful at the edges of a program where untrusted data can enter. The principle is the same. Make the type guarantee its own values.
When the set of values is small and known, the clearest type
is an Enum. An enumeration lists its
members by name in the class body, and those members are the
only values the type has. As an example, a
BirthDate contains a month, day, and year. A year
has twelve months, so Month is an
Enum. Each month carries its length and knows how
to check a Day against it. A BirthDate
then validates across its fields. The day must fit the
month.
# birth_date.py
from dataclasses import dataclass
from datetime import date
from enum import Enum
from validation import check
@dataclass(frozen=True)
class Day:
n: int
def __post_init__(self) -> None:
check(1 <= self.n <= 31, f"Day({self.n})")
@dataclass(frozen=True)
class Year:
n: int
def __post_init__(self) -> None:
check(1900 < self.n <= date.today().year,
f"Year({self.n})")
class Month(Enum):
JANUARY = (1, 31)
FEBRUARY = (2, 28) # Leap years are left as an exercise
MARCH = (3, 31)
APRIL = (4, 30)
MAY = (5, 31)
JUNE = (6, 30)
JULY = (7, 31)
AUGUST = (8, 31)
SEPTEMBER = (9, 30)
OCTOBER = (10, 31)
NOVEMBER = (11, 30)
DECEMBER = (12, 31)
@staticmethod
def of(month_number: int) -> Month:
check(1 <= month_number <= 12,
f"Month({month_number})")
return list(Month)[month_number - 1]
@property
def max_days(self) -> int:
return self.value[1]
def check_day(self, day: Day) -> None:
check(day.n <= self.max_days, f"Day({day.n})",
f"is past the end of {self.name}")
def __repr__(self) -> str:
return self.name
@dataclass(frozen=True)
class BirthDate:
month: Month
day: Day
year: Year
def __post_init__(self) -> None:
self.month.check_day(self.day)
if __name__ == "__main__":
print(BirthDate(Month.of(7), Day(8), Year(1957)))
#: BirthDate(month=JULY, day=Day(n=8), year=Year(n=1957))The Enum creates the constrained set of
Months. No thirteenth month can exist, because
Month defines no such value.
Each member is a pair rather than a bare day count because
Enum treats members with equal values as aliases of
one another. Writing APRIL = 30 and
JUNE = 30 makes JUNE a second name for
APRIL, and only three day counts are distinct, so
list(Month) returns three members instead of
twelve. Pairing each month with its number keeps all twelve
values distinct, which of() relies on when it
indexes list(Month). The member’s value is no
longer the month number, so Month(7) raises a
ValueError. of() is the replacement
lookup.
The tests confirm that Month.of(7) finds
JULY in a legal date, and that a day past the end
of its month and a month number outside the twelve both raise
TypeFailure:
# test_birth_date.py
import pytest
from birth_date import BirthDate, Day, Month, Year
from validation import TypeFailure
def test_of_finds_the_month() -> None:
bd = BirthDate(Month.of(7), Day(8), Year(1957))
assert bd.month is Month.JULY
@pytest.mark.parametrize("month_n, day_n", [
(2, 31),
(4, 31),
(9, 31),
])
def test_day_past_end_of_month(month_n: int,
day_n: int) -> None:
assert Month.of(month_n).max_days < day_n
with pytest.raises(TypeFailure):
BirthDate(Month.of(month_n), Day(day_n), Year(2020))
@pytest.mark.parametrize("bad", [0, 13, -1])
def test_month_number_outside_twelve(bad: int) -> None:
with pytest.raises(TypeFailure):
Month.of(bad)Enum Beats a Data ClassYou can build Month as a data class instead of
an Enum. It works, but it is more code for less
safety. You must construct the twelve months yourself and carry
them around, whereas the Enum is that set:
# month_dataclass.py
from dataclasses import dataclass, field
from birth_date import Day
from validation import check
@dataclass(frozen=True)
class Month:
name: str
n: int
max_days: int
def __post_init__(self) -> None:
check(1 <= self.n <= 12, f"Month({self.n})")
check(self.max_days in (28, 30, 31),
f"Month(max_days={self.max_days})",
"is not a month length")
def check_day(self, day: Day) -> None:
check(day.n <= self.max_days, f"Day({day.n})",
f"is past the end of {self.name}")
def make_months() -> list[Month]:
return [Month(name, n, days)
for n, (name, days) in enumerate([
("January", 31), ("February", 28), ("March", 31),
("April", 30), ("May", 31), ("June", 30),
("July", 31), ("August", 31), ("September", 30),
("October", 31), ("November", 30), ("December", 31),
], start=1)]
@dataclass(frozen=True)
class Months:
months: list[Month] = field(default_factory=make_months)
def of(self, month_number: int) -> Month:
check(1 <= month_number <= 12,
f"Month({month_number})")
return self.months[month_number - 1]
if __name__ == "__main__":
months = Months()
print(months.of(7))
#: Month(name='July', n=7, max_days=31)Months carries a list of its twelve
Months, so its months field needs
field(default_factory=make_months) rather than a
default value.
Choose the tool that makes the legal set easiest to express.
For a small known set, that is an Enum.
Months declares
months: list[Month] = field(default_factory=make_months).
@dataclass rejects = make_months() at
class-definition time, with
ValueError: mutable default <class 'list'> for field months is not allowed: use default_factory.
The rejection prevents shared storage. Python evaluates a
default value once, at class definition, so with that default
every Months reads and writes one list, the mutable-default
trap. field(default_factory=make_months)
supplies a function instead of a value, and each new
Months calls it and gets its own fresh list.
That rejection is narrower than it looks.
@dataclass refuses a default it recognizes as
shared storage, which covers list,
dict, and set. The test is
hashability, not mutability, so a mutable object of an ordinary
class you wrote passes as a default and every instance shares
it. That is the same bug the check exists to prevent. Use
default_factory for any mutable default.
default_factory accepts any callable that takes
no arguments. A named function like make_months()
is one. A type is another, which is why
field(default_factory=list) appears throughout this
book. Calling list builds an empty one. A
subscripted generic is callable too, so
field(default_factory=dict[str, str]) is legal and
produces an empty dict. That form seems redundant, because the
annotation on the left names the type, and the subscript
vanishes at runtime. It gains one thing:
# factory_checking.py
from dataclasses import dataclass, field
from exceptions import expected
@dataclass
class Unchecked:
# A set
data: dict[str, str] = field(default_factory=set)
@dataclass
class Checked:
data: dict[str, str] = field(
default_factory=dict[str, str])
print(type(Unchecked().data).__name__)
#: set
with expected(TypeError):
Unchecked().data["theme"] = "dark"
#: [TypeError] 'set' object does not support item assignment
print(Checked().data)
#: {}Nothing objects to Unchecked. ty
accepts it, the linter (ruff) accepts it, and the
declaration says dict[str, str], so every reader
expects a dict. What arrives is a set, and the
mistake surfaces at the first item assignment, which can be far
from the declaration that caused it. A bare list,
dict, or set produces a type loose
enough that ty accepts it against any annotation,
so ty does not compare the factory with the field.
Checkers differ here. Pyright infers set[Unknown]
for this factory and rejects it against
dict[str, str].
Subscripting makes the factory’s return type concrete, and
field(default_factory=dict[int, int]) on this field
then draws a type error. Use the bare form when the factory and
the annotation agree, which is most of the time. Subscript the
factory when you want that agreement checked.
NamedTuple
Cannot Validate ItselfA NamedTuple is the other immutable record, and
it is tempting for a small type like Stars.
typing.NamedTuple forbids overriding the methods
that build an instance, so validation must live in a factory
function beside the type, where a caller can go around it:
# test_namedtuple_no_hook.py
from typing import NamedTuple
import pytest
from validation import TypeFailure, check
class Stars(NamedTuple):
number: int
# Validation lives outside
def make_stars(number: int) -> Stars:
check(1 <= number <= 10, f"Stars({number})")
return Stars(number)
def test_the_factory_rejects_illegal_values() -> None:
assert make_stars(10).number == 10
with pytest.raises(TypeFailure):
make_stars(11)
def test_calling_the_type_skips_the_check() -> None:
assert Stars(11).number == 11
def test_the_check_cannot_move_inside() -> None:
with pytest.raises(AttributeError,
match="Cannot overwrite"):
class Validated(NamedTuple):
number: int
def __new__(cls, number: int) -> Validated: # type: ignore
check(1 <= number <= 10, f"Stars({number})")
return tuple.__new__(cls, (number,))test_the_factory_rejects_illegal_values() and
test_calling_the_type_skips_the_check() are test_stars.py inverted.
There, no illegal Stars can exist. Here,
Stars(11) builds one, because a factory function is
advice rather than a gate, a limit every factory in Factory
shares.
test_the_check_cannot_move_inside() shows why
the check cannot move inside the type. NamedTuple
refuses __new__(), refuses __init__()
the same way, and no class comes into existence. The error
arrives while Python is still executing the class
statement. ty reports the __new__()
override as invalid-named-tuple, which the
# type: ignore silences. Pyright accepts the
__new__() without comment.
A subclass of the NamedTuple may define
__new__(). That gets past the prohibition and moves
the hole rather than closing it. _replace()
rebuilds through tuple.__new__(), not through your
__new__(), so copy.replace() on a
validated instance quietly produces an unvalidated one.
A frozen data class runs __post_init__() on
every construction, including the ones you did not anticipate,
and replacement has no such back door. copy.replace()
goes through the constructor. That is the deciding difference
whenever a type must guarantee its own values. When it need not,
a NamedTuple is a fine immutable
record.
__init__()A data class builds its __init__() from its
fields and assigns them. It does not call the base class
__init__(), because it cannot know what arguments
that constructor expects. If you inherit from an ordinary class
that sets up state in its own constructor, the data class
silently skips that setup:
# dataclass_inherits_plain.py
from dataclasses import dataclass
class Connection:
def __init__(self, host: str) -> None:
self.host = host
self.url = f"tcp://{host}:5432"
@dataclass
class Logged(Connection):
name: str
c = Logged("db")
print(c.name)
#: db
# Connection.__init__ never ran, so host and url are unset:
print(hasattr(c, "host"), hasattr(c, "url"))
#: False FalseThe generated __init__() assigns
name and stops. Because nothing calls
Connection.__init__(), neither host
nor url exists. The omission is easy to miss
because Logged("db") still succeeds.
To run the base initializer, call it yourself from
__post_init__(), which runs after the generated
__init__() assigns the fields:
# dataclass_super_init.py
from dataclasses import dataclass
class Connection:
def __init__(self, host: str) -> None:
self.host = host
self.url = f"tcp://{host}:5432"
@dataclass
class Logged(Connection):
host: str
name: str
def __post_init__(self) -> None:
super().__init__(self.host)
c = Logged("localhost", "db")
print(c.url, c.name)
#: tcp://localhost:5432 dbNo field declaration produces url.
Connection.__init__() derives it from
host, so printing url proves that
__post_init__() called the base initializer. If you
delete __post_init__(), the same line raises an
AttributeError.
If a base __init__() instead replaces
self.__dict__, calling it from
__post_init__() discards the fields the data class
just assigned. The Borg
singleton is that case.
When the base class is also a data class, you need no
__post_init__(). The subclass generates one
__init__() covering the inherited fields and the
new ones, in order:
# dataclass_inherits_dataclass.py
from dataclasses import dataclass
@dataclass
class Connection:
host: str
@dataclass
class Logged(Connection):
name: str
c = Logged("localhost", "db")
print(c.host, c.name)
#: localhost dbA default in the base class is the one snag. If
host had a default, name would follow
it with none, and @dataclass refuses that order (More Data Class Tools has the
rule). kw_only=True on Logged makes
name keyword-only, and the rule stops applying.
Two data classes in one hierarchy must agree about
frozen. Mixing the settings fails in either
direction:
# frozen_inheritance.py
from dataclasses import dataclass
from exceptions import expected
@dataclass(frozen=True)
class Frozen:
a: int
@dataclass
class Plain:
a: int
with expected(TypeError):
@dataclass
class Thawed(Frozen): # type: ignore
b: int
#: [TypeError] cannot inherit non-frozen dataclass from a
#: frozen one
with expected(TypeError):
@dataclass(frozen=True)
class Chilled(Plain): # type: ignore
b: int
#: [TypeError] cannot inherit frozen dataclass from a non-
#: frozen oneBoth defenses fire again. The type checker reports each class
as an invalid frozen-dataclass subclass, which the
# type: ignore silences so the listing can reach
the runtime failure.
frozen=True works by installing a
__setattr__() that rejects an assignment to any
field of the frozen class, and a subclass inherits that method.
A plain subclass of a frozen class is half writable, its own
fields assignable and the inherited ones not. A frozen subclass
of a plain base is the opposite, freezing every attribute the
mutable base expects to set. So @dataclass refuses
at class-definition time rather than at the first surprising
assignment. Every validated type in this chapter carries
frozen=True, so anything you derive from one must
carry it too.
asdict() and astuple() convert an
instance to a dictionary or tuple, recursing into nested data
classes:
# asdict_astuple.py
from dataclasses import asdict, astuple, dataclass
@dataclass(frozen=True)
class Point:
x: int
y: int
p = Point(10, 20)
print(asdict(p))
#: {'x': 10, 'y': 20}
print(astuple(p))
#: (10, 20)
@dataclass(frozen=True)
class Line:
points: list[Point]
line = Line([Point(2, 7), Point(10, 4)])
print(asdict(line)) # Recurses into the list of Points
#: {'points': [{'x': 2, 'y': 7}, {'x': 10, 'y': 4}]}asdict() and astuple() copy as they
go, so every list and dict in the result is a new object rather
than the one inside the instance. Changing the result leaves the
original untouched.
KW_ONLY forces callers to pass the fields after
it by keyword:
# kw_only_config.py
from dataclasses import KW_ONLY, dataclass
@dataclass
class Config:
source: str
# Everything after this must be passed by keyword:
_: KW_ONLY
verbose: bool = False
retries: int = 3
print(Config("data.csv", retries=5))
#: Config(source='data.csv', verbose=False, retries=5)Passing kw_only=True to @dataclass
makes every field keyword-only. The _: KW_ONLY
marker limits that to the fields after it, leaving
source positional.
KW_ONLY also lifts the ordering rule. A field
with no default normally cannot follow one that has a default,
because the generated __init__() then needs a
required parameter after an optional one.
@dataclass refuses that with
TypeError: non-default argument 'b' follows default argument 'a'.
Because fields after _: KW_ONLY are keyword-only,
their order no longer matters and the rule stops applying.
replace()dataclasses.replace() works only on data
classes, but “same object, one field different” is what you do
with any immutable value. copy.replace() is the
general version, and it works on a frozen data class, a
NamedTuple, a date or
datetime, a SimpleNamespace, and
anything else that defines __replace__():
# copy_replace.py
import copy
from datetime import date
from typing import NamedTuple
from exceptions import expect
from stars import Stars
class Size(NamedTuple):
width: int
height: int
print(copy.replace(Stars(4), number=9))
#: Stars(number=9)
print(copy.replace(Size(4, 3), height=9))
#: Size(width=4, height=9)
print(copy.replace(date(2026, 8, 4), day=1))
#: 2026-08-01
expect(Exception, copy.replace, Stars(4), number=99)
#: [TypeFailure] Stars(99)For a data class, copy.replace() builds the new
object through the constructor, so
Stars.__post_init__() runs on the copy. A validated
type stays validated across a replacement, so
copy.replace() transforms one legal value into a
new legal value.
copy() and
deepcopy() Skip the Constructorcopy.copy() and copy.deepcopy()
copy without calling the constructor.
# replace_vs_copy.py
import copy
from dataclasses import dataclass
from validation import check
@dataclass(frozen=True)
class Stars:
number: int
def __post_init__(self) -> None:
print(f"checking {self.number}")
check(1 <= self.number <= 10,
f"Stars({self.number})")
s = Stars(4)
#: checking 4
print(copy.replace(s, number=2))
#: checking 2
#: Stars(number=2)
print(copy.copy(s))
#: Stars(number=4)
print(copy.deepcopy(s))
#: Stars(number=4)copy.copy() and copy.deepcopy()
create the copy with __new__() and write the saved
state into its __dict__, and so does
pickle. None of the three runs
__init__() or __post_init__(), so each
can produce a Stars holding a number that no check
saw. That is harmless while the copy comes from a legal
original, and dangerous when the state comes from a file written
by an older version of the class, the case Memento revisits.
copy.replace() is the one that keeps the guarantee,
because rebuilding through the constructor is the only way to
get the check back.
__replace__()__replace__() is a dunder like any other, so a
class that is not a data class can join by defining it:
# copy_replace_protocol.py
import copy
from typing import Final, Literal, Self
type Channel = Literal["red", "green", "blue"]
SHIFTS: Final[dict[Channel, int]] = {
"red": 16, "green": 8, "blue": 0}
MASK: Final[int] = 0xFF
class Color:
def __init__(self, red: int, green: int,
blue: int) -> None:
channels: dict[Channel, int] = {
"red": red, "green": green, "blue": blue}
self.packed = sum(v << SHIFTS[k]
for k, v in channels.items())
@property
def channels(self) -> dict[Channel, int]:
return {n: self.packed >> s & MASK
for n, s in SHIFTS.items()}
def __repr__(self) -> str:
channels = map(str, self.channels.values())
return f"Color({', '.join(channels)})"
def __replace__(self, **changes: int) -> Self:
return type(self)(**(self.channels | changes))
teal = Color(0, 128, 128)
print(teal, hex(teal.packed))
#: Color(0, 128, 128) 0x8080
lighter = copy.replace(teal, red=64)
print(lighter, hex(lighter.packed))
#: Color(64, 128, 128) 0x408080Color is an ordinary class with its channels
packed into one int, so
dataclasses.replace() rejects it with a
TypeError. __replace__() unpacks the
channels, applies the changes, and hands the result back through
the constructor. Every implementation takes that shape: recover
the constructor arguments, override the named ones, rebuild.
__init__() packs a channel dictionary using
SHIFTS, and channels unpacks one with
the same table, so the two are inverses. Calling
type(self)(...) rather than Color(...)
gives a subclass a copy of its own class, and the
Self return annotation tells the type checker
so.
Define __replace__() when your type is immutable
and callers need variants of it. Skip it when the type is
mutable, because the caller can assign to the attribute.
A data class has no built-in JSON support. Handed a data
class instance, json.dumps() raises
TypeError: Object of type Person is not JSON serializable.
asdict() turns the object into a nested
dictionary, and json.dumps() knows how to serialize
dictionaries. Decoding goes the other way.
from_json() parses the JSON into a dictionary, then
hands its parts to the constructors.
# json_round_trip.py
import json
from dataclasses import asdict
from typing import Any
from person import EmailAddress, FullName, Person
def to_json(person: Person) -> str:
return json.dumps(asdict(person), indent=2)
def from_json(text: str) -> Person:
data: dict[str, Any] = json.loads(text)
return Person(
FullName(data["name"]["text"]),
EmailAddress(data["email"]["text"]),
)
original = Person(FullName("Grace Hopper"),
EmailAddress("grace@example.com"))
text = to_json(original)
print(text)
#: {
#: "name": {
#: "text": "Grace Hopper"
#: },
#: "email": {
#: "text": "grace@example.com"
#: }
#: }
print(from_json(text) == original) # Round-trip
#: TrueJSON data typically arrives from outside the program,
untrusted. Rebuilding the value through Person,
FullName, and EmailAddress runs each
constructor’s validation, so the boundary rejects an illegal
value instead of leaking it into the rest of the code.
A custom JSONEncoder serializes any data class
it meets, including one nested inside other structures, by
converting each one to a dict:
# json_encoder.py
import json
from dataclasses import asdict, is_dataclass
from typing import Any, override
from person import EmailAddress, FullName, Person
class DataClassEncoder(json.JSONEncoder):
@override
def default(self, o: Any) -> Any:
if is_dataclass(o) and not isinstance(o, type):
return asdict(o)
return super().default(o)
people = [
Person(FullName("Ada Lovelace"),
EmailAddress("ada@example.com")),
Person(FullName("Alan Turing"),
EmailAddress("alan@example.com")),
]
print(json.dumps(people, cls=DataClassEncoder, indent=2))
#: [
#: {
#: "name": {
#: "text": "Ada Lovelace"
#: },
#: "email": {
#: "text": "ada@example.com"
#: }
#: },
#: {
#: "name": {
#: "text": "Alan Turing"
#: },
#: "email": {
#: "text": "alan@example.com"
#: }
#: }
#: ]json.dumps() calls default() for
any object it cannot serialize on its own. The encoder converts
each data class to a dictionary, and the base encoder handles
the dictionary from there, recursing through lists and nested
objects. is_dataclass() answers True
for the class object as well as for an instance, and
asdict() accepts only instances, so
not isinstance(o, type) keeps a bare class object
from reaching asdict().
Encoding is mechanical, but decoding must know which type to rebuild, and that part the standard library leaves to you. For deep or evolving structures, Pydantic and dataclasses-json automate the decode side, reconstructing nested types from the parsed JSON and validating as they go.
stars_unchecked.py spreads
the checks across every function that takes a rating, and stars_class.py spreads
them across every method that changes one. stars.py puts them in the
constructor, where they run once, for every value the program
constructs.
That trade has a price, and the price is at the edges. Every
place data enters your program now needs a constructor call: the
parsed JSON, the database row, the form field, the command-line
argument. from_json() is that boundary written out.
It reads untrusted text and hands the pieces to
FullName and EmailAddress, and that
boundary is the last point where a bad value is cheap to reject.
Past that line your code holds types rather than raw data, and a
function receiving one does its work without asking whether the
value makes sense.
The price also shows up away from the boundaries, in memory
and time. Every value is now an object: a constructor call and
attribute access where a bare int or
str needs neither. copy.replace()
re-validates the whole object on every change, even when only
one field moved. For a hot path, or a structure nesting many
values (many Points inside a Line),
that cost is worth
measuring before you pay it everywhere.
This chapter’s solutions give a hint, usually the shape of the code, and a full answer for each exercise.
Month, so February
allows 29 days when the BirthDate’s
Year is a leap year. Write the tests first.EmailAddress a stricter check (a single
@, with text on both sides). Add tests for the
values the check should now reject.test_namedtuple_no_hook.py’s
Stars and build the subclass workaround: a
_Stars(NamedTuple) holding the field, and a
Stars(_Stars) whose __new__() runs the
check. Show that Stars(11) now raises a
TypeFailure while
copy.replace(Stars(5), number=99) does not, and
explain why a frozen data class has no equivalent hole.from_json() a JSON string whose email has
no @, and confirm that it raises
TypeFailure. The validation you wrote once, in
EmailAddress, now also guards your JSON input.copy.replace() work on a
Stars that is not a data class: write an ordinary
class holding the rating, validate in __init__(),
define __replace__(), and confirm that
copy.replace() still runs your validation.ClassVar[int] counter to
Stars that counts every Stars created.
Predict whether it appears in the generated
__init__()’s parameter list before you run it, then
check by printing
inspect.signature(Stars.__init__). Incrementing the
counter as Stars.built += 1 from
__post_init__() works on a frozen class, while
self.built += 1 does not. Explain why.Months a second field, a
dict[str, Month] index written with a
= {} default, and read the error
@dataclass reports. Then fix it two ways, with
default_factory=dict and with
default_factory=dict[str, Month], and say which one
a type checker can verify.stars_float.py builds a
Stars holding 5.5. Add a type test to
the check in __post_init__() so that
Stars(5.5) raises TypeFailure.
Stars(True) also passes the range check. Explain
why an isinstance() test accepts it, and write the
test so that it rejects True as well.