Contents
Chapter 37

Pattern Refactoring

This chapter follows one problem through several designs. A first solution solves it, then you ask “what will change?” and reshape the design to absorb that change cheaply. This is the spirit of Martin Fowler’s Refactoring, applied to patterns rather than single statements.

It is also a Python lesson. Many patterns in GoF Design Patterns work around the limitations of statically typed languages: single dispatch, closed classes, and types that are not values. Python’s classes stay open, its types are values, and functools.singledispatch adds an operation from outside a class, so some of those patterns become unnecessary. This chapter points out each one as the example reaches it.

The example is a trash sorting simulation, and it evolves across the chapter: one design, then a requirement that breaks it, then a reshaping that absorbs the change, then a second axis of change the reshaped design does not touch. Read that evolution as a template for your own designs, which can start as an adequate fit for one problem and grow into a flexible fit for a class of problems.

Simulating a Trash Recycler

Trash arrives at the recycling plant mixed together. The program must sort it by material and report the total value of each kind. The trash starts out as an undifferentiated pile, and you must recover the type of each piece to sort it.

In the Trash hierarchy, each material carries a per-pound value. The base class keeps a registry of its subclasses, which __init_subclass__() fills automatically, and a create() method builds an instance from a material name, the dictionary factory from Factory:

Each Trash subclass registers itself, and sorting keys the bins dict by type(t) instead of naming any material 
# trash.py
from dataclasses import dataclass
from typing import ClassVar

type Bins = dict[type[Trash], list[Trash]]

@dataclass(frozen=True)
class Trash:
    weight: float
    # Dollars per pound (per subclass)
    value: ClassVar[float] = 0.0
    registry: ClassVar[dict[str, type[Trash]]] = {}

    def __init_subclass__(cls, **kwargs: object) -> None:
        super().__init_subclass__(**kwargs)
        Trash.registry[cls.__name__] = cls

    @classmethod
    def create(cls, name: str, weight: float) -> Trash:
        return cls.registry[name](weight)

class Aluminum(Trash):
    value = 1.67

class Paper(Trash):
    value = 0.10

class Glass(Trash):
    value = 0.23

class Cardboard(Trash):
    value = 0.79

def sum_value(items: list[Trash]) -> float:
    total = sum(t.weight * t.value for t in items)
    print(f"Total value = {total:.2f}")
    return total

Bins names the shape the sorting sections use, a dictionary from a material’s class to the pieces made of that material. The type Statement introduces this alias form. A type statement’s right side evaluates lazily, so the alias can name Trash several lines before the class statement that defines it.

Python implicitly makes __init_subclass__() a classmethod, so it needs no @classmethod decorator and its first parameter is the new subclass. It runs once per subclass, immediately after Python creates that subclass, so each one can register itself in Trash.registry automatically. create() is a class method reading cls.registry. Factory warns that this form can mislead: Aluminum.create("Paper", 1.0) is legal and returns a Paper. The lookup is safe here because every subclass writes to Trash.registry and none defines a registry of its own, so cls.registry always resolves to that one table. Call it as Trash.create().

@dataclass builds __init__() from the bare weight: float annotation alone: the two ClassVar attributes belong to the class, so they stay out of it (Data Classes as Types). Each subclass’s value = ... line creates a class attribute of its own, separate from Trash.value and from its siblings’. The subclasses omit the annotation because the name and its type carry over from the base declaration; restating ClassVar[float] would also keep the type checker’s guard on the override (Class Attributes).

Adding a new recyclable type is a single class definition. It registers itself, and create() builds it. sum_value() is an ordinary function. It reads t.value and t.weight polymorphically, and never asks what type a piece is.

The tests confirm that each subclass registers itself, create() builds one by name, and sum_value() totals weight times the per-pound value:

# test_trash.py
import pytest
from trash import Aluminum, Paper, Trash, sum_value

def test_subclasses_self_register() -> None:
    assert set(Trash.registry) == {
        "Aluminum", "Paper", "Glass", "Cardboard"}

def test_create_builds_by_name() -> None:
    t = Trash.create("Aluminum", 2.0)
    assert isinstance(t, Aluminum)
    assert t.weight == 2.0

def test_sum_value_totals_weight_times_value() -> None:
    items: list[Trash] = [Aluminum(2.0), Paper(5.0)]
    # 2*1.67 + 5*0.10
    assert sum_value(items) == pytest.approx(3.84)

A data file describes the trash to process, one Name:weight line per piece:

# trash.dat
Glass:54
Paper:22
Paper:11
Glass:17
Aluminum:89
Paper:88
Aluminum:76
Cardboard:96
Aluminum:25
Aluminum:34
Glass:11
Glass:68
Glass:43
Aluminum:27
Cardboard:44
Aluminum:18
Paper:91
Glass:63
Glass:50
Glass:80
Aluminum:81
Cardboard:12

Parsing it into Trash objects goes through the registry, so the parser never mentions a concrete material. If you add a new kind of trash, the parser keeps working unchanged:

# parse_trash.py
from pathlib import Path
from trash import Trash

def parse(filename: str | Path) -> list[Trash]:
    items: list[Trash] = []
    for line in Path(filename).read_text().splitlines():
        line = line.strip()
        if not line or line.startswith("#"):
            continue
        name, weight = line.split(":")
        items.append(
            Trash.create(name.strip(), float(weight)))
    return items

The test parses a small temporary file, so it does not depend on trash.dat:

# test_parse_trash.py
from pathlib import Path
from parse_trash import parse

def test_parse_reads_and_skips_comments(
    tmp_path: Path,
) -> None:
    data = tmp_path / "trash.dat"
    data.write_text("""\
# header
Aluminum:2.0

Glass:3.0
""")
    items = parse(data)
    assert [type(t).__name__ for t in items] == [
        "Aluminum", "Glass"]
    assert items[0].weight == 2.0
    assert items[1].weight == 3.0

The First Cut: Checking Every Type

The most obvious way to sort is to look at each piece and discover its type using match (the rtti in the file name is run-time type identification, the C++ name for discovering a type at runtime):

# recycle_rtti.py
from collections import defaultdict
from parse_trash import parse
from trash import (Aluminum, Bins, Cardboard, Glass,
                   Paper, sum_value)

bins: Bins = defaultdict(list)
for t in parse("trash.dat"):
    match t:
        case Aluminum():
            bins[Aluminum].append(t)
        case Paper():
            bins[Paper].append(t)
        case Glass():
            bins[Glass].append(t)
        case Cardboard():
            bins[Cardboard].append(t)
for kind, items in bins.items():
    print(f"--- {kind.__name__} ---")
    sum_value(items)
#: --- Glass ---
#: Total value = 88.78
#: --- Paper ---
#: Total value = 21.20
#: --- Aluminum ---
#: Total value = 584.50
#: --- Cardboard ---
#: Total value = 120.08

recycle_rtti.py satisfies the requirement, but it has a classic flaw. It tests for every type in the system. When a new material joins the system, say Plastic, you must find every case statement that enumerates specific types. Each one you miss silently drops trash on the floor. Readers of Composite and Interpreter may expect assert_never() to make the type checker catch the missed case. Exhaustiveness checking works on a closed union, and Trash is deliberately open, which is the point of the registry, so assert_never() has nothing to check against here. This match runs over an open set, which Pattern Matching warns against. A sorter over an open set must find the bin without naming any type, and the next section builds one. Testing for one type, or a small subset that needs special handling, is fine. Testing for all of them means you do dispatch’s job by hand. A case _: wildcard could catch what the named cases miss: case _: raise ValueError(f"unsorted {type(t).__name__}") turns the silent drop into a crash. That is worth doing, but it does not remove the flaw. Every new material still means editing this match, where bins[type(t)] needs no edit at all.

That is the argument. Here is the requirement that makes it concrete. The plant starts accepting plastic, which arrives as a new material class and some new lines in the data:

# plastic.dat
Glass:10
Plastic:20
Aluminum:30
Plastic:40
# plastic_dropped.py
from collections import defaultdict
from parse_trash import parse
from trash import (
    Aluminum,
    Bins,
    Cardboard,
    Glass,
    Paper,
    Trash,
    sum_value,
)

class Plastic(Trash):
    value = 0.15

pieces = parse("plastic.dat")
bins: Bins = defaultdict(list)
for t in pieces:
    match t:
        case Aluminum():
            bins[Aluminum].append(t)
        case Paper():
            bins[Paper].append(t)
        case Glass():
            bins[Glass].append(t)
        case Cardboard():
            bins[Cardboard].append(t)
for kind, items in bins.items():
    print(f"--- {kind.__name__} ---")
    sum_value(items)
binned = sum(len(v) for v in bins.values())
print(f"parsed {len(pieces)}, binned {binned}")
#: --- Glass ---
#: Total value = 2.30
#: --- Aluminum ---
#: Total value = 50.10
#: parsed 4, binned 2

Nothing failed. The parser built two Plastic objects, the sorter matched neither, and the report totals the trash it recognized. Two of four pieces reached a bin, and the sixty pounds of plastic never appeared in the totals the plant uses. “Silently drop trash on the floor” means a number that is wrong and looks right, not an exception to debug. The leak is in the match. The registry accepted Plastic the moment its class statement ran, and without that class statement, create() would have raised a KeyError at the first Plastic: line, loudly, at parse time. The match alone loses trash silently.

Let a Dictionary Do the Sorting

You can use a dictionary keyed by type:

# recycle_dict.py
from collections import defaultdict
from parse_trash import parse
from trash import Bins, sum_value

bins: Bins = defaultdict(list)

for t in parse("trash.dat"):
    bins[type(t)].append(t)  # Bin chosen by the trash piece

for kind, items in bins.items():
    print(f"--- {kind.__name__} ---")
    sum_value(items)
#: --- Glass ---
#: Total value = 88.78
#: --- Paper ---
#: Total value = 21.20
#: --- Aluminum ---
#: Total value = 584.50
#: --- Cardboard ---
#: Total value = 120.08

type(t) is the perfect key because it adapts to new types, including ones added at runtime. Nothing needs maintaining, and nothing gets forgotten. The key is the exact class. That is the same dictionary-probe dispatch as the tables in State Machines and Multiple Dispatching, and it first appeared in Function Objects’s event bus. If you derive CrushedAluminum from Aluminum, it sorts into its own bin rather than its parent’s: usually what a sorter needs, but keep it in mind before you subclass a material. Subclasses are another place where the two sorters disagree: case Aluminum() matches any subclass, so recycle_rtti.py files a CrushedAluminum under Aluminum. Swapping the match for the dictionary is a redesign, not a rename.

The defaultdict(list) creates a bin the first time a material turns up. Bins is an alias for a plain dict, so a type checker accepts bins: Bins = {} too, and that version raises a KeyError on the first piece of trash.

Point this sorter at plastic.dat, the file that defeated the match in plastic_dropped.py. The listing defines Plastic the same way plastic_dropped.py did:

# recycle_dict_plastic.py
from collections import defaultdict
from parse_trash import parse
from trash import Bins, Trash, sum_value

class Plastic(Trash):
    value = 0.15

pieces = parse("plastic.dat")
bins: Bins = defaultdict(list)
for t in pieces:
    bins[type(t)].append(t)  # Bin chosen by the trash piece

for kind, items in bins.items():
    print(f"--- {kind.__name__} ---")
    sum_value(items)
binned = sum(len(v) for v in bins.values())
print(f"parsed {len(pieces)}, binned {binned}")
#: --- Glass ---
#: Total value = 2.30
#: --- Plastic ---
#: Total value = 9.00
#: --- Aluminum ---
#: Total value = 50.10
#: parsed 4, binned 4

Every piece reaches a bin, plastic included: parsed 4, binned 4. Defining Plastic and naming the new data file are the only changes to the program’s logic. The sorting loop needed no edit, unlike the match in recycle_rtti.py and plastic_dropped.py.

Adding Operations: Visitor, and Why Python Skips It

So far the chapter has made new types cheap. The other axis of change is adding new operations, and a design that makes new types cheap ordinarily makes new operations expensive: that trade is the expression problem from Pattern Matching.

Here is the requirement that makes the second axis concrete. The plant already prints a recycling instruction for each material. Now the safety officer wants a disposal hazard printed beside it. That is a second operation that varies by material, and the obvious home for it is a method on each material class:

# note_methods.py
from dataclasses import dataclass
from typing import ClassVar

@dataclass(frozen=True)
class Trash:
    weight: float
    value: ClassVar[float] = 0.0

    def note(self) -> str:
        return f"{type(self).__name__}: nothing special"

    # New requirement, so a new method here
    def hazard(self) -> str:
        return "none"

class Aluminum(Trash):
    value = 1.67

    def note(self) -> str:
        return "Aluminum: crush and bale"

    def hazard(self) -> str:
        return "sharp edges"

class Glass(Trash):
    value = 0.23

    def note(self) -> str:
        return "Glass: sort by color, then crush"

    def hazard(self) -> str:
        return "sharp edges"

class Cardboard(Trash):
    value = 0.79

    def note(self) -> str:
        return "Cardboard: flatten and bundle"

    def hazard(self) -> str:
        return "none"

materials = [Aluminum, Glass, Cardboard]
for cls in materials:
    t = cls(1.0)
    print(f"{t.note()} | hazard: {t.hazard()}")
edited = [c for c in materials if "hazard" in c.__dict__]
print(f"classes edited for one operation: {len(edited)}")
#: Aluminum: crush and bale | hazard: sharp edges
#: Glass: sort by color, then crush | hazard: sharp edges
#: Cardboard: flatten and bundle | hazard: none
#: classes edited for one operation: 3

Both operations answer correctly, and the cost is the last line. One new question cost an edit to all three material classes, and the question after it costs three more edits. Those edits sit in each class body, as note_methods.py shows; in the real program they would go in trash.py. A method belongs in the body of its own class by design: you can assign a function onto a class from outside, but behavior scattered that way is unmaintainable. A plant that buys its material classes from a supplier has no class body to edit.

The method form is not a strawman. This hierarchy is small and the book owns every subclass, so note() on each material is a real option here. The method wins while you own the hierarchy and the operations stay few: each subclass answers for itself, with no separate table to keep in step with the class list. It loses once you do not own the hierarchy, or once operations start to outnumber materials.

Visitor is the classic escape, and it is elaborate: a visitor class, an accept() method on every element, and double dispatch to reach the right overload, all to work around a language that cannot add a method to a class from outside. functools.singledispatch reaches the same implementation in one call, and any module can register an implementation for a new type.

In Python, a single-dispatch function implements Visitor:

# recycling_note.py
from functools import singledispatch
from trash import Aluminum, Cardboard, Glass, Trash

@singledispatch
def recycling_note(t: Trash) -> str:
    return f"{type(t).__name__}: no special handling"

@recycling_note.register
def _(t: Aluminum) -> str:
    return "Aluminum: crush and bale"

@recycling_note.register
def _(t: Glass) -> str:
    return "Glass: sort by color, then crush"

@recycling_note.register
def _(t: Cardboard) -> str:
    return "Cardboard: flatten and bundle"

for cls in Trash.registry.values():
    print(recycling_note(cls(1.0)))
#: Aluminum: crush and bale
#: Paper: no special handling
#: Glass: sort by color, then crush
#: Cardboard: flatten and bundle

Each implementation above takes the name _. Visitor explains that placeholder. recycling_note() is a new operation that lives outside the Trash hierarchy. Paper has no registered note, so it falls through to the base function. That fallback is also the risk: a material nobody registers gets the default answer, with no exception at runtime and no complaint from the type checker. Here “no special handling” is a genuine answer, so the fallback earns its keep. When no default makes sense, the Visitor chapter advises making the base function raise NotImplementedError, so a forgotten registration fails at the first call.

Now give the safety officer’s question the same treatment. It arrives as its own file, and edits no material class:

# disposal_hazard.py
from functools import singledispatch
from trash import Aluminum, Glass, Trash

@singledispatch
def hazard(t: Trash) -> str:
    return "none"

@hazard.register
def _(t: Aluminum) -> str:
    return "sharp edges"

@hazard.register
def _(t: Glass) -> str:
    return "sharp edges"

for cls in Trash.registry.values():
    print(f"{cls.__name__}: {hazard(cls(1.0))}")
edited = [c for c in Trash.registry.values()
          if "hazard" in c.__dict__]
print(f"classes edited for one operation: {len(edited)}")
#: Aluminum: sharp edges
#: Paper: none
#: Glass: sharp edges
#: Cardboard: none
#: classes edited for one operation: 0

The counter reads zero. hazard() reaches every material through the registry, and trash.py is the file that did not change. A third question and a fourth cost one more file each, where note_methods.py charges one edit per material every time. Adding a Plastic material means defining the class, plus one registration for each operation that must answer differently for plastic. Python still has the expression problem, but both sides now cost a line instead of an edit spread across classes.

singledispatch is for behavior that differs by type. The earlier sum_value() does the same thing for every type, so it stays an ordinary function. For an operation that belongs on an object and still varies by type, functools.singledispatchmethod provides the same dispatch in method form.

The chapter now holds two kinds of dispatch that disagree about subclasses. bins[type(t)] keys on the exact class, so a CrushedAluminum derived from Aluminum gets a bin of its own. singledispatch resolves through the MRO, so that same piece answers with Aluminum’s note. Each is right for its job. Multiple Dispatching draws the same distinction between a table keyed by class and dispatch that follows inheritance.

Choosing the Lightest Construct

Design patterns are about separating things that change from things that stay the same. Polymorphism is one way to do that, but not the only one. The deeper skill is spotting the vector of change (Design Patterns), here new types versus new operations, and choosing the lightest construct that isolates it. This chapter met each vector through a concrete requirement: plastic for new types, and the disposal hazard for new operations. Each vector costs a single line at the point of use: bins[type(t)] absorbs a new material, and one @recycling_note.register teaches an existing operation about it. Neither is a pattern in the GoF sense. In Python the lightest construct is often a language feature, not a multi-class pattern. A pattern is worth keeping only when it is still useful once the language does part of the work.

Exercises

  1. Add a Plastic material to trash.py, then point recycle_dict.py at plastic.dat and run it. Confirm that its sorting loop and parse_trash.py need no other changes, then account for every pound of plastic that plastic_dropped.py loses. Which test in test_trash.py fails, and why is that failure correct?
  2. Write a price() operation as a function over a list of Trash, and a heaviest() operation that returns the single heaviest piece. Decide for each whether it needs singledispatch.
  3. Replace the recycling_note() single-dispatch function with a singledispatchmethod on a Sorter class, and explain what changed.
  4. Derive CrushedAluminum from Aluminum, add it to the data recycle_dict.py reads, then run that and recycling_note.py. Explain why it gets its own bin but not its own note. Then change recycle_dict.py so a subclass shares its parent’s bin, without naming any material in the sorting loop.