Index
Contents
Chapter 45 · Solutions

Generators: Solutions

1. tally(), driven with next() and send()

Write tally(), a generator that yields a prompt string, receives an int for each prompt, and returns the total once it has three. Give it the full three-parameter annotation, then drive it with next() and send() and read the total off StopIteration.

Where to look

Annotating a Generator describes the three parameters of Generator: what it yields, what it receives, and what it returns. Give each channel its own type so that a swap is a type error. After the third send(), the generator finishes. The total is the value of the StopIteration that send() raises.

The shape
# The shape of exercise_1.py
from collections.abc import Generator
from typing import NewType

Prompt = NewType("Prompt", str)
Amount = NewType("Amount", int)
Total = NewType("Total", int)

def tally() -> Generator[Prompt, Amount, Total]:
    ...
Done when

your script prints the three prompts and then the sum of your three sent amounts, read from the StopIteration’s value, and uv run ty check passes. If you swap the second and third type parameters of your annotation, the type checker reports errors on your send() calls.

Solution

If you read the total as the result of the last send(), writing total: Total = t.send(Amount(12)) with no try, the script prints the three prompts and then stops with StopIteration: 42. The type checker rejects that line before it runs. send() returns the YieldType, so ty reports an invalid-assignment of a Prompt to a Total. A generator’s return value arrives only on its StopIteration, so the solution catches that exception and reads its value.

# exercise_1.py
from collections.abc import Generator
from typing import NewType

Prompt = NewType("Prompt", str)
Amount = NewType("Amount", int)
Total = NewType("Total", int)

def tally() -> Generator[Prompt, Amount, Total]:
    total = 0
    for n in (1, 2, 3):
        amount: Amount = yield Prompt(f"amount {n} of 3")
        total += amount
    return Total(total)

t = tally()
print(next(t))
#: amount 1 of 3
print(t.send(Amount(10)))
#: amount 2 of 3
print(t.send(Amount(20)))
#: amount 3 of 3
try:
    t.send(Amount(12))
except StopIteration as stop:
    total: Total = stop.value
print(total)
#: 42

Keep the three channels apart. Three NewType definitions over str, int, and int keep the two integer channels apart, so transposing the SendType and the ReturnType is a type checker error. With bare int on both channels, the checker accepts the swap.

Drive the conversation with next() and send(). The third send() resumes the last yield, so the loop ends and tally() returns.

total is a local of the generator, and it keeps its value across three suspensions because the suspended generator preserves its locals. Those locals are the state.

Before you move on, name what separates your attempt from this one: a different design, a detail, or an equivalent alternative.

2. A driver that answers from an iterator

drive() answers from a dict. Write a second driver that answers from an Iterator[Answer], in order, and run interview() under both. Explain what, if anything, needed to change in interview(), and why. Give your driver fewer answers than questions and say what it returns. StopIteration now means two different things in the same loop. Keep them apart.

Where to look

A Generator Waits for Its Driver explains why interview() accepts answers from any driver. Write the new driver as a loop that fetches the next answer, then calls send(). Both the answer iterator and the generator signal the end with StopIteration, so choose carefully which call sits inside the try.

The shape
# The shape of exercise_2.py
from collections.abc import Generator, Iterator
from typing import NewType

Question = NewType("Question", str)
Answer = NewType("Answer", str)
Result = NewType("Result", str)

def interview() -> Generator[Question, Answer, Result]:
    ...

def drive_from_dict(
        conversation: Generator[Question, Answer, Result],
        answers: dict[Question, Answer]) -> Result:
    ...

def drive_naive(
        conversation: Generator[Question, Answer, Result],
        answers: Iterator[Answer]) -> Result:
    ...

def drive_in_order(
        conversation: Generator[Question, Answer, Result],
        answers: Iterator[Answer]) -> Result:
    ...
Done when

interview() runs the same under both drivers, and each driver returns Alice of Wonderland, friend Rabbit from the chapter’s three answers. Given one answer for three questions, your iterator driver passes the answers’ StopIteration on to the caller instead of returning None.

Solution

If you put next(answers) inside the try, as drive_naive() does, a short answer source ends the run quietly. Given one answer for three questions, drive_naive() returns None. The type checker accepts drive_naive(). The except StopIteration meant for the conversation catches the one from the exhausted answers and treats it as a finished interview. The solution’s drive_in_order() fetches each reply outside the try, so the StopIteration from the answers reaches the caller.

# exercise_2.py
from collections.abc import Generator, Iterator
from typing import NewType

Question = NewType("Question", str)
Answer = NewType("Answer", str)
Result = NewType("Result", str)

def interview() -> Generator[Question, Answer, Result]:
    name: Answer = yield Question("name")
    town: Answer = yield Question("town")
    friend: Answer = yield Question("friend")
    return Result(f"{name} of {town}, friend {friend}")

def drive_from_dict(
        conversation: Generator[Question, Answer, Result],
        answers: dict[Question, Answer]) -> Result:
    try:
        request = next(conversation)
    except StopIteration as stop:
        return stop.value
    while True:
        answer = answers[request]
        print(f"{request = }, {answer = }")
        try:
            request = conversation.send(answer)
        except StopIteration as stop:
            return stop.value

def drive_naive(
        conversation: Generator[Question, Answer, Result],
        answers: Iterator[Answer]) -> Result:
    try:
        request = next(conversation)
    except StopIteration as stop:
        return stop.value
    while True:
        try:  # next() and send() share one except clause
            reply = next(answers)
            print(f"{request = }, {reply = }")
            request = conversation.send(reply)
        except StopIteration as stop:
            return stop.value

def drive_in_order(
        conversation: Generator[Question, Answer, Result],
        answers: Iterator[Answer]) -> Result:
    try:
        request = next(conversation)
    except StopIteration as stop:
        return stop.value
    while True:
        # Fetched outside the try on purpose
        reply = next(answers)
        print(f"{request = }, {reply = }")
        try:
            request = conversation.send(reply)
        except StopIteration as stop:
            return stop.value

by_name = {
    Question("name"): Answer("Alice"),
    Question("town"): Answer("Wonderland"),
    Question("friend"): Answer("Rabbit"),
}
print(drive_from_dict(interview(), by_name))
#: request = 'name', answer = 'Alice'
#: request = 'town', answer = 'Wonderland'
#: request = 'friend', answer = 'Rabbit'
#: Alice of Wonderland, friend Rabbit
in_order = iter([Answer("Alice"), Answer("Wonderland"),
                 Answer("Rabbit")])
print(drive_in_order(interview(), in_order))
#: request = 'name', reply = 'Alice'
#: request = 'town', reply = 'Wonderland'
#: request = 'friend', reply = 'Rabbit'
#: Alice of Wonderland, friend Rabbit
# One answer, three questions:
try:
    drive_in_order(interview(), iter([Answer("Alice")]))
except StopIteration:
    print("answer source ran out")
#: request = 'name', reply = 'Alice'
#: answer source ran out
print(repr(drive_naive(interview(),
                       iter([Answer("Alice")]))))
#: request = 'name', reply = 'Alice'
#: None

Reuse the generator unchanged. interview() stays as it was, because it yields a Question and receives an Answer, from either driver. The generator describes the conversation. The driver interprets it. Swapping one interpreter for another leaves the description untouched.

Find the answer for each request. The two drivers differ in the property on which they rely. The dictionary driver looks up each answer by the request, so it answers correctly when the questions arrive in any order, and it answers a repeated question the same way twice. The iterator driver goes by position, so it depends on the generator asking its questions in the order the driver holds the replies. The conversation type the two drivers share declares that a Question goes out and an Answer comes back, and leaves open how the driver chooses each answer.

Keep the two endings apart. In drive_in_order(), next(answers) sits outside the try, as its comment says, because the except StopIteration meant for the conversation otherwise catches a StopIteration raised by an exhausted answer list. Two iterators that raise the same exception type are a hazard for a driver that holds both.

Test a short answer source. The last two runs show that hazard.

None is wrong twice over. The interview stopped after one answer, waiting at its second question, before it built a Result, and None is not a Result in any case.

The type checker accepts return stop.value as a declared Result, because StopIteration.value has type Any. The runtime accepts it too. The program first fails later, when code that expects a string receives that None, far from the driver that returned it.

Keeping the two meanings apart is a one-line discipline: put inside the try only the call whose StopIteration you mean to interpret.

Before you move on, name what separates your attempt from this one: a different design, a detail, or an equivalent alternative.

3. A third delegation in yield_from_send.py

Predict the output of yield_from_send.py after adding a third yield from collect("gamma") to both(), adding its sum to the total, and extending the loop to [1, 2, 3, 4, 5, 6]. Write down the sequence of printed lines before running it.

Where to look

The Send Channel follows each sent value to the delegated generator suspended at that moment. Count how many values one collect() consumes, then count how many the loop supplies in total. Write the printed lines before you run the script, and note which sends produce one line and which produce two.

The shape
# The shape of exercise_3.py
from collections.abc import Generator
from typing import NewType

Prompt = NewType("Prompt", str)
type Value = int
Total = NewType("Total", int)

def collect(name: str) -> Generator[Prompt, Value, Total]:
    ...

def both() -> Generator[Prompt, Value]:
    ...
Done when

you wrote the full sequence of lines before running the modified yield_from_send.py, and it matches the run line for line, including which sends print two lines and what send(6) prints.

Solution
# exercise_3.py
from collections.abc import Generator
from typing import NewType

Prompt = NewType("Prompt", str)
type Value = int
Total = NewType("Total", int)

def collect(name: str) -> Generator[Prompt, Value, Total]:
    first: Value = yield Prompt(f"{name} needs a value")
    second: Value = yield Prompt(f"{name} needs another")
    print(f"{name} got {first} and {second}")
    return Total(first + second)

def both() -> Generator[Prompt, Value]:
    alpha: Total = yield from collect("alpha")
    beta: Total = yield from collect("beta")
    gamma: Total = yield from collect("gamma")
    yield Prompt(f"total {alpha + beta + gamma}")

g = both()
print(next(g))
#: alpha needs a value
for value in [1, 2, 3, 4, 5, 6]:
    print(f"g.send({value}) -> {g.send(value)}")
#: g.send(1) -> alpha needs another
#: alpha got 1 and 2
#: g.send(2) -> beta needs a value
#: g.send(3) -> beta needs another
#: beta got 3 and 4
#: g.send(4) -> gamma needs a value
#: g.send(5) -> gamma needs another
#: gamma got 5 and 6
#: g.send(6) -> total 21

Count the values the collectors need. Each collect() consumes two values, so the three collectors need six, and the loop supplies six. The yield in both() gives send(6) a value to return, so the loop ends with both() suspended and no StopIteration.

Read the output in pairs. The two-line responses mark where one collector ends and the next begins. A send that supplies a collector’s first value produces one line: that collector’s second prompt. A send that supplies a collector’s second value produces two: the completed collector’s print(), then the next prompt.

yield from relays in both directions on its own: prompts up to the driver, numbers down to the suspended yield inside the current collect().

Before you move on, name what separates your attempt from this one: a different design, a detail, or an equivalent alternative.

4. Removing the yield from

Remove yield from in yield_from_nested.py, leaving profile: Result = interview(). Run ty check and the script, and explain both results. Which one told you more, and what does the type checker say once you remove profile’s annotation?

Where to look

Relaying, Not Answering shows survey() delegating to interview() with yield from. Without yield from, the call to interview() builds a generator object, and that object sits suspended before its first line. Compare the declared type of profile with what the call produces, then compare the type checker’s report with the script’s output.

Done when

your explanation accounts for both results: ty reports an invalid-assignment on the profile line, and the script runs and prints <generator object interview at ...>, color blue. It says which result locates the mistake and why, and you have confirmed that ty check passes once the annotation is gone.

Solution
def survey() -> Generator[Question, Answer, Result]:
    profile: Result = interview()
    color: Answer = yield from ask(Question("color"))
    return Result(f"{profile}, color {color}")

ty rejects it:

error[invalid-assignment]: Object of type
`Generator[Question, Answer, Result]` is not assignable to `Result`
 --> yield_from_nested.py:8:23
  |
8 |     profile: Result = interview()
  |              ------   ^^^^^^^^^^^ Incompatible value of type
  |              |        `Generator[Question, Answer, Result]`
  |              Declared type

The script still runs, and produces:

request = 'color', answer = 'blue'
<generator object interview at 0x000001C5FB3FDE40>, color blue

Both results describe the same mistake. Calling a generator function produces a description rather than a conversation. The call interview() returns a generator object, suspended before the first line of its body. The generator stays suspended for the rest of the run, so color is the one request drive() receives. The return statement in survey() interpolates the object’s repr into the sentence where the interview’s Result belonged.

The type checker told you more. Its message names the two types and points at the assignment that mismatches them. That assignment is the defect. The runtime output shows a consequence two lines later, in the return statement, where the f-string formats the object. A reader must work backward from a <generator object ...> in a report to the missing yield from.

The runtime failure is quiet: no exception, an exit code of zero, and output that a log scraper accepts. When Python reclaims an unawaited coroutine, it issues a RuntimeWarning, as Concurrency shows. Python reclaims an unused generator silently.

Once you remove the annotation, the type checker accepts survey(). For profile = interview() it infers the expression’s own type, Generator[Question, Answer, Result], and no declared type contradicts that inference. The f-string then accepts any object, since formatting one calls str() on it. A generator object’s str() is its repr, so the output shows the repr.

profile’s annotation alone gives the checker a declared type to compare with the call’s type. So annotate a local whenever you want the checker to verify the type of the call that produces its value.

Before you move on, name what separates your attempt from this one: a different design, a detail, or an equivalent alternative.

5. A generic sequence()

Rewrite sequence() in yield_from_arguments.py as a generic function, sequence[R](*steps: Generator[Question, Answer, R]), returning Generator[Question, Answer, list[R]]. Run ty check and the script, and explain why drive() rejects the call while the script runs. Then write a wrapper that joins the list into a Result, so drive() stays unchanged, and say what the type checker infers for R at each call.

Where to look

A Generator in the Middle shows sequence() collecting each step’s return value into a list[str] and joining the list. With a type parameter in the ReturnType position, each step returns a type of its own, and yield from evaluates to that type. Compare what drive() declares for its conversation parameter with the type of the generic call.

The shape
# The shape of exercise_5.py
from collections.abc import Generator
from typing import NewType

Question = NewType("Question", str)
Answer = NewType("Answer", str)
Result = NewType("Result", str)

def ask(
    question: Question
) -> Generator[Question, Answer, Answer]:
    ...

def interview() -> Generator[Question, Answer, Result]:
    ...

def drive(conversation: Generator[Question, Answer, Result],
          answers: dict[Question, Answer]) -> Result:
    ...

def sequence[R](
    *steps: Generator[Question, Answer, R]
) -> Generator[Question, Answer, list[R]]:
    ...

def joined(
    *steps: Generator[Question, Answer, str]
) -> Generator[Question, Answer, Result]:
    ...
Done when

ty reports an invalid-argument-type on the drive() call and names list[Result | Answer] as the return type it found, the script still runs and prints a list, and your wrapper checks clean under ty and prints a joined Result.

Solution

The generic version keeps the loop and returns the list:

def sequence[R](
    *steps: Generator[Question, Answer, R]
) -> Generator[Question, Answer, list[R]]:
    results: list[R] = []
    for step in steps:
        result = yield from step
        results.append(result)
    return results

ty rejects the drive() call:

error[invalid-argument-type]: Argument to function `drive`
is incorrect
  --> yield_from_arguments.py:17:5
   |
17 |     sequence(interview(), ask(Question("color"))),
   |     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Expected `Generator[Question, Answer, Result]`, found
`Generator[Question, Answer, list[Result | Answer]]`

The checker solves R from both arguments at once. interview() returns Result and ask() returns Answer, so R is their union, Result | Answer, and the call’s type carries list[Result | Answer] in the return position. drive() declares Generator[Question, Answer, Result] for its conversation, and it ends by returning stop.value as its Result. A list is not a Result, so the argument fails where drive() would hand the list to its caller.

Python ignores the annotations, so the script runs to the end and print() shows the list:

request = 'name', answer = 'Alice'
request = 'town', answer = 'Wonderland'
request = 'friend', answer = 'Rabbit'
request = 'color', answer = 'blue'
['Alice of Wonderland, friend Rabbit', 'blue']

The wrapper delegates to the generic sequence() and joins what comes back, so drive() sees the Result it declares. This file carries its own ask(), interview(), and drive(), since a solution cannot import the chapter’s listings:

# exercise_5.py
from collections.abc import Generator
from typing import NewType

Question = NewType("Question", str)
Answer = NewType("Answer", str)
Result = NewType("Result", str)

def ask(
    question: Question
) -> Generator[Question, Answer, Answer]:
    answer: Answer = yield question
    return answer

def interview() -> Generator[Question, Answer, Result]:
    name: Answer = yield from ask(Question("name"))
    town: Answer = yield from ask(Question("town"))
    return Result(f"{name} of {town}")

def drive(conversation: Generator[Question, Answer, Result],
          answers: dict[Question, Answer]) -> Result:
    try:
        request: Question = next(conversation)
    except StopIteration as stop:
        return stop.value
    while True:
        answer = answers[request]
        print(f"{request = }, {answer = }")
        try:
            request = conversation.send(answer)
        except StopIteration as stop:
            return stop.value

def sequence[R](
    *steps: Generator[Question, Answer, R]
) -> Generator[Question, Answer, list[R]]:
    results: list[R] = []
    for step in steps:
        result = yield from step
        results.append(result)
    return results

def joined(
    *steps: Generator[Question, Answer, str]
) -> Generator[Question, Answer, Result]:
    results = yield from sequence(*steps)
    return Result(", ".join(results))

answers = {
    Question("name"): Answer("Alice"),
    Question("town"): Answer("Wonderland"),
    Question("color"): Answer("blue"),
}
print(drive(
    joined(interview(), ask(Question("color"))), answers
))
#: request = 'name', answer = 'Alice'
#: request = 'town', answer = 'Wonderland'
#: request = 'color', answer = 'blue'
#: Alice of Wonderland, blue

The checker solves R at each call from the argument types. Inside joined(), the call sequence(*steps) sees steps through its annotation, Generator[Question, Answer, str], so R is str there and results is a list[str], which ", ".join() accepts. At the call joined(interview(), ask(Question("color"))), the checker matches interview()’s Generator[Question, Answer, Result] against Generator[Question, Answer, str]. Generator is covariant in its return type and Result is a str, so the match succeeds, and ask()’s Answer passes the same way.

The annotations on sequence() and joined() do different jobs. sequence()’s type parameter lets it collect any return type, so one definition serves steps of every return type. joined()’s str annotation makes the join type-check, and it turns Result | Answer into str before sequence() sees the steps.

Before you move on, name what separates your attempt from this one: a different design, a detail, or an equivalent alternative.

6. report() with a return value

report() in yield_from_return.py yields its lines and returns None. Rewrite it to return the character count as well, and give it the full annotation. Then write a caller that delegates to it with yield from and yields that count in a line of its own, and say which type parameter carries each of the two values.

Where to look

The Return Channel shows how yield from delivers a delegate’s return value to the delegating generator. Iterator[str] cannot declare a return value, so report() needs the full Generator annotation. The caller binds the count with yield from, then yields it as a string of its own.

The shape
# The shape of exercise_6.py
from collections.abc import Generator
from typing import NewType

Size = NewType("Size", int)

def emit(items: list[str]) -> Generator[str, None, Size]:
    ...

def report(items: list[str]) -> Generator[str, None, Size]:
    ...

def summarize(items: list[str]) -> Generator[str]:
    ...
Done when

uv run ty check passes, and your caller’s output for the chapter’s ["red", "green", "blue"] ends with a line carrying 12. With report()’s annotation changed back to Iterator[str], ty reports invalid-explicit-return, and you can name the type parameter through which each of your two values travels.

Solution

The solution returns the count from report(), so the count reaches summarize() through a second return:

# exercise_6.py
from collections.abc import Generator
from typing import NewType

Size = NewType("Size", int)

def emit(items: list[str]) -> Generator[str, None, Size]:
    total = 0
    for item in items:
        yield item
        total += len(item)
    return Size(total)

def report(items: list[str]) -> Generator[str, None, Size]:
    size: Size = yield from emit(items)
    yield f"{size} characters"
    return size

def summarize(items: list[str]) -> Generator[str]:
    counted: Size = yield from report(items)
    yield f"total: {counted}"

print(list(summarize(["red", "green", "blue"])))
#: ['red', 'green', 'blue', '12 characters', 'total: 12']

Declare the return channel. report()’s annotation changes from Iterator[str] to Generator[str, None, Size], because a generator that returns something needs the long form. Under Iterator[str] a generator must return None, so ty rejects report()’s return size with an invalid-explicit-return: expected None, found Size.

If you change report()’s annotation but leave out return size, the script still runs, and the printed list ends with 'total: None'. report() returns None, and the yield from in summarize() delivers that None into counted. ty reports an invalid-implicit-return on the annotation, because report() declares a Size and, with no return statement, finishes with None.

Keep the strings and the count apart. The strings and the count travel by different channels. Every string that emit() or report() yields travels through the YieldType and comes out in the list. The count travels through the ReturnType: emit() returns it, yield from delivers it into report()’s size, report() returns it again, and the second yield from delivers it into summarize()’s counted. The SendType is None throughout, since list() drives the chain with next(), the runtime equivalent of send(None).

The count 12 therefore reaches summarize() through two returns. '12 characters' takes the other route: report() yields it, the yield from in summarize() relays it, and the driver receives it. The count ends up in size and then in counted; the string ends up in the list that list() builds. A generator can yield a number or return it. The choice decides which code receives the number: a yielded value goes to the driver, and a returned value goes to the delegating generator.

Before you move on, name what separates your attempt from this one: a different design, a detail, or an equivalent alternative.

7. Why a driver primes with next()

Explain why a driver must prime with next() rather than send(None), given that the two are equivalent at runtime. send_none_is_next.py has the answer. State it in terms of the SendType.

Where to look

Annotating a Generator defines the SendType, and send_none_is_next.py calls send(None) on a generator whose SendType is Answer. Read the signature of send() and ask what it accepts for interview(). Then ask what next() accepts, and where a generator that has yet to run would put a sent value.

Done when

your explanation predicts what ty reports if you delete the # type: ignore in send_none_is_next.py (an invalid-argument-type on None), states the parameter type of send() in terms of the SendType, and says why priming with next() type-checks for any generator.

Solution

next(g) and g.send(None) do the same thing at runtime. The SendType tells them apart.

send()’s signature is send(self, value: _SendT_contra, /) -> _YieldT_co, so its parameter type is the generator’s SendType. For interview() that is Answer, and None is not an Answer, so the priming call is a type error. send_none_is_next.py carries a # type: ignore to suppress it. If you remove that comment, the type checker reports:

error[invalid-argument-type]: Argument to bound method
`Generator.send` is incorrect
 --> send_none_is_next.py:4:27
  |
4 | print(f"{interview().send(None) = }")
  |                           ^^^^ Expected `Answer`, found `None`

next() takes the generator and an optional default to return at exhaustion. It asks for the generator’s next yielded value and ignores the SendType, so priming with it type-checks for any generator.

The mismatch is real rather than a type checker limitation. A generator’s SendType describes what a suspended yield expression can receive. A just-started generator waits before the first line of its body rather than at a yield, so the first send() arrives before any yield expression is ready to receive a value. Sending a non-None value to a fresh generator raises TypeError: can't send non-None value to a just-started generator. So at runtime the first send() accepts None alone, a value the SendType excludes.

An annotation cannot express “None for the first call, Answer afterward,” because a single SendType covers every call. Widening the SendType to Answer | None states the exception in the type, but the type checker then applies Answer | None to every yield expression. The generator must handle a None at each yield whose value it uses as an Answer. That None exists only in the type, because the priming None arrives before the generator reaches its first yield. interview() binds each received value to a local annotated Answer, so under the wider SendType the type checker rejects all three assignments as invalid-assignment, and the generator would need a None test at each yield. Priming with next() keeps the SendType at Answer. The one call that cannot carry a value comes from the one function that cannot pass one.

Before you move on, name what separates your attempt from this one: a different design, a detail, or an equivalent alternative.

8. A vending machine as a single generator

A Vending Machine keeps its current state in an attribute and looks up each transition in a table. Write a simplified version as a single generator instead. It collects money, takes two digits, then dispenses or refuses. It yields its current state and receives each event with send(), so the position in the generator’s body carries the state. This generator’s yield runs in the opposite direction from interview()’s yield. Here yield reports the state the machine reached instead of requesting something the machine needs. Say which of the two versions you would rather extend with another state, and why.

Where to look

A Vending Machine holds its state in an attribute and a transition table. Here the position inside the generator’s body is the state, so use loops and isinstance() checks on the event that yield returns. Because yield reports the state reached, the driver’s send() delivers events rather than answers.

The shape
# The shape of exercise_8.py
from collections.abc import Generator
from typing import Final, Literal
from record import record

@record
class Coin:
    cents: int

@record
class Digit:
    value: str

type Event = Coin | Digit
type Report = Literal[
    "QUIESCENT", "COLLECTING", "SELECTING",
    "UNAVAILABLE", "NEED_MONEY", "DISPENSED"]
type Code = str

PRICES: Final[dict[Code, int]] = {"11": 25, "12": 75}
STOCK: Final[dict[Code, int]] = {"11": 0, "12": 3}

def machine() -> Generator[Report, Event]:
    ...
Done when

uv run ty check passes, and one driven run reports COLLECTING after a coin, SELECTING after a first digit, UNAVAILABLE for an item out of stock, a refusal for too little money, and a sale once the money suffices. A second, fresh machine starts with the full stock after the first one sells.

Solution

If you write stock = STOCK instead of copying with dict(STOCK), each sale changes the module-level table from which every machine starts. After three "12" sales, by any mix of machines, a fresh machine() answers the next "12" order with UNAVAILABLE. The Final annotation forbids rebinding the name and permits changing the dictionary, so the type checker passes the shared version. The solution copies STOCK into a local, so each generator holds its own stock.

# exercise_8.py
from collections.abc import Generator
from typing import Final, Literal
from record import record

@record
class Coin:
    cents: int

@record
class Digit:
    value: str

type Event = Coin | Digit
type Report = Literal[
    "QUIESCENT", "COLLECTING", "SELECTING",
    "UNAVAILABLE", "NEED_MONEY", "DISPENSED"]
type Code = str

PRICES: Final[dict[Code, int]] = {"11": 25, "12": 75}
STOCK: Final[dict[Code, int]] = {"11": 0, "12": 3}

def machine() -> Generator[Report, Event]:
    stock = dict(STOCK)
    amount = 0
    event: Event = yield "QUIESCENT"
    while True:
        while isinstance(event, Coin):
            amount += event.cents
            event = yield "COLLECTING"
        row = event.value  # Not a Coin, so a first Digit
        second: Event = yield "SELECTING"
        # A coin instead of a digit
        if isinstance(second, Coin):
            amount += second.cents
            event = yield "COLLECTING"
            continue
        code = row + second.value
        if stock.get(code, 0) == 0:
            event = yield "UNAVAILABLE"
        elif amount < PRICES.get(code, 0):
            event = yield "NEED_MONEY"
        else:
            amount -= PRICES[code]
            stock[code] -= 1
            event = yield "DISPENSED"

m = machine()
print(next(m))
#: QUIESCENT
for event in [Coin(25), Digit("1"), Digit("1"), Digit("1"),
              Digit("2"), Coin(50), Digit("1"), Digit("2")]:
    print(f"{event} -> {m.send(event)}")
#: Coin(cents=25) -> COLLECTING
#: Digit(value='1') -> SELECTING
#: Digit(value='1') -> UNAVAILABLE
#: Digit(value='1') -> SELECTING
#: Digit(value='2') -> NEED_MONEY
#: Coin(cents=50) -> COLLECTING
#: Digit(value='1') -> SELECTING
#: Digit(value='2') -> DISPENSED

Let position carry the state. The machine holds no state attribute and consults no transition table. Where the generator pauses is the state. A pause in the coin loop means COLLECTING, and a pause after yield "SELECTING" means a first digit has arrived and the machine waits for a second. amount, row, and stock are locals that keep their values between sends because the suspended generator preserves them.

Report the state reached. The yield in machine() runs the opposite direction from the one in interview(). Neither signature says so. Both yield strings, but interview() yields a request the driver must satisfy, while machine() yields a report the driver may ignore. send(Coin(25)) delivers an event rather than an answer. The COLLECTING it returns reports the state that event produced. A generator’s type declares what each channel carries and leaves open whether a yielded value is a request or a report.

For another state, take the table. The generator stays compact because the states form a line, so control flow can express the sequence. The two states here that break the line need extra control flow. An if chain reaches UNAVAILABLE and NEED_MONEY. From each one, the machine loops back to the top, a goto written as a while True. Suppose you add a state reachable from several others, the way the table handles Quit from every state but QUIESCENT. No position in the body corresponds to that state. The new state becomes a flag, or a check repeated at several yields. Either one breaks the correspondence between position and state that makes this version readable.

With the table, adding any state means one new Enum member and a few new rules. Those rules sit next to the existing ones, where you can read the whole machine at once.

The generator is the better choice for a conversation with a beginning and an end, like interview(). The table is the better choice for a machine that runs forever and can go anywhere from anywhere.

Before you move on, name what separates your attempt from this one: a different design, a detail, or an equivalent alternative.

9. A proxy that relays everything

Replace proxy()’s loop in answering_proxy.py with the single line return (yield from inner) and run drive() over the same known table. Explain from the output why this proxy() cannot answer town, and say what a generator must do to answer one request and relay the rest.

Where to look

Relaying, Not Answering contrasts drive(), which answers a request, with yield from, which relays it. proxy() answers from known in one branch and yields the request in the other, because its own loop sees each request before deciding. Ask where proxy()’s body waits while inner runs under a yield from.

The shape
# The shape of exercise_9.py
from collections.abc import Generator
from typing import NewType

Question = NewType("Question", str)
Answer = NewType("Answer", str)
Result = NewType("Result", str)

def interview() -> Generator[Question, Answer, Result]:
    ...

def drive(conversation: Generator[Question, Answer, Result],
          answers: dict[Question, Answer]) -> Result:
    ...

def proxy(
    inner: Generator[Question, Answer, Result],
    known: dict[Question, Answer],
) -> Generator[Question, Answer, Result]:
    ...
Done when

drive() prints three request lines, with town answered Wonderland rather than Oz, the Result is Alice of Wonderland, friend Rabbit, and your explanation says that proxy()’s body waits inside the yield from until inner finishes.

Solution

The relaying proxy() is short, and this file carries its own interview() and drive():

# exercise_9.py
from collections.abc import Generator
from typing import NewType

Question = NewType("Question", str)
Answer = NewType("Answer", str)
Result = NewType("Result", str)

def interview() -> Generator[Question, Answer, Result]:
    name: Answer = yield Question("name")
    town: Answer = yield Question("town")
    friend: Answer = yield Question("friend")
    return Result(f"{name} of {town}, friend {friend}")

def drive(conversation: Generator[Question, Answer, Result],
          answers: dict[Question, Answer]) -> Result:
    try:
        request: Question = next(conversation)
    except StopIteration as stop:
        return stop.value
    while True:
        answer = answers[request]
        print(f"{request = }, {answer = }")
        try:
            request = conversation.send(answer)
        except StopIteration as stop:
            return stop.value

def proxy(
    inner: Generator[Question, Answer, Result],
    known: dict[Question, Answer],
) -> Generator[Question, Answer, Result]:
    return (yield from inner)

answers = {
    Question("name"): Answer("Alice"),
    Question("town"): Answer("Wonderland"),
    Question("friend"): Answer("Rabbit"),
}
print(drive(
    proxy(interview(), {Question("town"): Answer("Oz")}),
    answers,
))
#: request = 'name', answer = 'Alice'
#: request = 'town', answer = 'Wonderland'
#: request = 'friend', answer = 'Rabbit'
#: Alice of Wonderland, friend Rabbit

proxy() ignores known. yield from inner relays every request inner yields up to drive() and passes every answer back down, and while that happens proxy()’s body waits inside the yield from expression. The body resumes when inner finishes, with inner’s Result as the expression’s value, and by then drive() has answered every request. town reaches drive(), which answers Wonderland, so the Result is the chapter’s unproxied one. yield from relays the request; drive() answers it.

To answer one request and relay the rest, a generator must see each request before it goes up. That means driving inner with next() and send() in a loop of its own, answering from known when it can and yielding the request otherwise, which is the chapter’s proxy(). The chapter’s relaying yield hands one request to the driver and receives one answer; a yield from would hand over the whole conversation.

Before you move on, name what separates your attempt from this one: a different design, a detail, or an equivalent alternative.

10. A tree walk as an iterator class

PEP 255 asks its reader to picture a recursive tree walk as an iterator: “to cast that into an iterator framework requires removing the recursion manually and maintaining the state of the traversal by hand.” Write that iterator for visit() in color_count.py. Paths is an iterator class, with no yield in it, that produces the path lines visit() yields, in the same order, and leaves out the counts. Keep the walk’s position in a list that you use as a stack. Then say what Paths stores that visit() leaves to Python.

Where to look

Half a Coroutine by Design shows Squasher storing its position in a field, because __next__() starts at its first line on every call. A walk in progress inside visit() is several calls deep. Each of those calls has a path prefix and a place in its dictionary’s items, so give each call in progress one entry on the stack. iter(tree.items()) returns an iterator that keeps its place between next() calls.

The shape
# The shape of exercise_10.py
import inspect
from collections.abc import Generator, Iterator
from enum import StrEnum
from types import GeneratorType
from typing import override

class Color(StrEnum):
    RED = "red"
    GREEN = "green"
    BLUE = "blue"

type Name = str
type Path = str
type Tree = dict[Name, Tree | Color]
type Item = tuple[Name, Tree | Color]
type Level = tuple[Path, Iterator[Item]]

def visit(tree: Tree, at: Path = "") -> Generator[str]:
    ...

def level(tree: Tree, at: Path = "") -> Level:
    ...

class Paths(Iterator[str]):
    def __init__(self, tree: Tree) -> None:
        ...

    @override
    def __next__(self) -> str:
        ...

def prefixes(walk: Iterator[str] | None) -> list[Path]:
    ...
Done when

list(Paths(garden)) equals the six path lines that color_count.py prints, a tree with an empty subtree produces the lines of its other entries, Paths contains no yield, and your answer names the two things in each stack entry and says where visit() keeps them.

Solution

This file carries visit() without its counts, to compare with Paths:

# exercise_10.py
import inspect
from collections.abc import Generator, Iterator
from enum import StrEnum
from types import GeneratorType
from typing import override

class Color(StrEnum):
    RED = "red"
    GREEN = "green"
    BLUE = "blue"

type Name = str
type Path = str
type Tree = dict[Name, Tree | Color]
type Item = tuple[Name, Tree | Color]
type Level = tuple[Path, Iterator[Item]]

def visit(tree: Tree, at: Path = "") -> Generator[str]:
    for name, node in tree.items():
        if isinstance(node, dict):
            yield from visit(node, f"{at}{name}/")
        else:
            yield f"{at}{name}: {node}"

def level(tree: Tree, at: Path = "") -> Level:
    return at, iter(tree.items())

class Paths(Iterator[str]):
    def __init__(self, tree: Tree) -> None:
        self.stack: list[Level] = [level(tree)]

    @override
    def __next__(self) -> str:
        while self.stack:
            at, items = self.stack[-1]
            match next(items, None):
                case None:
                    self.stack.pop()  # [1]
                case name, dict() as subtree:
                    self.stack.append(  # [2]
                        level(subtree, f"{at}{name}/"))
                case name, color:
                    return f"{at}{name}: {color}"  # [3]
        raise StopIteration

def prefixes(walk: Iterator[str] | None) -> list[Path]:
    found: list[Path] = []
    while isinstance(walk, GeneratorType):
        found.append(inspect.getgeneratorlocals(walk)["at"])
        walk = walk.gi_yieldfrom
    return found

garden: Tree = {
    "rose": Color.RED,
    "beds": {
        "fern": Color.GREEN,
        "tulip": Color.RED,
        "pond": {"iris": Color.BLUE, "reed": Color.GREEN},
    },
    "ivy": Color.GREEN,
}
print(list(Paths(garden)) == list(visit(garden)))
#: True
print(list(Paths({"bed": {}, "ivy": Color.GREEN})))
#: ['ivy: green']
paths = Paths(garden)
print([next(paths) for _ in range(4)][-1])
#: beds/pond/iris: blue
print([at for at, _ in paths.stack])
#: ['', 'beds/', 'beds/pond/']
walk = visit(garden)
print([next(walk) for _ in range(4)][-1])
#: beds/pond/iris: blue
print(prefixes(walk))
#: ['', 'beds/', 'beds/pond/']

A stack entry is a call in progress. A Level holds the two things one visit() call needs to continue: the prefix at, and an iterator that keeps the call’s place in its dictionary’s items. level() takes the two arguments that visit() takes, so [2] pushes a level where visit() calls itself. [1] pops a level where a visit() call finishes its loop, and [3] returns a line where visit() yields one. For {"bed": {}, "ivy": Color.GREEN}, [2] pushes a level for bed. The next pass finds that level empty and pops it at [1].

__next__() finds its place on every call. visit() continues after its yield when the consumer asks for another line. __next__() starts at its first line, so its while loop reads the top of the stack to find the level where the walk stopped. That level’s iterator supplies the place within the level.

Paths stores the chain of calls. After four lines, with the walk at beds/pond/iris, paths.stack holds three levels: the garden, beds/, and beds/pond/. The generator holds the same three levels, as three suspended visit() generators. Each one waits at its place in the loop over tree.items(), with at among its locals: the outer two at a yield from, the innermost at its yield. A delegating generator’s gi_yieldfrom attribute refers to the generator to which it delegates. prefixes() follows gi_yieldfrom from the outermost visit() to the innermost and reads the same three prefixes.

Python adds a generator to the chain at each yield from and discards that generator when it returns. In Paths you write the chain as a list, with an append() for each call and a pop() for each return. That list is the PEP’s “state of the traversal.”

GoF Design Patterns makes the same observation about an iterator that the client advances over a recursive structure such as a Composite: the iterator “has to store a path” through the structure “to keep track of the current object.” paths.stack is that path.

A chain of generators stops at Python’s recursion limit, as the chapter’s tail_delegation.py shows. On a tree 1,500 levels deep, visit() raises RecursionError. Paths walks that tree, because its stack is a list.

Before you move on, name what separates your attempt from this one: a different design, a detail, or an equivalent alternative.

11. Saga steps as functions

step() in saga.py takes a name and prints its undo. Replace name with two functions: action, which the step calls and whose result it yields, and undo, which the except clause calls. Build one booking of a flight, a hotel, and a card whose three actions succeed, and one whose card action raises Failed. The driver of the second booking calls only next(). Say where Failed starts, and why the hotel and the flight are still undone.

Where to look

Through yield from to the Innermost Generator says that an exception leaving the innermost generator propagates into the generator that delegates to it, as if the yield from expression had raised it. An exception that the innermost generator’s own body raises takes that path too, with no throw(). Ask which generators are waiting at a yield from when the card step calls its action.

The shape
# The shape of exercise_11.py
from collections.abc import Callable, Generator
from exceptions import expect

class Failed(Exception):
    pass

type Action = Callable[[], str]
type Undo = Callable[[], None]

def step(
    action: Action, undo: Undo,
    rest: Generator[str] | None = None,
) -> Generator[str]:
    ...

def doing(name: str) -> Action:
    ...

def undoing(name: str) -> Undo:
    ...

def decline() -> str:
    ...

def book(card: Action) -> Generator[str]:
    ...
Done when

the first booking lists three results, the second prints undo hotel and undo flight before Failed reaches the driver, no line calls throw(), and your answer says which try blocks are open when the card’s action runs.

Solution

doing() and undoing() build a step’s action and its undo from its name. decline() is the card action that fails:

# exercise_11.py
from collections.abc import Callable, Generator
from exceptions import expect

class Failed(Exception):
    pass

type Action = Callable[[], str]
type Undo = Callable[[], None]

def step(
    action: Action, undo: Undo,
    rest: Generator[str] | None = None,
) -> Generator[str]:
    yield action()  # [1]
    if rest is None:
        return
    try:
        yield from rest
    except Failed:
        undo()  # [2]
        raise

def doing(name: str) -> Action:
    return lambda: f"do {name}"

def undoing(name: str) -> Undo:
    return lambda: print(f"undo {name}")

def decline() -> str:
    raise Failed("card declined")

def book(card: Action) -> Generator[str]:
    return step(
        doing("flight"), undoing("flight"),
        step(doing("hotel"), undoing("hotel"),
             step(card, undoing("card"))),
    )

print(list(book(doing("card"))))
#: ['do flight', 'do hotel', 'do card']
booking = book(decline)
print(next(booking))
#: do flight
print(next(booking))
#: do hotel
expect(Failed, next, booking)  # [3]
#: undo hotel
#: undo flight
#: [Failed] card declined

The step calls its action. [1] calls action() and yields the result, where saga.py yields a string built from name. [2] calls undo() where saga.py prints. The try, the yield from, and the bare raise stay as saga.py has them.

Failed starts in the card step’s body. The next() at [3] resumes the flight step, which waits at its yield from. That yield from resumes the hotel step, and the hotel step’s yield from starts the card step. The card step runs [1], and decline() raises Failed there. No try in the card step encloses [1], so the exception leaves the card step. It arrives at the hotel step’s yield from, which sits inside the hotel step’s try. The hotel step’s except clause calls its undo() and raises Failed again, and the flight step’s except clause does the same. Both try blocks are open when the card’s action runs.

throw() raises an exception at a yield for the driver. Here the generator’s own code raises the exception, and it propagates through the two yield from expressions as it does in saga.py. The driver’s part is the next() call, where Failed arrives after both undo lines.

Before you move on, name what separates your attempt from this one: a different design, a detail, or an equivalent alternative.

12. Undoing a step that fails partway

Drive a booking from saga_generators.py to issue ticket and throw Failed there. Say which command the driver has carried out and which undo commands it receives. Then change step() so that a step that fails partway through its do also undoes what it did. Throw Failed at issue ticket and at charge card, and say what an undo must now tolerate.

Where to look

Sagas with Generators puts each step’s try around yield from rest. Find the line where step() is paused when the driver has just received issue ticket, and check whether that line is inside the try. For the change, move one line and remove the early return.

The shape
# The shape of exercise_12.py
from collections.abc import Generator

class Failed(Exception):
    pass

type Commands = Generator[str]

def commands(*names: str) -> Commands:
    ...

def step(
    *, do: Commands, undo: Commands,
    rest: Commands | None = None,
) -> Commands:
    ...

def book() -> Commands:
    ...

def unwind(saga: Commands, error: Failed) -> None:
    ...
Done when

you can say why the chapter’s step() leaves the seat reserved, your step() yields void ticket and release seat for a failure at issue ticket, it yields refund card first for a failure at charge card, and your answer names the undo commands whose actions failed.

Solution

With the chapter’s step(), the driver has carried out reserve seat when it throws Failed at issue ticket. The flight step is paused inside yield from do, which comes before its try. Failed leaves the flight step and reaches the driver with no undo command, and the seat stays reserved.

This step() puts do inside the try:

# exercise_12.py
from collections.abc import Generator

class Failed(Exception):
    pass

type Commands = Generator[str]

def commands(*names: str) -> Commands:
    yield from names

def step(
    *, do: Commands, undo: Commands,
    rest: Commands | None = None,
) -> Commands:
    try:
        yield from do  # [1]
        if rest is not None:
            yield from rest
    except Failed:
        yield from undo
        raise

def book() -> Commands:
    return step(
        do=commands("reserve seat", "issue ticket"),
        undo=commands("void ticket", "release seat"),
        rest=step(
            do=commands("hold room"),
            undo=commands("release room"),
            rest=step(
                do=commands("book taxi"),
                undo=commands("cancel taxi"),
                rest=step(
                    do=commands("charge card"),
                    undo=commands("refund card"),
                ),
            ),
        ),
    )

def unwind(saga: Commands, error: Failed) -> None:
    try:
        print(saga.throw(error))
        for command in saga:
            print(command)
    except Failed as e:
        print(f"Failed: {e}")

booking = book()
print([next(booking) for _ in range(2)][-1])
#: issue ticket
unwind(booking, Failed("printer jammed"))  # [2]
#: void ticket
#: release seat
#: Failed: printer jammed
booking = book()
print([next(booking) for _ in range(5)][-1])
#: charge card
unwind(booking, Failed("card declined"))  # [3]
#: refund card
#: cancel taxi
#: release room
#: void ticket
#: release seat
#: Failed: card declined

Every command of a step is inside its try. [1] moves yield from do into the try, so a Failed thrown at any of a step’s commands reaches that step’s except clause. At [2] the flight step yields void ticket and release seat, and the driver releases the seat.

An undo now runs for an action that failed. At [2], void ticket asks the driver to void a ticket whose printing failed. At [3] the failure is in the card step, and its undo runs first: refund card asks the driver to refund a charge that the bank declined. Each undo command must therefore be safe to carry out when its action did not complete. The chapter’s step() undoes completed steps, which is enough while each do is a single command.

Before you move on, name what separates your attempt from this one: a different design, a detail, or an equivalent alternative.