tally(), driven with next() and
send()Write
tally(), a generator that yields a prompt string, receives anintfor each prompt, and returns the total once it has three. Give it the full three-parameter annotation, then drive it withnext()andsend()and read the total offStopIteration.
Annotating
a Generator describes the three parameters of
Generator: what it yields, what it receives, and
what it returns. Give each channel its own type so that a swap
is a type error. After the third send(), the
generator finishes. The total is the value of the
StopIteration that send() raises.
# The shape of exercise_1.py
from collections.abc import Generator
from typing import NewType
Prompt = NewType("Prompt", str)
Amount = NewType("Amount", int)
Total = NewType("Total", int)
def tally() -> Generator[Prompt, Amount, Total]:
...your script prints the three prompts and then the sum of your
three sent amounts, read from the StopIteration’s
value, and uv run ty check passes. If
you swap the second and third type parameters of your
annotation, the type checker reports errors on your
send() calls.
If you read the total as the result of the last
send(), writing
total: Total = t.send(Amount(12)) with no
try, the script prints the three prompts and then
stops with StopIteration: 42. The type checker
rejects that line before it runs. send() returns
the YieldType, so ty reports an
invalid-assignment of a Prompt to a
Total. A generator’s return value arrives only on
its StopIteration, so the solution catches that
exception and reads its value.
# exercise_1.py
from collections.abc import Generator
from typing import NewType
Prompt = NewType("Prompt", str)
Amount = NewType("Amount", int)
Total = NewType("Total", int)
def tally() -> Generator[Prompt, Amount, Total]:
total = 0
for n in (1, 2, 3):
amount: Amount = yield Prompt(f"amount {n} of 3")
total += amount
return Total(total)
t = tally()
print(next(t))
#: amount 1 of 3
print(t.send(Amount(10)))
#: amount 2 of 3
print(t.send(Amount(20)))
#: amount 3 of 3
try:
t.send(Amount(12))
except StopIteration as stop:
total: Total = stop.value
print(total)
#: 42Keep the three channels apart. Three
NewType definitions over str,
int, and int keep the two integer
channels apart, so transposing the SendType and the
ReturnType is a type checker error. With bare
int on both channels, the checker accepts the
swap.
Drive the conversation with next() and
send(). The third send()
resumes the last yield, so the loop ends and
tally() returns.
total is a local of the generator, and it keeps
its value across three suspensions because the suspended
generator preserves its locals. Those locals are the state.
Before you move on, name what separates your attempt from this one: a different design, a detail, or an equivalent alternative.
drive()answers from adict. Write a second driver that answers from anIterator[Answer], in order, and runinterview()under both. Explain what, if anything, needed to change ininterview(), and why. Give your driver fewer answers than questions and say what it returns.StopIterationnow means two different things in the same loop. Keep them apart.
A
Generator Waits for Its Driver explains why
interview() accepts answers from any driver. Write
the new driver as a loop that fetches the next answer, then
calls send(). Both the answer iterator and the
generator signal the end with StopIteration, so
choose carefully which call sits inside the
try.
# The shape of exercise_2.py
from collections.abc import Generator, Iterator
from typing import NewType
Question = NewType("Question", str)
Answer = NewType("Answer", str)
Result = NewType("Result", str)
def interview() -> Generator[Question, Answer, Result]:
...
def drive_from_dict(
conversation: Generator[Question, Answer, Result],
answers: dict[Question, Answer]) -> Result:
...
def drive_naive(
conversation: Generator[Question, Answer, Result],
answers: Iterator[Answer]) -> Result:
...
def drive_in_order(
conversation: Generator[Question, Answer, Result],
answers: Iterator[Answer]) -> Result:
...interview() runs the same under both drivers,
and each driver returns
Alice of Wonderland, friend Rabbit from the
chapter’s three answers. Given one answer for three questions,
your iterator driver passes the answers’
StopIteration on to the caller instead of returning
None.
If you put next(answers) inside the
try, as drive_naive() does, a short
answer source ends the run quietly. Given one answer for three
questions, drive_naive() returns None.
The type checker accepts drive_naive(). The
except StopIteration meant for the conversation
catches the one from the exhausted answers and treats it as a
finished interview. The solution’s drive_in_order()
fetches each reply outside the try, so the
StopIteration from the answers reaches the
caller.
# exercise_2.py
from collections.abc import Generator, Iterator
from typing import NewType
Question = NewType("Question", str)
Answer = NewType("Answer", str)
Result = NewType("Result", str)
def interview() -> Generator[Question, Answer, Result]:
name: Answer = yield Question("name")
town: Answer = yield Question("town")
friend: Answer = yield Question("friend")
return Result(f"{name} of {town}, friend {friend}")
def drive_from_dict(
conversation: Generator[Question, Answer, Result],
answers: dict[Question, Answer]) -> Result:
try:
request = next(conversation)
except StopIteration as stop:
return stop.value
while True:
answer = answers[request]
print(f"{request = }, {answer = }")
try:
request = conversation.send(answer)
except StopIteration as stop:
return stop.value
def drive_naive(
conversation: Generator[Question, Answer, Result],
answers: Iterator[Answer]) -> Result:
try:
request = next(conversation)
except StopIteration as stop:
return stop.value
while True:
try: # next() and send() share one except clause
reply = next(answers)
print(f"{request = }, {reply = }")
request = conversation.send(reply)
except StopIteration as stop:
return stop.value
def drive_in_order(
conversation: Generator[Question, Answer, Result],
answers: Iterator[Answer]) -> Result:
try:
request = next(conversation)
except StopIteration as stop:
return stop.value
while True:
# Fetched outside the try on purpose
reply = next(answers)
print(f"{request = }, {reply = }")
try:
request = conversation.send(reply)
except StopIteration as stop:
return stop.value
by_name = {
Question("name"): Answer("Alice"),
Question("town"): Answer("Wonderland"),
Question("friend"): Answer("Rabbit"),
}
print(drive_from_dict(interview(), by_name))
#: request = 'name', answer = 'Alice'
#: request = 'town', answer = 'Wonderland'
#: request = 'friend', answer = 'Rabbit'
#: Alice of Wonderland, friend Rabbit
in_order = iter([Answer("Alice"), Answer("Wonderland"),
Answer("Rabbit")])
print(drive_in_order(interview(), in_order))
#: request = 'name', reply = 'Alice'
#: request = 'town', reply = 'Wonderland'
#: request = 'friend', reply = 'Rabbit'
#: Alice of Wonderland, friend Rabbit
# One answer, three questions:
try:
drive_in_order(interview(), iter([Answer("Alice")]))
except StopIteration:
print("answer source ran out")
#: request = 'name', reply = 'Alice'
#: answer source ran out
print(repr(drive_naive(interview(),
iter([Answer("Alice")]))))
#: request = 'name', reply = 'Alice'
#: NoneReuse the generator unchanged.
interview() stays as it was, because it yields a
Question and receives an Answer, from
either driver. The generator describes the conversation. The
driver interprets it. Swapping one interpreter for another
leaves the description untouched.
Find the answer for each request. The two
drivers differ in the property on which they rely. The
dictionary driver looks up each answer by the request, so it
answers correctly when the questions arrive in any order, and it
answers a repeated question the same way twice. The iterator
driver goes by position, so it depends on the generator asking
its questions in the order the driver holds the replies. The
conversation type the two drivers share declares
that a Question goes out and an Answer
comes back, and leaves open how the driver chooses each
answer.
Keep the two endings apart. In
drive_in_order(), next(answers) sits
outside the try, as its comment says, because the
except StopIteration meant for the conversation
otherwise catches a StopIteration raised by an
exhausted answer list. Two iterators that raise the same
exception type are a hazard for a driver that holds both.
Test a short answer source. The last two runs show that hazard.
None is wrong twice over. The interview stopped
after one answer, waiting at its second question, before it
built a Result, and None is not a
Result in any case.
The type checker accepts return stop.value as a
declared Result, because
StopIteration.value has type Any. The
runtime accepts it too. The program first fails later, when code
that expects a string receives that None, far from
the driver that returned it.
Keeping the two meanings apart is a one-line discipline: put
inside the try only the call whose
StopIteration you mean to interpret.
Before you move on, name what separates your attempt from this one: a different design, a detail, or an equivalent alternative.
yield_from_send.pyPredict the output of
yield_from_send.pyafter adding a thirdyield from collect("gamma")toboth(), adding its sum to the total, and extending the loop to[1, 2, 3, 4, 5, 6]. Write down the sequence of printed lines before running it.
The
Send Channel follows each sent value to the delegated
generator suspended at that moment. Count how many values one
collect() consumes, then count how many the loop
supplies in total. Write the printed lines before you run the
script, and note which sends produce one line and which produce
two.
# The shape of exercise_3.py
from collections.abc import Generator
from typing import NewType
Prompt = NewType("Prompt", str)
type Value = int
Total = NewType("Total", int)
def collect(name: str) -> Generator[Prompt, Value, Total]:
...
def both() -> Generator[Prompt, Value]:
...you wrote the full sequence of lines before running the
modified yield_from_send.py, and it
matches the run line for line, including which sends print two
lines and what send(6) prints.
# exercise_3.py
from collections.abc import Generator
from typing import NewType
Prompt = NewType("Prompt", str)
type Value = int
Total = NewType("Total", int)
def collect(name: str) -> Generator[Prompt, Value, Total]:
first: Value = yield Prompt(f"{name} needs a value")
second: Value = yield Prompt(f"{name} needs another")
print(f"{name} got {first} and {second}")
return Total(first + second)
def both() -> Generator[Prompt, Value]:
alpha: Total = yield from collect("alpha")
beta: Total = yield from collect("beta")
gamma: Total = yield from collect("gamma")
yield Prompt(f"total {alpha + beta + gamma}")
g = both()
print(next(g))
#: alpha needs a value
for value in [1, 2, 3, 4, 5, 6]:
print(f"g.send({value}) -> {g.send(value)}")
#: g.send(1) -> alpha needs another
#: alpha got 1 and 2
#: g.send(2) -> beta needs a value
#: g.send(3) -> beta needs another
#: beta got 3 and 4
#: g.send(4) -> gamma needs a value
#: g.send(5) -> gamma needs another
#: gamma got 5 and 6
#: g.send(6) -> total 21Count the values the collectors need. Each
collect() consumes two values, so the three
collectors need six, and the loop supplies six. The
yield in both() gives
send(6) a value to return, so the loop ends with
both() suspended and no
StopIteration.
Read the output in pairs. The two-line
responses mark where one collector ends and the next begins. A
send that supplies a collector’s first value produces one line:
that collector’s second prompt. A send that supplies a
collector’s second value produces two: the completed collector’s
print(), then the next prompt.
yield from relays in both directions on its own:
prompts up to the driver, numbers down to the suspended
yield inside the current
collect().
Before you move on, name what separates your attempt from this one: a different design, a detail, or an equivalent alternative.
yield fromRemove
yield frominyield_from_nested.py, leavingprofile: Result = interview(). Runty checkand the script, and explain both results. Which one told you more, and what does the type checker say once you removeprofile’s annotation?
Relaying,
Not Answering shows survey() delegating to
interview() with yield from. Without
yield from, the call to interview()
builds a generator object, and that object sits suspended before
its first line. Compare the declared type of
profile with what the call produces, then compare
the type checker’s report with the script’s output.
your explanation accounts for both results: ty
reports an invalid-assignment on the
profile line, and the script runs and prints
<generator object interview at ...>, color blue.
It says which result locates the mistake and why, and you have
confirmed that ty check passes once the annotation
is gone.
def survey() -> Generator[Question, Answer, Result]:
profile: Result = interview()
color: Answer = yield from ask(Question("color"))
return Result(f"{profile}, color {color}")ty rejects it:
error[invalid-assignment]: Object of type
`Generator[Question, Answer, Result]` is not assignable to `Result`
--> yield_from_nested.py:8:23
|
8 | profile: Result = interview()
| ------ ^^^^^^^^^^^ Incompatible value of type
| | `Generator[Question, Answer, Result]`
| Declared type
The script still runs, and produces:
request = 'color', answer = 'blue'
<generator object interview at 0x000001C5FB3FDE40>, color blue
Both results describe the same mistake. Calling a generator
function produces a description rather than a conversation. The
call interview() returns a generator object,
suspended before the first line of its body. The generator stays
suspended for the rest of the run, so color is the
one request drive() receives. The
return statement in survey()
interpolates the object’s repr into the sentence where the
interview’s Result belonged.
The type checker told you more. Its message names the two
types and points at the assignment that mismatches them. That
assignment is the defect. The runtime output shows a consequence
two lines later, in the return statement, where the
f-string formats the object. A reader must work backward from a
<generator object ...> in a report to the
missing yield from.
The runtime failure is quiet: no exception, an exit code of
zero, and output that a log scraper accepts. When Python
reclaims an unawaited coroutine, it issues a
RuntimeWarning, as Concurrency
shows. Python reclaims an unused generator silently.
Once you remove the annotation, the type checker accepts
survey(). For profile = interview() it
infers the expression’s own type,
Generator[Question, Answer, Result], and no
declared type contradicts that inference. The f-string then
accepts any object, since formatting one calls
str() on it. A generator object’s
str() is its repr, so the output shows the
repr.
profile’s annotation alone gives the checker a
declared type to compare with the call’s type. So annotate a
local whenever you want the checker to verify the type of the
call that produces its value.
Before you move on, name what separates your attempt from this one: a different design, a detail, or an equivalent alternative.
sequence()Rewrite
sequence()inyield_from_arguments.pyas a generic function,sequence[R](*steps: Generator[Question, Answer, R]), returningGenerator[Question, Answer, list[R]]. Runty checkand the script, and explain whydrive()rejects the call while the script runs. Then write a wrapper that joins the list into aResult, sodrive()stays unchanged, and say what the type checker infers forRat each call.
A
Generator in the Middle shows sequence()
collecting each step’s return value into a
list[str] and joining the list. With a type
parameter in the ReturnType position, each step
returns a type of its own, and yield from evaluates
to that type. Compare what drive() declares for its
conversation parameter with the type of the generic
call.
# The shape of exercise_5.py
from collections.abc import Generator
from typing import NewType
Question = NewType("Question", str)
Answer = NewType("Answer", str)
Result = NewType("Result", str)
def ask(
question: Question
) -> Generator[Question, Answer, Answer]:
...
def interview() -> Generator[Question, Answer, Result]:
...
def drive(conversation: Generator[Question, Answer, Result],
answers: dict[Question, Answer]) -> Result:
...
def sequence[R](
*steps: Generator[Question, Answer, R]
) -> Generator[Question, Answer, list[R]]:
...
def joined(
*steps: Generator[Question, Answer, str]
) -> Generator[Question, Answer, Result]:
...ty reports an invalid-argument-type
on the drive() call and names
list[Result | Answer] as the return type it found,
the script still runs and prints a list, and your wrapper checks
clean under ty and prints a joined
Result.
The generic version keeps the loop and returns the list:
def sequence[R](
*steps: Generator[Question, Answer, R]
) -> Generator[Question, Answer, list[R]]:
results: list[R] = []
for step in steps:
result = yield from step
results.append(result)
return resultsty rejects the drive() call:
error[invalid-argument-type]: Argument to function `drive`
is incorrect
--> yield_from_arguments.py:17:5
|
17 | sequence(interview(), ask(Question("color"))),
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Expected `Generator[Question, Answer, Result]`, found
`Generator[Question, Answer, list[Result | Answer]]`
The checker solves R from both arguments at
once. interview() returns Result and
ask() returns Answer, so
R is their union, Result | Answer, and
the call’s type carries list[Result | Answer] in
the return position. drive() declares
Generator[Question, Answer, Result] for its
conversation, and it ends by returning
stop.value as its Result. A list is
not a Result, so the argument fails where
drive() would hand the list to its caller.
Python ignores the annotations, so the script runs to the end
and print() shows the list:
request = 'name', answer = 'Alice'
request = 'town', answer = 'Wonderland'
request = 'friend', answer = 'Rabbit'
request = 'color', answer = 'blue'
['Alice of Wonderland, friend Rabbit', 'blue']
The wrapper delegates to the generic sequence()
and joins what comes back, so drive() sees the
Result it declares. This file carries its own
ask(), interview(), and
drive(), since a solution cannot import the
chapter’s listings:
# exercise_5.py
from collections.abc import Generator
from typing import NewType
Question = NewType("Question", str)
Answer = NewType("Answer", str)
Result = NewType("Result", str)
def ask(
question: Question
) -> Generator[Question, Answer, Answer]:
answer: Answer = yield question
return answer
def interview() -> Generator[Question, Answer, Result]:
name: Answer = yield from ask(Question("name"))
town: Answer = yield from ask(Question("town"))
return Result(f"{name} of {town}")
def drive(conversation: Generator[Question, Answer, Result],
answers: dict[Question, Answer]) -> Result:
try:
request: Question = next(conversation)
except StopIteration as stop:
return stop.value
while True:
answer = answers[request]
print(f"{request = }, {answer = }")
try:
request = conversation.send(answer)
except StopIteration as stop:
return stop.value
def sequence[R](
*steps: Generator[Question, Answer, R]
) -> Generator[Question, Answer, list[R]]:
results: list[R] = []
for step in steps:
result = yield from step
results.append(result)
return results
def joined(
*steps: Generator[Question, Answer, str]
) -> Generator[Question, Answer, Result]:
results = yield from sequence(*steps)
return Result(", ".join(results))
answers = {
Question("name"): Answer("Alice"),
Question("town"): Answer("Wonderland"),
Question("color"): Answer("blue"),
}
print(drive(
joined(interview(), ask(Question("color"))), answers
))
#: request = 'name', answer = 'Alice'
#: request = 'town', answer = 'Wonderland'
#: request = 'color', answer = 'blue'
#: Alice of Wonderland, blueThe checker solves R at each call from the
argument types. Inside joined(), the call
sequence(*steps) sees steps through
its annotation, Generator[Question, Answer, str],
so R is str there and
results is a list[str], which
", ".join() accepts. At the call
joined(interview(), ask(Question("color"))), the
checker matches interview()’s
Generator[Question, Answer, Result] against
Generator[Question, Answer, str].
Generator is covariant in its return type and
Result is a str, so the match
succeeds, and ask()’s Answer passes
the same way.
The annotations on sequence() and
joined() do different jobs.
sequence()’s type parameter lets it collect any
return type, so one definition serves steps of every return
type. joined()’s str annotation makes
the join type-check, and it turns Result | Answer
into str before sequence() sees the
steps.
Before you move on, name what separates your attempt from this one: a different design, a detail, or an equivalent alternative.
report()
with a return value
report()inyield_from_return.pyyields its lines and returnsNone. Rewrite it to return the character count as well, and give it the full annotation. Then write a caller that delegates to it withyield fromand yields that count in a line of its own, and say which type parameter carries each of the two values.
The
Return Channel shows how yield from delivers a
delegate’s return value to the delegating generator.
Iterator[str] cannot declare a return value, so
report() needs the full Generator
annotation. The caller binds the count with
yield from, then yields it as a string of its
own.
# The shape of exercise_6.py
from collections.abc import Generator
from typing import NewType
Size = NewType("Size", int)
def emit(items: list[str]) -> Generator[str, None, Size]:
...
def report(items: list[str]) -> Generator[str, None, Size]:
...
def summarize(items: list[str]) -> Generator[str]:
...uv run ty check passes, and your caller’s output
for the chapter’s ["red", "green", "blue"] ends
with a line carrying 12. With
report()’s annotation changed back to
Iterator[str], ty reports
invalid-explicit-return, and you can name the type
parameter through which each of your two values travels.
The solution returns the count from report(), so
the count reaches summarize() through a second
return:
# exercise_6.py
from collections.abc import Generator
from typing import NewType
Size = NewType("Size", int)
def emit(items: list[str]) -> Generator[str, None, Size]:
total = 0
for item in items:
yield item
total += len(item)
return Size(total)
def report(items: list[str]) -> Generator[str, None, Size]:
size: Size = yield from emit(items)
yield f"{size} characters"
return size
def summarize(items: list[str]) -> Generator[str]:
counted: Size = yield from report(items)
yield f"total: {counted}"
print(list(summarize(["red", "green", "blue"])))
#: ['red', 'green', 'blue', '12 characters', 'total: 12']Declare the return channel.
report()’s annotation changes from
Iterator[str] to
Generator[str, None, Size], because a generator
that returns something needs the long form. Under
Iterator[str] a generator must return
None, so ty rejects
report()’s return size with an
invalid-explicit-return: expected
None, found Size.
If you change report()’s annotation but leave
out return size, the script still runs, and the
printed list ends with 'total: None'.
report() returns None, and the
yield from in summarize() delivers
that None into counted.
ty reports an invalid-implicit-return
on the annotation, because report() declares a
Size and, with no return statement,
finishes with None.
Keep the strings and the count apart. The
strings and the count travel by different channels. Every string
that emit() or report() yields travels
through the YieldType and comes out in the list.
The count travels through the ReturnType:
emit() returns it, yield from delivers
it into report()’s size,
report() returns it again, and the second
yield from delivers it into
summarize()’s counted. The
SendType is None throughout, since
list() drives the chain with next(),
the runtime equivalent of send(None).
The count 12 therefore reaches
summarize() through two returns.
'12 characters' takes the other route:
report() yields it, the yield from in
summarize() relays it, and the driver receives it.
The count ends up in size and then in
counted; the string ends up in the list that
list() builds. A generator can yield a number or
return it. The choice decides which code receives the number: a
yielded value goes to the driver, and a returned value goes to
the delegating generator.
Before you move on, name what separates your attempt from this one: a different design, a detail, or an equivalent alternative.
next()Explain why a driver must prime with
next()rather thansend(None), given that the two are equivalent at runtime.send_none_is_next.pyhas the answer. State it in terms of theSendType.
Annotating
a Generator defines the SendType, and send_none_is_next.py calls
send(None) on a generator whose
SendType is Answer. Read the signature
of send() and ask what it accepts for
interview(). Then ask what next()
accepts, and where a generator that has yet to run would put a
sent value.
your explanation predicts what ty reports if you
delete the # type: ignore in send_none_is_next.py (an
invalid-argument-type on None), states
the parameter type of send() in terms of the
SendType, and says why priming with
next() type-checks for any generator.
next(g) and g.send(None) do the
same thing at runtime. The SendType tells them
apart.
send()’s signature is
send(self, value: _SendT_contra, /) -> _YieldT_co,
so its parameter type is the generator’s SendType.
For interview() that is Answer, and
None is not an Answer, so the priming
call is a type error. send_none_is_next.py
carries a # type: ignore to suppress it. If you
remove that comment, the type checker reports:
error[invalid-argument-type]: Argument to bound method
`Generator.send` is incorrect
--> send_none_is_next.py:4:27
|
4 | print(f"{interview().send(None) = }")
| ^^^^ Expected `Answer`, found `None`
next() takes the generator and an optional
default to return at exhaustion. It asks for the generator’s
next yielded value and ignores the SendType, so
priming with it type-checks for any generator.
The mismatch is real rather than a type checker limitation. A
generator’s SendType describes what a suspended
yield expression can receive. A just-started
generator waits before the first line of its body rather than at
a yield, so the first send() arrives
before any yield expression is ready to receive a
value. Sending a non-None value to a fresh
generator raises
TypeError: can't send non-None value to a just-started generator.
So at runtime the first send() accepts
None alone, a value the SendType
excludes.
An annotation cannot express “None for the first
call, Answer afterward,” because a single
SendType covers every call. Widening the
SendType to Answer | None states the
exception in the type, but the type checker then applies
Answer | None to every yield
expression. The generator must handle a None at
each yield whose value it uses as an
Answer. That None exists only in the
type, because the priming None arrives before the
generator reaches its first yield.
interview() binds each received value to a local
annotated Answer, so under the wider
SendType the type checker rejects all three
assignments as invalid-assignment, and the
generator would need a None test at each
yield. Priming with next() keeps the
SendType at Answer. The one call that
cannot carry a value comes from the one function that cannot
pass one.
Before you move on, name what separates your attempt from this one: a different design, a detail, or an equivalent alternative.
A Vending Machine keeps its current state in an attribute and looks up each transition in a table. Write a simplified version as a single generator instead. It collects money, takes two digits, then dispenses or refuses. It yields its current state and receives each event with
send(), so the position in the generator’s body carries the state. This generator’syieldruns in the opposite direction frominterview()’syield. Hereyieldreports the state the machine reached instead of requesting something the machine needs. Say which of the two versions you would rather extend with another state, and why.
A
Vending Machine holds its state in an attribute and a
transition table. Here the position inside the generator’s body
is the state, so use loops and isinstance() checks
on the event that yield returns. Because
yield reports the state reached, the driver’s
send() delivers events rather than answers.
# The shape of exercise_8.py
from collections.abc import Generator
from typing import Final, Literal
from record import record
@record
class Coin:
cents: int
@record
class Digit:
value: str
type Event = Coin | Digit
type Report = Literal[
"QUIESCENT", "COLLECTING", "SELECTING",
"UNAVAILABLE", "NEED_MONEY", "DISPENSED"]
type Code = str
PRICES: Final[dict[Code, int]] = {"11": 25, "12": 75}
STOCK: Final[dict[Code, int]] = {"11": 0, "12": 3}
def machine() -> Generator[Report, Event]:
...uv run ty check passes, and one driven run
reports COLLECTING after a coin,
SELECTING after a first digit,
UNAVAILABLE for an item out of stock, a refusal for
too little money, and a sale once the money suffices. A second,
fresh machine starts with the full stock after the first one
sells.
If you write stock = STOCK instead of copying
with dict(STOCK), each sale changes the
module-level table from which every machine starts. After three
"12" sales, by any mix of machines, a fresh
machine() answers the next "12" order
with UNAVAILABLE. The Final annotation
forbids rebinding the name and permits changing the dictionary,
so the type checker passes the shared version. The solution
copies STOCK into a local, so each generator holds
its own stock.
# exercise_8.py
from collections.abc import Generator
from typing import Final, Literal
from record import record
@record
class Coin:
cents: int
@record
class Digit:
value: str
type Event = Coin | Digit
type Report = Literal[
"QUIESCENT", "COLLECTING", "SELECTING",
"UNAVAILABLE", "NEED_MONEY", "DISPENSED"]
type Code = str
PRICES: Final[dict[Code, int]] = {"11": 25, "12": 75}
STOCK: Final[dict[Code, int]] = {"11": 0, "12": 3}
def machine() -> Generator[Report, Event]:
stock = dict(STOCK)
amount = 0
event: Event = yield "QUIESCENT"
while True:
while isinstance(event, Coin):
amount += event.cents
event = yield "COLLECTING"
row = event.value # Not a Coin, so a first Digit
second: Event = yield "SELECTING"
# A coin instead of a digit
if isinstance(second, Coin):
amount += second.cents
event = yield "COLLECTING"
continue
code = row + second.value
if stock.get(code, 0) == 0:
event = yield "UNAVAILABLE"
elif amount < PRICES.get(code, 0):
event = yield "NEED_MONEY"
else:
amount -= PRICES[code]
stock[code] -= 1
event = yield "DISPENSED"
m = machine()
print(next(m))
#: QUIESCENT
for event in [Coin(25), Digit("1"), Digit("1"), Digit("1"),
Digit("2"), Coin(50), Digit("1"), Digit("2")]:
print(f"{event} -> {m.send(event)}")
#: Coin(cents=25) -> COLLECTING
#: Digit(value='1') -> SELECTING
#: Digit(value='1') -> UNAVAILABLE
#: Digit(value='1') -> SELECTING
#: Digit(value='2') -> NEED_MONEY
#: Coin(cents=50) -> COLLECTING
#: Digit(value='1') -> SELECTING
#: Digit(value='2') -> DISPENSEDLet position carry the state. The machine
holds no state attribute and consults no transition
table. Where the generator pauses is the state. A pause in the
coin loop means COLLECTING, and a pause after
yield "SELECTING" means a first digit has arrived
and the machine waits for a second. amount,
row, and stock are locals that keep
their values between sends because the suspended generator
preserves them.
Report the state reached. The
yield in machine() runs the opposite
direction from the one in interview(). Neither
signature says so. Both yield strings, but
interview() yields a request the driver must
satisfy, while machine() yields a report the driver
may ignore. send(Coin(25)) delivers an event rather
than an answer. The COLLECTING it returns reports
the state that event produced. A generator’s type declares what
each channel carries and leaves open whether a yielded value is
a request or a report.
For another state, take the table. The generator stays
compact because the states form a line, so control flow can
express the sequence. The two states here that break the line
need extra control flow. An if chain reaches
UNAVAILABLE and NEED_MONEY. From each
one, the machine loops back to the top, a goto
written as a while True. Suppose you add a state
reachable from several others, the way the table handles
Quit from every state but QUIESCENT.
No position in the body corresponds to that state. The new state
becomes a flag, or a check repeated at several
yields. Either one breaks the correspondence
between position and state that makes this version readable.
With the table, adding any state means one new
Enum member and a few new rules. Those rules sit
next to the existing ones, where you can read the whole machine
at once.
The generator is the better choice for a conversation with a
beginning and an end, like interview(). The table
is the better choice for a machine that runs forever and can go
anywhere from anywhere.
Before you move on, name what separates your attempt from this one: a different design, a detail, or an equivalent alternative.
Replace
proxy()’s loop inanswering_proxy.pywith the single linereturn (yield from inner)and rundrive()over the sameknowntable. Explain from the output why thisproxy()cannot answertown, and say what a generator must do to answer one request and relay the rest.
Relaying,
Not Answering contrasts drive(), which answers
a request, with yield from, which relays it.
proxy() answers from known in one
branch and yields the request in the other, because its own loop
sees each request before deciding. Ask where
proxy()’s body waits while inner runs
under a yield from.
# The shape of exercise_9.py
from collections.abc import Generator
from typing import NewType
Question = NewType("Question", str)
Answer = NewType("Answer", str)
Result = NewType("Result", str)
def interview() -> Generator[Question, Answer, Result]:
...
def drive(conversation: Generator[Question, Answer, Result],
answers: dict[Question, Answer]) -> Result:
...
def proxy(
inner: Generator[Question, Answer, Result],
known: dict[Question, Answer],
) -> Generator[Question, Answer, Result]:
...drive() prints three request lines, with
town answered Wonderland rather than
Oz, the Result is
Alice of Wonderland, friend Rabbit, and your
explanation says that proxy()’s body waits inside
the yield from until inner
finishes.
The relaying proxy() is short, and this file
carries its own interview() and
drive():
# exercise_9.py
from collections.abc import Generator
from typing import NewType
Question = NewType("Question", str)
Answer = NewType("Answer", str)
Result = NewType("Result", str)
def interview() -> Generator[Question, Answer, Result]:
name: Answer = yield Question("name")
town: Answer = yield Question("town")
friend: Answer = yield Question("friend")
return Result(f"{name} of {town}, friend {friend}")
def drive(conversation: Generator[Question, Answer, Result],
answers: dict[Question, Answer]) -> Result:
try:
request: Question = next(conversation)
except StopIteration as stop:
return stop.value
while True:
answer = answers[request]
print(f"{request = }, {answer = }")
try:
request = conversation.send(answer)
except StopIteration as stop:
return stop.value
def proxy(
inner: Generator[Question, Answer, Result],
known: dict[Question, Answer],
) -> Generator[Question, Answer, Result]:
return (yield from inner)
answers = {
Question("name"): Answer("Alice"),
Question("town"): Answer("Wonderland"),
Question("friend"): Answer("Rabbit"),
}
print(drive(
proxy(interview(), {Question("town"): Answer("Oz")}),
answers,
))
#: request = 'name', answer = 'Alice'
#: request = 'town', answer = 'Wonderland'
#: request = 'friend', answer = 'Rabbit'
#: Alice of Wonderland, friend Rabbitproxy() ignores known.
yield from inner relays every request
inner yields up to drive() and passes
every answer back down, and while that happens
proxy()’s body waits inside the
yield from expression. The body resumes when
inner finishes, with inner’s
Result as the expression’s value, and by then
drive() has answered every request.
town reaches drive(), which answers
Wonderland, so the Result is the
chapter’s unproxied one. yield from relays the
request; drive() answers it.
To answer one request and relay the rest, a generator must
see each request before it goes up. That means driving
inner with next() and
send() in a loop of its own, answering from
known when it can and yielding the request
otherwise, which is the chapter’s proxy(). The
chapter’s relaying yield hands one request to the
driver and receives one answer; a yield from would
hand over the whole conversation.
Before you move on, name what separates your attempt from this one: a different design, a detail, or an equivalent alternative.
PEP 255 asks its reader to picture a recursive tree walk as an iterator: “to cast that into an iterator framework requires removing the recursion manually and maintaining the state of the traversal by hand.” Write that iterator for
visit()incolor_count.py.Pathsis an iterator class, with noyieldin it, that produces the path linesvisit()yields, in the same order, and leaves out the counts. Keep the walk’s position in a list that you use as a stack. Then say whatPathsstores thatvisit()leaves to Python.
Half
a Coroutine by Design shows Squasher storing
its position in a field, because __next__() starts
at its first line on every call. A walk in progress inside
visit() is several calls deep. Each of those calls
has a path prefix and a place in its dictionary’s items, so give
each call in progress one entry on the stack.
iter(tree.items()) returns an iterator that keeps
its place between next() calls.
# The shape of exercise_10.py
import inspect
from collections.abc import Generator, Iterator
from enum import StrEnum
from types import GeneratorType
from typing import override
class Color(StrEnum):
RED = "red"
GREEN = "green"
BLUE = "blue"
type Name = str
type Path = str
type Tree = dict[Name, Tree | Color]
type Item = tuple[Name, Tree | Color]
type Level = tuple[Path, Iterator[Item]]
def visit(tree: Tree, at: Path = "") -> Generator[str]:
...
def level(tree: Tree, at: Path = "") -> Level:
...
class Paths(Iterator[str]):
def __init__(self, tree: Tree) -> None:
...
@override
def __next__(self) -> str:
...
def prefixes(walk: Iterator[str] | None) -> list[Path]:
...list(Paths(garden)) equals the six path lines
that color_count.py prints, a
tree with an empty subtree produces the lines of its other
entries, Paths contains no yield, and
your answer names the two things in each stack entry and says
where visit() keeps them.
This file carries visit() without its counts, to
compare with Paths:
# exercise_10.py
import inspect
from collections.abc import Generator, Iterator
from enum import StrEnum
from types import GeneratorType
from typing import override
class Color(StrEnum):
RED = "red"
GREEN = "green"
BLUE = "blue"
type Name = str
type Path = str
type Tree = dict[Name, Tree | Color]
type Item = tuple[Name, Tree | Color]
type Level = tuple[Path, Iterator[Item]]
def visit(tree: Tree, at: Path = "") -> Generator[str]:
for name, node in tree.items():
if isinstance(node, dict):
yield from visit(node, f"{at}{name}/")
else:
yield f"{at}{name}: {node}"
def level(tree: Tree, at: Path = "") -> Level:
return at, iter(tree.items())
class Paths(Iterator[str]):
def __init__(self, tree: Tree) -> None:
self.stack: list[Level] = [level(tree)]
@override
def __next__(self) -> str:
while self.stack:
at, items = self.stack[-1]
match next(items, None):
case None:
self.stack.pop() # [1]
case name, dict() as subtree:
self.stack.append( # [2]
level(subtree, f"{at}{name}/"))
case name, color:
return f"{at}{name}: {color}" # [3]
raise StopIteration
def prefixes(walk: Iterator[str] | None) -> list[Path]:
found: list[Path] = []
while isinstance(walk, GeneratorType):
found.append(inspect.getgeneratorlocals(walk)["at"])
walk = walk.gi_yieldfrom
return found
garden: Tree = {
"rose": Color.RED,
"beds": {
"fern": Color.GREEN,
"tulip": Color.RED,
"pond": {"iris": Color.BLUE, "reed": Color.GREEN},
},
"ivy": Color.GREEN,
}
print(list(Paths(garden)) == list(visit(garden)))
#: True
print(list(Paths({"bed": {}, "ivy": Color.GREEN})))
#: ['ivy: green']
paths = Paths(garden)
print([next(paths) for _ in range(4)][-1])
#: beds/pond/iris: blue
print([at for at, _ in paths.stack])
#: ['', 'beds/', 'beds/pond/']
walk = visit(garden)
print([next(walk) for _ in range(4)][-1])
#: beds/pond/iris: blue
print(prefixes(walk))
#: ['', 'beds/', 'beds/pond/']A stack entry is a call in progress. A
Level holds the two things one visit()
call needs to continue: the prefix at, and an
iterator that keeps the call’s place in its dictionary’s items.
level() takes the two arguments that
visit() takes, so [2] pushes a level
where visit() calls itself. [1] pops a
level where a visit() call finishes its loop, and
[3] returns a line where visit()
yields one. For {"bed": {}, "ivy": Color.GREEN},
[2] pushes a level for bed. The next
pass finds that level empty and pops it at [1].
__next__() finds its place on every
call. visit() continues after its
yield when the consumer asks for another line.
__next__() starts at its first line, so its
while loop reads the top of the stack to find the
level where the walk stopped. That level’s iterator supplies the
place within the level.
Paths stores the chain of
calls. After four lines, with the walk at
beds/pond/iris, paths.stack holds
three levels: the garden, beds/, and
beds/pond/. The generator holds the same three
levels, as three suspended visit() generators. Each
one waits at its place in the loop over
tree.items(), with at among its
locals: the outer two at a yield from, the
innermost at its yield. A delegating generator’s
gi_yieldfrom attribute refers to the generator to
which it delegates. prefixes() follows
gi_yieldfrom from the outermost
visit() to the innermost and reads the same three
prefixes.
Python adds a generator to the chain at each
yield from and discards that generator when it
returns. In Paths you write the chain as a list,
with an append() for each call and a
pop() for each return. That list is the PEP’s
“state of the traversal.”
GoF Design Patterns makes the same observation about
an iterator that the client advances over a recursive structure
such as a Composite:
the iterator “has to store a path” through the structure “to
keep track of the current object.” paths.stack is
that path.
A chain of generators stops at Python’s recursion limit, as
the chapter’s tail_delegation.py shows.
On a tree 1,500 levels deep, visit() raises
RecursionError. Paths walks that tree,
because its stack is a list.
Before you move on, name what separates your attempt from this one: a different design, a detail, or an equivalent alternative.
step()insaga.pytakes a name and prints its undo. Replacenamewith two functions:action, which the step calls and whose result it yields, andundo, which theexceptclause calls. Build one booking of a flight, a hotel, and a card whose three actions succeed, and one whose card action raisesFailed. The driver of the second booking calls onlynext(). Say whereFailedstarts, and why the hotel and the flight are still undone.
Through
yield from to the Innermost Generator says that
an exception leaving the innermost generator propagates into the
generator that delegates to it, as if the
yield from expression had raised it. An exception
that the innermost generator’s own body raises takes that path
too, with no throw(). Ask which generators are
waiting at a yield from when the card step calls
its action.
# The shape of exercise_11.py
from collections.abc import Callable, Generator
from exceptions import expect
class Failed(Exception):
pass
type Action = Callable[[], str]
type Undo = Callable[[], None]
def step(
action: Action, undo: Undo,
rest: Generator[str] | None = None,
) -> Generator[str]:
...
def doing(name: str) -> Action:
...
def undoing(name: str) -> Undo:
...
def decline() -> str:
...
def book(card: Action) -> Generator[str]:
...the first booking lists three results, the second prints
undo hotel and undo flight before
Failed reaches the driver, no line calls
throw(), and your answer says which
try blocks are open when the card’s action
runs.
doing() and undoing() build a
step’s action and its undo from its name. decline()
is the card action that fails:
# exercise_11.py
from collections.abc import Callable, Generator
from exceptions import expect
class Failed(Exception):
pass
type Action = Callable[[], str]
type Undo = Callable[[], None]
def step(
action: Action, undo: Undo,
rest: Generator[str] | None = None,
) -> Generator[str]:
yield action() # [1]
if rest is None:
return
try:
yield from rest
except Failed:
undo() # [2]
raise
def doing(name: str) -> Action:
return lambda: f"do {name}"
def undoing(name: str) -> Undo:
return lambda: print(f"undo {name}")
def decline() -> str:
raise Failed("card declined")
def book(card: Action) -> Generator[str]:
return step(
doing("flight"), undoing("flight"),
step(doing("hotel"), undoing("hotel"),
step(card, undoing("card"))),
)
print(list(book(doing("card"))))
#: ['do flight', 'do hotel', 'do card']
booking = book(decline)
print(next(booking))
#: do flight
print(next(booking))
#: do hotel
expect(Failed, next, booking) # [3]
#: undo hotel
#: undo flight
#: [Failed] card declinedThe step calls its action. [1]
calls action() and yields the result, where saga.py yields a string
built from name. [2] calls
undo() where saga.py prints. The
try, the yield from, and the bare
raise stay as saga.py has them.
Failed starts in the card step’s
body. The next() at [3]
resumes the flight step, which waits at its
yield from. That yield from resumes
the hotel step, and the hotel step’s yield from
starts the card step. The card step runs [1], and
decline() raises Failed there. No
try in the card step encloses [1], so
the exception leaves the card step. It arrives at the hotel
step’s yield from, which sits inside the hotel
step’s try. The hotel step’s except
clause calls its undo() and raises
Failed again, and the flight step’s
except clause does the same. Both try
blocks are open when the card’s action runs.
throw() raises an exception at a
yield for the driver. Here the generator’s own code
raises the exception, and it propagates through the two
yield from expressions as it does in saga.py. The driver’s part
is the next() call, where Failed
arrives after both undo lines.
Before you move on, name what separates your attempt from this one: a different design, a detail, or an equivalent alternative.
Drive a booking from
saga_generators.pytoissue ticketand throwFailedthere. Say which command the driver has carried out and which undo commands it receives. Then changestep()so that a step that fails partway through itsdoalso undoes what it did. ThrowFailedatissue ticketand atcharge card, and say what anundomust now tolerate.
Sagas
with Generators puts each step’s try around
yield from rest. Find the line where
step() is paused when the driver has just received
issue ticket, and check whether that line is inside
the try. For the change, move one line and remove
the early return.
# The shape of exercise_12.py
from collections.abc import Generator
class Failed(Exception):
pass
type Commands = Generator[str]
def commands(*names: str) -> Commands:
...
def step(
*, do: Commands, undo: Commands,
rest: Commands | None = None,
) -> Commands:
...
def book() -> Commands:
...
def unwind(saga: Commands, error: Failed) -> None:
...you can say why the chapter’s step() leaves the
seat reserved, your step() yields
void ticket and release seat for a
failure at issue ticket, it yields
refund card first for a failure at
charge card, and your answer names the undo
commands whose actions failed.
With the chapter’s step(), the driver has
carried out reserve seat when it throws
Failed at issue ticket. The flight
step is paused inside yield from do, which comes
before its try. Failed leaves the
flight step and reaches the driver with no undo command, and the
seat stays reserved.
This step() puts do inside the
try:
# exercise_12.py
from collections.abc import Generator
class Failed(Exception):
pass
type Commands = Generator[str]
def commands(*names: str) -> Commands:
yield from names
def step(
*, do: Commands, undo: Commands,
rest: Commands | None = None,
) -> Commands:
try:
yield from do # [1]
if rest is not None:
yield from rest
except Failed:
yield from undo
raise
def book() -> Commands:
return step(
do=commands("reserve seat", "issue ticket"),
undo=commands("void ticket", "release seat"),
rest=step(
do=commands("hold room"),
undo=commands("release room"),
rest=step(
do=commands("book taxi"),
undo=commands("cancel taxi"),
rest=step(
do=commands("charge card"),
undo=commands("refund card"),
),
),
),
)
def unwind(saga: Commands, error: Failed) -> None:
try:
print(saga.throw(error))
for command in saga:
print(command)
except Failed as e:
print(f"Failed: {e}")
booking = book()
print([next(booking) for _ in range(2)][-1])
#: issue ticket
unwind(booking, Failed("printer jammed")) # [2]
#: void ticket
#: release seat
#: Failed: printer jammed
booking = book()
print([next(booking) for _ in range(5)][-1])
#: charge card
unwind(booking, Failed("card declined")) # [3]
#: refund card
#: cancel taxi
#: release room
#: void ticket
#: release seat
#: Failed: card declinedEvery command of a step is inside its
try. [1] moves
yield from do into the try, so a
Failed thrown at any of a step’s commands reaches
that step’s except clause. At [2] the
flight step yields void ticket and
release seat, and the driver releases the seat.
An undo now runs for an action that
failed. At [2], void ticket
asks the driver to void a ticket whose printing failed. At
[3] the failure is in the card step, and its
undo runs first: refund card asks the
driver to refund a charge that the bank declined. Each undo
command must therefore be safe to carry out when its action did
not complete. The chapter’s step() undoes completed
steps, which is enough while each do is a single
command.
Before you move on, name what separates your attempt from this one: a different design, a detail, or an equivalent alternative.