Contents
Chapter 44

Effect Management

A function does more than its signature says: it reads a clock, writes a log, opens a connection. Effect Management makes the extra visible to a reader, a test, and a type checker.

A test you wrote last week starts failing about one run in five. The function it calls computes a total price, and the math is right. Three calls down, a helper that formats currency reads a configuration service and writes to an audit log. None of that is in any signature on the path.

This book has emphasized the benefits of pure functions in numerous places:

In every one of those cases you can settle the question of purity by reading one function. That stops working as soon as the function calls others. If any of them has a side effect, the calling function is impure too. To discover whether a function is impure, you must either trust the documentation or examine the code of every function it calls.

Reading every callee soon becomes tedious and error-prone. That reading is the work a type system would do for you, if it verified purity. A system that does so is an Effect Management System.

What Is an Effect?

An Effect is anything a caller takes on by making a call, beyond receiving the return value. If you call a function that writes to an audit log, your function writes to an audit log. If you call one that reads the time of day, your result depends on the time of day. If you call one that raises a ValueError, your function raises a ValueError. You inherit each of these by calling, and ordinarily no signature on the path mentions any of them.

That inheritance is the difficulty. Something that stays inside the function performing it needs no system to manage it, because one read of that function settles the question. An Effect passes from callee to caller instead, one call at a time, and no signature on the path shows it until something in the types records it.

Three things pass from callee to caller that way. The first is a side effect: calling the function does something besides return a result, changing the environment outside the function. For example, the function might:

A side effect is easy to spot in the function that performs it, because the side effect changes something outside that function. One call up, it is invisible.

The second is a side cause, the counterpart of a side effect: what the environment does to the function. Suppose your function reads the time of day, or a random number. The read changes nothing in the environment, yet the result differs from one call to the next. Information a function uses beyond its arguments is a side cause whenever it can change between calls. The usual sources are I/O: the time of day, a random number, a database or network read. Reading a global variable that something else can rebind is enough on its own. A captured constant, as in Closures, is not.

The third is an exception, which propagates from callee to caller the same way and appears in no signature either. People argue about whether an exception makes a function impure.

Are Exceptions Impure?

# divide_by_zero_impurity.py

def slope(rise: int, run: int) -> float:
    return rise / run

slope() produces the same result for the same inputs, except when run is zero, where it raises an exception instead of returning a result. Does the exception break purity?

  1. Pure. Raising ZeroDivisionError instead of returning a number does not break purity. The same arguments still produce that same exception every time. The function reads nothing outside itself and changes nothing outside itself. Purity says the outcome depends on the arguments alone.

    Computer science theory backs this view. Pure languages like Haskell treat an unhandled runtime exception or crash as a bottom value, denoted ⊥. A bottom value stands for a computation that never produces an ordinary value. Because ⊥ is a value in the theory, raising an error that nothing catches is referentially transparent. You can replace the function call with the crash itself, and the program behaves the same.

  2. Functional. Exceptions bypass normal control flow, and that bypass makes it difficult to reason about code, so functional programming avoids them altogether. A Total Function returns errors as data in explicit wrapper types instead of raising them.

The argument over purity leaves this chapter’s question open. If you write a function a() that calls a function b() that raises an exception, then a() raises that exception too, unless a() catches it. a()’s signature says nothing about the exception. a() takes the exception on by calling b(), whichever school you join, so an exception is an Effect alongside the side effect and the side cause.

Converting Effectful to Pure

Transforming the exception Effect in slope() from divide_by_zero_impurity.py makes the function pure.

Return a Result Type

Wrap the answer and the failure in a Result, the way Error Handling does. If you decorate the original slope(), unchanged, with that chapter’s @safe, every exception it raises becomes a value instead of a crash:

# slope_result.py
from result import Err, Ok
from safe import safe

@safe
def slope(rise: int, run: int) -> float:
    return rise / run

for args in [(10, 2), (10, 0)]:
    match slope(*args):
        case Ok(answer):
            print(f"slope{args} = {answer}")
        case Err(error):
            print(f"slope{args}: {type(error).__name__}")
#: slope(10, 2) = 5.0
#: slope(10, 0): ZeroDivisionError

@safe catches whatever slope() raises, so the fix is the decorator, not a change to the function it wraps. slope() is now total. Every call returns a Result, and the caller must unpack it to reach the number.

Catch the Exception You Expect

If you catch and handle the exception within the function, the exception stays inside that function, so it is not an Effect. slope() can catch the one exception it names and turn the failure into an ordinary float, its existing return type, instead of introducing a new type:

# slope_catch.py
from exceptions import expect

def validate(run: int) -> int:
    if run < 0:
        raise ValueError(f"run cannot be negative: {run}")
    return run

def slope(rise: int, run: int) -> float:
    try:
        return rise / validate(run)
    except ZeroDivisionError:
        return float("inf")

print(slope(10, 2))
#: 5.0
print(slope(10, 0))
#: inf
expect(ValueError, slope, 10, -1)
#: [ValueError] run cannot be negative: -1

Catching the exception works with the types you have. But it guards only the exceptions slope()’s try names. validate() raises ValueError for a negative run, and the try around it catches ZeroDivisionError and lets ValueError propagate. Because slope() calls validate(), validate()’s Effect becomes slope()’s Effect. This listing puts validate() directly above slope(), so the uncaught ValueError is easy to spot. In a real call stack the raise is usually in another file, several calls down, and finding it means reading every callee: the tedious, error-prone work an Effect Management System replaces. Catching by hand covers exactly the exceptions you know a callee can raise. Knowing every one of them is the tracking problem an Effect Management System solves.

C++ and Java tried to track exceptions with exception specifications, a list of exceptions written by hand on each function. The compiler did not compute that list from the functions a body called, so an exception introduced three levels down meant editing every signature above it by hand. Programmers usually avoided that work by widening the specification until it permitted every exception. The specifications exposed implementation details, and most people now count them a failure. C++ reduced its version to a single bit: whether a function can throw.

Make the Bad Value Impossible

The third approach removes the failure instead of handling it. Data Classes as Types makes illegal values impossible to construct. If you give run a type that excludes zero, the check moves out of slope() and into the type:

# slope_nonzero.py
from exceptions import expected
from record import record

@record
class NonZero:
    value: int

    def __post_init__(self) -> None:
        if self.value == 0:
            raise ValueError("NonZero cannot hold 0")

def slope(rise: int, run: NonZero) -> float:
    return rise / run.value

print(slope(10, NonZero(2)))
#: 5.0
with expected(ValueError):
    NonZero(0)
#: [ValueError] NonZero cannot hold 0

The check runs once, in __post_init__(), when NonZero(...) builds the instance. Every function that receives a NonZero, including slope(), inherits that guarantee. slope() cannot divide by zero, so it needs no try and no Result to say so.

Combine the First and Third

All three approaches keep a division by zero from escaping slope() as an exception, but each puts the handling in a different place. A Result makes every caller handle failure explicitly, at every call site. @safe catches Exception, so slope_result.py’s Result[float, Exception] cannot distinguish ZeroDivisionError from a bug, the same limitation Error Handling names. Catching by hand puts the fix inside slope(), and an exception the try does not name still propagates out. A restrictive type checks once, at construction, and every function downstream is pure because it receives the type, not because its author remembered a check. None of the three makes the failure disappear. A Result turns it into a value, a try catches it, and NonZero moves it to the one line that builds the NonZero. They differ in how many functions must know about it.

Standard practice combines the first and third, the Result and the restrictive type. The boundary parses untrusted input into the restrictive type, using a Result to report a bad value instead of raising an exception, and every function past that boundary takes NonZero and stays total:

# slope_edge.py
from record import record
from result import Err, Ok
from safe import safe

@record
class NonZero:
    value: int

    def __post_init__(self) -> None:
        if self.value == 0:
            raise ValueError("NonZero cannot hold 0")

def slope(rise: int, run: NonZero) -> float:
    return rise / run.value

@safe
def parse_run(text: str) -> NonZero:
    return NonZero(int(text))

for text in ["2", "0"]:
    match parse_run(text):
        case Ok(run):
            print(slope(10, run))
        case Err(error):
            print(f"{text!r}: {type(error).__name__}")
#: 5.0
#: '0': ValueError

parse_run() is the only place that can fail, and @safe turns that failure into a Result its caller must unpack. Past that one match, slope() checks nothing. NonZero guarantees run.value isn’t 0. The Result handles the input a caller doesn’t trust, and NonZero lets every function downstream trust what it receives.

A Program Can Never Be Pure

A perfectly pure program computes something and writes the result nowhere. It reads nothing from its environment and changes nothing in its environment, so its result reaches no screen, file, or socket, not even the exit code the operating system checks. From outside the process, that program is indistinguishable from one that computes nothing.

# pure_and_pointless.py
import timeit
from benchmark import report

def compute_and_discard() -> None:
    total = 0
    for i in range(2_000_000):
        total += i * i

def do_nothing() -> None:
    pass

busy = timeit.timeit(compute_and_discard, number=5)
idle = timeit.timeit(do_nothing, number=5)
report(busy=busy, idle=idle, ratio=busy / idle)
print(f"burned real CPU time for nothing: "
      f"{busy > idle * 100}")
#: burned real CPU time for nothing: True

Both functions return None and touch nothing outside themselves, so a caller sees the same thing from each. compute_and_discard() still takes measurably longer, because Python runs every loop you write, worthless or not. A perfectly pure computation, followed to its logical end, is a space heater with extra steps.

Effects are not a defect to design away. They are the reason a program exists. Effect Management keeps the Effects and isolates them, so the rest of the program can stay pure. People call this “pushing the Effects to the edges.”

What do you gain by keeping the rest pure? The first and most obvious answer is parallelism. A function with no Effects reads and writes nothing shared, so it is safe to run in parallel. The same guarantee makes testing trivial. A pure function needs no setup or teardown, and nothing to mock. Call it with arguments and check the result.

Two Phases of Effect Analysis

Think of Effect analysis as two phases. The first phase separates pure from impure, and produces parallelism, caching, and easy testing for the pure part.

The second phase divides the impure part by kind, and each kind yields its own benefit:

Each of those benefits is a testing benefit, for one reason. A test must run in an environment it controls, and an untracked Effect is part of the environment outside that control. Every Effect you isolate is one your tests can control.

Every one of those benefits depends on knowing where the Effects are. In a small program you find them by inspection. As programs grow, inspection stops scaling. An Effect Management System replaces it.

Effect Management Systems

The helper behind the failing test in the chapter’s opening is the ordinary case. Most functions in most programs perform Effects their signatures leave out, and that makes code hard to understand:

You cannot answer these questions by reading the function’s signature. You must read the implementation, then trust that you found everything it needs, everything it changes, and everything that might go wrong. In a small codebase you can hold that knowledge in your head. In a large one you cannot. A function you understand today gets called by a function written next week, which gets called by code a colleague writes next month. Each step adds dependencies no signature names, and no one knows the whole set.

The missing piece is tracking. With it you know what a function does: whether it is safe to run in parallel with another, and what it does when you call it twice in a row. That knowledge lets you compose functions, and composition is how programs grow large.

Tracking and Management

An Effect Management System (EMS) keeps track of Effects in functions. If your function calls an effectful function, that Effect belongs in your function’s type. If another function then calls yours, the same Effect belongs in that function’s type, and so on out to the edge of the program. A native system adds each Effect for you. A library like Stateless has you declare each one, and the type checker verifies each declaration. With an EMS, the function signature tells you whether the function is pure, and for an impure function it names the kinds of impurity.

A full EMS does three things:

  1. Tracks Effects. The type system records which Effects a function may perform.
  2. Separates each Effect’s interface from its implementation. A function declares which Effects it uses, not how to fulfill them.
  3. Binds the implementation later. Some caller or context supplies the implementation, at a point after the function’s definition.

The first item can stand alone. The difference between it and the whole list matters in the chapters that follow. Effect tracking tells you which Effects a function can perform. Tracking alone tells you whether a function is pure, and names the kinds of impurity a caller inherits. Effect management is tracking plus the second and third items. Once an Effect is an interface and a caller binds the implementation later, you can replace what the Effect does without editing the function that performs it. Tracking reports a function’s Effects, and management lets you control them. Tracking without management appears twice later in this chapter: in most of the languages in Custom AI Languages with Effects, and in Python’s async, which Effect Management for Python? examines.

The third item names delayed binding. Delayed binding exists so that the same codebase can serve many contexts (test, production, retry-wrapped) without edits. When a hundred functions declare “I need something that can read from storage,” none of them names a storage implementation. Each declaration propagates to its caller, up to a single point, usually the edge of the program, where storage binds to an implementation. Changing that one binding changes the behavior of all hundred functions at once. A test provides an in-memory binding, production provides the real database, and none of the hundred functions change. Cross-cutting behavior works the same way. To add caching, tracing, or retries to every storage access, you insert a layer at the binding point instead of touching every call site. Variation concentrates at the boundary of the program, and the functions below it stay simple and uniform.

Effects by Hand

Every technique in Converting Effectful to Pure manually manages one Effect, the exception. A Result tracks failure in the return type. A try binds the failure to a handler. A restrictive type removes the failure at construction. Each is a hand-built version of something an EMS automates.

Side effects and side causes also have a by-hand technique: pass the implementation in as a parameter. Instead of calling the built-in input() and print(), greet() declares what it needs:

# ask_tell.py
from dataclasses import dataclass, field
from typing import Protocol

class Ask(Protocol):
    def ask(self, prompt: str) -> str: ...

class Tell(Protocol):
    def tell(self, message: str) -> None: ...

def greet(ask: Ask, tell: Tell) -> None:
    name = ask.ask("What is your name? ")
    tell.tell(f"Hello, {name}!")

class Scripted:
    def ask(self, prompt: str) -> str:
        return "Alice"

@dataclass
class Capture:
    messages: list[str] = field(default_factory=list)

    def tell(self, message: str) -> None:
        self.messages.append(message)

captured = Capture()
greet(Scripted(), captured)
print(captured.messages)
#: ['Hello, Alice!']

greet() performs an Ask Effect and a Tell Effect, and its signature says so, because the parameters are the Effects. The bindings come later. The demo binds them to test stand-ins, Scripted and Capture, and checks the greeting with no console in sight. A production caller passes objects that read with input() and write with print(), and greet() stays the same. Delayed binding by hand explains why “pass in your dependencies” is such durable advice.

The signature says what greet() needs, not everything greet() might do. A print() in the body is still invisible.

The technique works, but you do the bookkeeping. Every function that calls greet() must accept an Ask and a Tell so it can pass them down. Parameters accumulate at every level of the call stack:

# bookkeeping_scales.py
from dataclasses import dataclass, field
from typing import Protocol

class Ask(Protocol):
    def ask(self, prompt: str) -> str: ...

class Tell(Protocol):
    def tell(self, message: str) -> None: ...

def greet(ask: Ask, tell: Tell) -> None:
    name = ask.ask("What is your name? ")
    tell.tell(f"Hello, {name}!")

def session(ask: Ask, tell: Tell) -> None:
    greet(ask, tell)

def menu(ask: Ask, tell: Tell) -> None:
    session(ask, tell)

def main(ask: Ask, tell: Tell) -> None:
    menu(ask, tell)

class Scripted:
    def ask(self, prompt: str) -> str:
        return "Alice"

@dataclass
class Capture:
    messages: list[str] = field(default_factory=list)

    def tell(self, message: str) -> None:
        self.messages.append(message)

captured = Capture()
main(Scripted(), captured)
print(captured.messages)
#: ['Hello, Alice!']

session(), menu(), and main() do not call ask.ask() or tell.tell(), yet each must name both parameters only to pass them to the function below it. Nothing propagates automatically. If a new helper that greet() calls needs a Log Effect, you edit every signature on the path: greet(), session(), menu(), and main(), plus the new helper, five signatures in all (see exercise 2). Dependency injection frameworks relocate this bookkeeping into a wiring layer, but you must tell the injector what every function needs, and tell it again when that changes. Only a runtime failure verifies the wiring.

Python has one mechanism that propagates on its own. A ContextVar holds a value for the current task, and any function the task calls reads the value without receiving it as an argument. That is the automatic propagation the parameter list lacks, but the ContextVar removes the parameter along with the one benefit the parameter provided. greet(ask, tell) states its Effects in its signature, and a greet() that reads two ContextVars states nothing. Setting the wrong one, or forgetting to set one, fails at the read, in whatever frame reads it. The bookkeeping stays, and the type checker can no longer verify it.

An EMS moves the bookkeeping into the type system, where a native system maintains it for you, or, with a library like Stateless, the type checker verifies every declaration you write. Moving the bookkeeping into the type system takes a second channel in the signature, one that carries Effect information outside the argument list.

Native Effect Management

Ideally, Effect tracking comes built into the language, as a native Effect system. In a native system, the type system holds Effects alongside ordinary types. A function’s signature carries two pieces of information: what it returns, and what Effects it performs. The body looks like ordinary sequential code. The compiler reads what the body calls and infers the Effects, the same way it tracks whether a value is an integer or a string.

The examples in this section and the next come from my research, in which I build the same small programs in four Effect-managing languages. The first is Koka, a research language with native Effects. Its greeting program has the same ask and tell as ask_tell.py:

// Effect declarations: the interface, not the implementation
effect ask
  fun ask(prompt : string) : string

effect tell
  fun tell(message : string) : ()

// Core logic: the Effect row <ask,tell> is part of the type
fun greet() : <ask,tell> ()
  val name = ask("What is your name? ")
  tell("Hello, " ++ name ++ "!")

// Main binds each Effect to an implementation
fun main() : <console,exn> ()
  with fun ask(prompt)
    print(prompt)
    readline()
  with fun tell(message)
    println(message)
  greet()

The angle brackets in greet()’s signature hold the Effect row, the set of Effects the function performs. The row is the second channel. ask and tell are part of the type without adding a parameter. The compiler infers the row from what the body calls, so you rarely write one by hand. You annotate explicitly when you want a constraint, such as declaring that a function must remain Effect-free. If another function calls greet(), the compiler adds ask and tell to that function’s row automatically. The by-hand version makes you add them yourself, one parameter per signature.

Something must eventually fulfill every Effect, and the construct that fulfills one is a handler. The compiler rejects a program that performs an Effect with no handler in scope, so every Effect a running program performs has a handler. Think of a handler as a generalized except block. An except block intercepts exceptions and decides what the program does next. A handler intercepts any Effect operation and decides what it means. In main(), the with fun ask(prompt) handler decides that ask means “prompt the console and read a line.”

Handling an Effect also discharges it. main()’s row is not <ask,tell> but <console,exn>. The handlers remove ask and tell, and the row that remains holds the Effects the handler bodies perform: console from the printing and reading, exn because readline() can fail. A test installs a different handler, one that returns a known name, and greet() runs unchanged.

Separating the request from its fulfillment is the core of every Effect system. The code that requests an Effect stands apart from the code that performs it, and a handler sits between them. greet() names ask and tell without deciding what either one means. The handler decides, and a different handler decides differently.

Handlers can do more than except blocks can. When an operation runs, the handler receives the continuation: the rest of the computation from that point forward. An except block has two options, catch or propagate, and both discard the continuation. A handler can resume the continuation once, and the operation then behaves like a normal function return. A handler can discard the continuation, and the operation then behaves like an exception. A handler can even invoke the continuation several times, which is how native systems express retries and backtracking as ordinary handlers. Algebraic effects is the name for this design: operations declared as an interface, plus handlers that receive the continuation.

A Python generator suspends a computation, hands control to whoever is driving it, and resumes it with a value. That makes a suspended generator a continuation you can resume once or discard, but not resume twice, because nothing can copy a generator’s paused frame. Generators covers the full two-way form, the mechanism behind the Python Effect library in Stateless.

Flix expresses the same model with different notation. The Effect set follows a backslash:

def greet(): Unit \ {Ask, Tell} =
    let name = Ask.ask("What is your name? ");
    Tell.tell("Hello, ${name}!")

Languages in this family include Koka, Flix, Eff, Effekt, and Unison. OCaml 5 added the handler mechanism, though it does not yet track Effects in function types.

Library Effect Management

Changing languages is rarely an option. A team committed to Scala or TypeScript cannot use native Effects, so designers built library Effect systems on top of existing type systems. In this approach the library, rather than the compiler, does the tracking, by encoding Effect information into the return type of every function. That encoding changes the mechanism. Instead of writing a computation and letting the compiler observe its Effects, you build a description of a computation, and execute the description later. ZIO is such a library, for Scala. In ZIO, “Hello, World!” is a description first and a run second:

import zio.*
import zio.Console.printLine

// The Effect's interface
trait Tell:
  def tell(message: String): UIO[Unit]

// Accessor: lifts the interface method into a ZIO description
object Tell:
  def tell(message: String): ZIO[Tell, Nothing, Unit] =
    ZIO.serviceWithZIO[Tell](_.tell(message))

// Core logic: a value, not an action; nothing runs here
val hello: ZIO[Tell, Nothing, Unit] =
  Tell.tell("Hello, World!")

// The implementation, packaged for delayed binding
val consoleTell: ULayer[Tell] = ZLayer.succeed(new Tell:
  def tell(message: String): UIO[Unit] =
    printLine(message).orDie)

// Entry point: bind the implementation, then execute
object Main extends ZIOAppDefault:
  def run = hello.provide(consoleTell)

The three type parameters of ZIO[Tell, Nothing, Unit] carry the Effect information. Tell is the environment the computation requires. Nothing is the error type, meaning hello cannot fail. Unit is what it produces on success. The signature does the same job as Koka’s Effect row. It tells you what hello needs, what can go wrong, and what comes back.

Everything else in the listing is machinery: a trait for the interface, a companion object to lift that interface into the ZIO type, a ZLayer to package the implementation, and a provide() call to bind it. All of that, to print one string. The machinery exists because the language cannot intercept an Effect at the point where it runs, the way a native handler can. A library can act only on values, so every Effect must become a value. hello is a data structure describing a program. Nothing executes until the ZIO runtime interprets that structure at run, the boundary between description and action (sometimes called “the edge”).

The TypeScript Effect library works the same way:

import { Context, Effect, Layer } from "effect"

// The Effect's interface, as a service tag
class Tell extends Context.Tag("Tell")<
  Tell,
  { tell: (message: string) => Effect.Effect<void> }
>() {}

// Core logic: still just a description
const hello = Effect.gen(function* () {
  const tell = yield* Tell
  yield* tell.tell("Hello, World!")
})

// The implementation, packaged for delayed binding
const ConsoleTell = Layer.succeed(Tell, {
  tell: (message) => Effect.sync(() => console.log(message)),
})

// The boundary: descriptions above, execution here
Effect.runPromise(hello.pipe(Effect.provide(ConsoleTell)))

The description/execution split is an artifact of building the system as a library, rather than a feature of Effect Management. Native systems deliver tracking, interface separation, and delayed binding while the code runs eagerly, with no description trees and no interpreter. A library has only the description route. In a language not designed for Effects, a library gets delayed binding by deferring execution. That deferral adds one question to every value you handle: is it a description or an action? Code that mixes the two compiles cleanly but misbehaves, because the imperative part runs when the program builds the description, not when the runtime executes it.

Libraries in this family include ZIO, Cats Effect, and Kyo in Scala, polysemy and effectful in Haskell, Effect in TypeScript, and Stateless in Python. Stateless builds on generators, so Generators covers that mechanism first. Stateless then writes these programs again in the language this book teaches, and Stateless in Practice rebuilds the ask/tell pair from Effects by Hand.

Custom AI Languages with Effects

At this writing, experimental languages designed for AI code generation are proliferating. Their designers try to balance better code generation for the AI against human verifiability. A language written for an AI can drop the conveniences that help a person read code, and an AI can start using that language as soon as it works, so adoption skips the years a human language spends waiting for people to learn it.

Most of these languages are tracking systems, in the sense Tracking and Management gives the term. They provide the first part of a full EMS and stop there. For their purpose the other two parts, interface separation and delayed binding, are liabilities, because a host that supplies every implementation can guarantee what generated code can do.

Two go further. In Pact, a function declares a needs clause, and a separate using clause rebinds each implementation, so tests can swap Effects deterministically. Lumen writes source as markdown with algebraic effects, and its bind effect rebinds a handler separately from its use. Both separate an Effect’s interface from its implementation and bind the implementation later, the second and third properties of a full EMS.

Effect Management for Python?

The Python language has no Effect Management System, but it has a start. Python tracks one Effect in function signatures, and enforces that tracking on every caller: async.

# coroutines_are_descriptions.py
import asyncio

ran: list[str] = []

async def greet() -> str:
    ran.append("body")
    return "Hello"

description = greet()  # Nothing runs
print(type(description).__name__, ran)
#: coroutine []
print(asyncio.run(description), ran)
#: Hello ['body']

Calling greet() builds a coroutine object, a description of work, and ran stays empty. The body runs only when something drives the description: an await, a task, or asyncio.run(). That is the library Effect system model. Descriptions compose inside async def functions, and asyncio.run() is the boundary where description becomes action.

Python enforces the tracking the way an EMS does. await is a syntax error outside an async def, so any function that awaits a coroutine must become async, and so must its callers, all the way up to the edge. If you replace “async” with “network access” or “database write” in that sentence, you have described Effect tracking. Python demonstrates that the machinery can work, and hard-codes it to a single Effect, concurrency, rather than letting you declare your own.

Third-party libraries supply pieces of the rest. The returns library provides a Result container like the one in Error Handling, a Maybe container for a value that might be absent, an IO container that marks a value as having come from input/output, and a RequiresContext container for delayed binding of dependencies. The effect library ports the description/execution split to Python. It shares only its name with the TypeScript library. Code builds objects describing intents, and separate performers execute them, swappable for tests. The eff library models Effect handlers. Each of these supplies part of an EMS, and none supplies all three parts.

One library supplies all three parts. Stateless encodes each Effect’s dependencies and failures into the return type of every function that performs that Effect, and a type checker verifies that each caller carries them forward. Leaving a declared dependency unbound is a type error. Calling an effectful function from one annotated as pure is a type error. That is tracking, interface separation, and delayed binding, the three properties of a full EMS, inside Python’s existing type system.

The guarantee has a limit. Stateless verifies that the Effects you declare propagate consistently. A function can still call print(), next to its carefully declared Effects. In Koka, that call changes the function’s Effect row, and every caller’s row. In Python, neither Stateless nor the type checker reports it. A library checks the Effects you wrote down, but checking the ones you left out takes the language.

Could Python itself gain Effect tracking, so that a tool infers every declaration instead of you writing it? The annotation syntax could carry the Effect row. Imagine a signature that declares its Effects the way async def already declares one. The hard part is propagation, not syntax. A type checker must compute the Effect row of every function from the functions it calls, across every library on PyPI, almost all of which carry no Effect annotations. async succeeded because it arrived as part of the language. The interpreter enforces it, and its keyword marks each function that carries the Effect. An Effect row must instead reach every library in an ecosystem of untracked code. Gradual typing faced the same problem, and took a decade.

No PEP proposes Effect tracking today. If one arrives, it will contain the ideas in this chapter. Effect Tracking works through one candidate design: an Effect row carried in typing.Annotated metadata, and an analysis of the tool that would propagate and check it. The appendix calls the result algebraic effect tracking: the row a native system keeps, without the handlers. An Effect Checker builds the part of that tool that needs no type inference.

Effects Are the Next Barrier

The history of programming is a history of scaling barriers. Each time, the pattern is the same. Something the programmer tracks by hand works in small programs. Systems grow until hand-tracking fails. The solution moves that tracking into the language or the toolchain, and a generation later, nobody can imagine doing it by hand.

Namespaces are the clearest example. Early languages put every name in one global namespace, and the programmer prevented collisions by hand. Collisions were often silent, producing hidden bugs, and third-party libraries made the problem worse. The solution divided names among separate namespaces. In Python, every module is automatically a namespace, and the practice is so settled that the Zen of Python ends by celebrating it: “Namespaces are one honking great idea – let’s do more of those!” Nobody audits their imports for name collisions anymore. The language does the bookkeeping.

The same pattern repeats across the field. Version control gave every state of the code a name to which you can return, so experimenting became safe. Automated testing moved “does it still work?” from a manual ritual into the build. Garbage collection took the tracking of memory ownership out of the programmer’s head. Each of these met resistance as unnecessary overhead, then won adoption, then faded as a question.

Effects are the barrier we are inside right now, and a barrier is hardest to see from inside. We build programs from other people’s code, and we don’t know what that code does. It might change something in the world. It might read from an unreliable source. It might fail and take the system down. You discover these behaviors by trusting documentation, reading source, and observing failures. Then you write compensating code. An enormous share of professional programming is this activity, and it has been normal for so long that it goes unnoticed. Like every hand-tracked concern before it, this one stops scaling.

An Effect Management System moves the bookkeeping into the type system. The function signature answers the questions from Effect Management Systems: what does this function need, what does it change, what can go wrong. The compiler checks every composition, comparing each callee’s Effects with the caller’s declaration at each call.

The languages that do this today are young, and the libraries that retrofit it are demanding. That was true of every solution to every previous barrier at this stage. Namespaces once looked like ceremony. Effect tracking will look obvious in hindsight, and future programmers will regard a function with hidden Effects the way you regard a program written in one global namespace.

Python offers no native Effect tracking beyond async, and will not soon. Generators supplies the mechanism a library needs, Stateless builds a library Effect system on that mechanism, and Stateless in Practice puts the library to work.

Exercises

This chapter’s solutions give a hint, usually the shape of the code, and a full answer for each exercise.

  1. Write the production bindings for ask_tell.py: a Console class whose ask() calls input() and whose tell() calls print(), and run greet(Console(), Console()) interactively. Confirm greet() requires no change. Serving a new context without edits is what delayed binding provides.
  2. Do the bookkeeping the chapter describes. Starting from bookkeeping_scales.py, add a Log Effect (a protocol with log(message)) used by a new helper that greet() calls, and log from greet() too. The chapter counts five signatures for that version. Say how many of the five use the Log they name, and then what an EMS would do instead.
  3. Classify every Effect in slope_catch.py, withdraw() from Foundations, and the Thermometer that keeps a _celsius from Observer: side effect, side cause, or exception. Which of the three conversions from Converting Effectful to Pure applies to the exceptions, and which technique from Effects by Hand applies to the rest?
  4. NonZero guards zero but not negative values, while validate() in slope_catch.py rejects negatives but not zero. Build a PositiveInt that makes both bad values unconstructable, rewrite slope() to take it, and note which checks disappear from slope() as a result.
  5. coroutines_are_descriptions.py shows that async tracks one Effect. Write a synchronous total_price() that calls a helper, then make the helper async and follow what the type checker and the interpreter force you to change, all the way up to asyncio.run(). Name the two properties of a full EMS that async does not have, using the three-item list in Tracking and Management.