Chapter 6

Modules and Packages

A program outgrows one file, and the pieces must find each other by name.

Each Python file is a module you can import into another Python file. If the file is in the same directory, you import it by its file name, without the .py (PYTHONPATH covers where else Python looks):

# module.py

def useful_function():
    return "I'm being useful!"

# use_module.py
import module

print("'module' imported")
#: 'module' imported

if __name__ == "__main__":
    print(module.useful_function())
#: I'm being useful!

Importing a module makes its namespace, the set of names the module defines, reachable in the importing file under the module’s name. Reaching those names through the module’s name keeps them from clashing with the local ones. To call useful_function(), you must qualify it with the name of the module: module.useful_function().

The code at the end of use_module.py starts with an if clause that checks whether the standard variable __name__ equals the string "__main__". Any identifier that begins and ends with double underscores (commonly called a “dunder”) is special in some way. Dunder methods, for example, connect your class to the language’s operators and built-in functions. Classes shows how print() and str() reach a class’s __str__().

The if exists because you can also use any file as a library module within another program. In that case, you want its definitions and none of the code at the bottom of the file. Python programmers call this check the main guard. The condition is true only when you run this file as a script. That is, __name__ is "__main__" when you use the command line:

python use_module.py

However, if another program imports use_module.py as a module, __name__ is "use_module" instead, so the "__main__" block does not run. The next program consists of one statement, import use_module:

# import_module.py
import use_module
#: 'module' imported

Importing use_module runs its top-level code, including the print(), but not its "__main__" block. Python runs that code once, however many times the program imports the module:

# import_once.py
import sys
import use_module
import use_module as second

#: 'module' imported
print(use_module is second)
#: True
print(sys.modules["use_module"] is use_module)
#: True

use_module’s body runs once, although two import statements name it. The first import stores the finished module object in sys.modules, a dict keyed by dotted module name. Every later import of that name, from any file in the program, finds it there and binds the same object instead of re-running the file, so use_module and second are one object. One consequence: editing a module’s file changes nothing in a program that has imported it. Restart the program, or call importlib.reload(use_module), which re-runs the body into the existing module object. Reloading leaves every name bound by a from ... import pointing at the old objects, so restarting is the reliable choice.

Importing Names with `from` and `as`

To bring a name into the current namespace, use the from keyword:

# using_from.py
from module import useful_function

if __name__ == "__main__":
    print(useful_function())
#: I'm being useful!

from works like an assignment. It binds a name in this file to the object the module’s name refers to at that moment. The two names are separate, so rebinding the module’s name afterward leaves this file’s name as it was:

# app_settings.py

debug: bool = False

def show() -> None:
    print(f"app_settings.debug is {debug}")

# from_snapshot.py
import app_settings
from app_settings import debug

app_settings.debug = True
print(debug, app_settings.debug)
#: False True
app_settings.show()
#: app_settings.debug is True

app_settings.debug is now True, and show() sees the new value because it looks the name up in its own module every time it runs. The local debug is a separate binding that from set once, at import, so it still holds False.

Import the module and write app_settings.debug when the value can change. Use from ... import for names that keep the same value, such as functions and classes.

The as keyword binds the imported module to a name you choose:

# using_as.py
import module as m

if __name__ == "__main__":
    print(m.useful_function())
#: I'm being useful!

The Module Namespace

A module’s namespace is an ordinary dict you can read and write. globals() returns it, the same dict Python searches when it looks up a top-level name. A dotted name reads that same dict. module.__dict__ from outside is the same object globals() returns inside module, so module.useful_function() and a top-level lookup inside module.py find the same function. Assigning into that dict works like writing the assignment as a statement:

# globals_demo.py
x = 10
print(globals()["x"])
#: 10

globals()["y"] = 42
print(y)  # type: ignore  # noqa: F821
#: 42

The two directives on the print(y) line silence the type checker and the linter, neither of which can see a name that appears only as a dict key. Losing both checks on that name is the cost of creating names this way, and a reason to keep it rare. Assigning into globals() matters whenever code needs to define a module-level name known only at runtime, such as a class built dynamically under a computed name. Metaprogramming builds classes that way, though it collects them in a dictionary instead of in the module’s namespace.

Packages

As your programs get larger, you further organize your code into packages. A package is a directory that contains multiple modules, and it forms its own namespace with the name of that directory.

To make a directory a package, you put a special file named __init__.py in it. __init__.py runs once, the first time a program imports the package or any module inside it. An empty __init__.py, the common case, only flags the directory as a package.¹ An __init__.py with content usually re-exports the package’s public names. If a_package/__init__.py re-exports function1, from a_package import function1 works without callers naming the submodule that defines function1. You can still import a directory without __init__.py as a namespace package, but an explicit __init__.py makes the package’s identity and boundary clear, so this book uses one by default.

To explore packages, create a directory called a_package whose __init__.py announces itself, so the markers below show when it runs:

# a_package/__init__.py

print("initializing a_package")

Now add two modules to the package:

# a_package/module1.py

print("importing module1 in a_package")

def function1():
    return "function1 in module1 in a_package"

# a_package/module2.py

print("importing module2 in a_package")

def function2():
    return "function2 in module2 in a_package"

To import a module from a package, you must qualify it with the package name:

# using_packages.py
import a_package.module1
import a_package.module2

#: initializing a_package
#: importing module1 in a_package
#: importing module2 in a_package
print("a_package" in dir(), "module1" in dir())
#: True False
print(a_package.module1.function1())
#: function1 in module1 in a_package
print(a_package.module2.function2())
#: function2 in module2 in a_package

import a_package.module1 binds only the name a_package in this file. Loading module1 also stores that submodule as an attribute of a_package, so a_package.module1.function1() resolves. The shorter module1.function1() fails here, since nothing binds module1 in this file. module1.function1() works in from_packages.py, below, where from binds module1 in that file.

Importing the package alone does not import what is inside it:

# package_only.py
import a_package

#: initializing a_package
print(hasattr(a_package, "module1"))
#: False

No submodule loads, and no message prints beyond the package’s own. a_package.module1.function1() here raises AttributeError: module 'a_package' has no attribute 'module1'. The submodule becomes an attribute of the package only when something imports it by name.

You can also name the package with from:

# from_packages.py
from a_package import module1, module2

#: initializing a_package
#: importing module1 in a_package
#: importing module2 in a_package
print(module1.function1())
#: function1 in module1 in a_package
print(module2.function2())
#: function2 in module2 in a_package

Here you no longer need to qualify the module with the package name.

You can bring specific functions into the namespace by naming both the package and the module:

# no_qualification.py
from a_package.module1 import function1
from a_package.module2 import function2

#: initializing a_package
#: importing module1 in a_package
#: importing module2 in a_package
print(function1())
#: function1 in module1 in a_package
print(function2())
#: function2 in module2 in a_package

no_qualification.py, from_packages.py, and using_packages.py print the same loading messages, because from loads what import loads. The whole module runs either way. The statement decides which names this file binds.

Nested Packages

You can put a second package underneath the first one:

# a_package/b_package/__init__.py

print("initializing b_package")

# a_package/b_package/module3.py

print("importing module3 in b_package")

def function3():
    return "function3 in module3 in b_package"

To import module3 you must specify both packages:

# two_levels.py
from a_package.b_package import module3

#: initializing a_package
#: initializing b_package
#: importing module3 in b_package
print(module3.function3())
#: function3 in module3 in b_package

Imports Within a Package

The listings so far import a package’s modules from outside the package. A package’s own modules also import each other, and there the leading dot of a relative import means “the package containing this module”:

# a_package/module4.py
from .module1 import function1

def function4():
    return f"function4 calls {function1()}"

# use_module4.py
from a_package.module4 import function4

#: initializing a_package
#: importing module1 in a_package
print(function4())
#: function4 calls function1 in module1 in a_package

module4 finds its sibling without naming a_package, so renaming the package breaks nothing inside it. Two dots reach the parent package, so b_package/module3.py could import from a_package with from .. import module1 or from ..module1 import function1. The absolute form from a_package.module1 import function1 works from inside the package too, and is the better choice when the import crosses a package boundary. Keep relative imports for a package’s own submodules.

A relative import needs a package context, and a file run as a script has none. Running python a_package/module4.py raises ImportError: attempted relative import with no known parent package. Run it as python -m a_package.module4 instead.

Circular Imports

Two modules in a package can end up importing each other. Python places the first one to load in sys.modules before its body finishes, so a from import in the second finds a partially initialized module and fails with ImportError: cannot import name ... (most likely due to a circular import). A plain import of that same module succeeds at this point, since it needs the module to exist in sys.modules, not to have finished running. The failure then surfaces later, as an AttributeError, wherever the second module first uses a name the first module has not defined yet.

The “circular import” wording appears when the module’s file comes from anywhere but the directory of the script you ran, which includes every module inside a package. When the file sits in that directory (sys.path[0]), the usual case for two modules beside your script, Python suspects a name collision with a library instead of a cycle: ImportError: cannot import name 'f' from 'modx' (consider renaming '.../modx.py' if it has the same name as a library you intended to import). Python names the offending file by its full path, abbreviated here as ....

A cycle is a design signal. Move the shared piece into a third module both can import.

When the cycle exists only in annotations, an if TYPE_CHECKING: import breaks it. typing.TYPE_CHECKING is False at runtime, so the import under it runs for the type checker alone. The annotations still work at runtime, because Python does not evaluate them at import time. A robot in a maze imports Room that way, and every use of the name is an annotation.

What a Module Exports

Every name at a module’s top level is importable, because Python has no private keyword. A module states its boundary by convention and by one optional list.

A leading underscore marks a name as internal. It is a signal to a reader rather than a barrier. For a module accounting that defines _Engine, accounting._Engine still resolves, and from accounting import _Engine still works. The underscore changes one mechanical thing: from accounting import * skips every name that begins with one.

__all__ states the export list explicitly. You assign it at module level as a list of strings naming the public names, and from module import * then imports those names and no others. A name listed in __all__ arrives even when it begins with an underscore. An __all__ also gathers the public names into one readable place, and documentation tools read it to report what a module offers.

# exporting.py

__all__ = ["public", "helper"]

def public():
    return "public"

def helper():
    return "helper"

def _internal():
    return "internal"

def undeclared():
    return "undeclared"

# star_import.py
from exporting import *  # noqa: F403

print(sorted(n for n in dir() if not n.startswith("__")))
#: ['helper', 'public']

Without __all__, the star import binds public, helper, and undeclared: everything not underscored. With it, only the listed names arrive, and the star import skips _internal either way.

The # noqa: F403 silences the linter’s objection to the star import, which hides the source of each name from a reader and from the tools. Name what you import, and keep the star for experiments at the interactive prompt.

Neither the underscore nor __all__ stops module._name. Both say which names a caller should use, and that agreement lets a module serve as a façade over its internals or as a shared single instance (Singleton).

File Names

To be the target of an import statement, a module’s file name must be a valid Python identifier and not a keyword. import my-mod is a syntax error. (importlib.import_module("my-mod") still loads the file, and plugin loaders reach oddly named files that way, but give your own modules importable names.)

Modules (.py files): short, all-lowercase, with underscores between words if that improves readability, the style called snake_case.

Good: result.py, cache_singleton.py, list_comprehension.py
Avoid: Result.py (CapWords is for classes), cacheSingleton.py (camelCase), cache-singleton.py (hyphens aren’t importable)

Packages (directories with __init__.py): also short and all-lowercase, but convention discourages underscores. Prefer a single run-together word when you can.

Good: mypackage, and underscores only when they genuinely help (a_package)

Tests follow pytest’s discovery convention: test_*.py (or *_test.py). This book uses test_*.py, e.g. test_result.py.

Don’t shadow standard-library modules. A file named random.py, string.py, or weakref.py can hide the stdlib one and break imports, because Python searches the script’s directory before the standard library. Give a shared module a distinctive name for the same reason. The first config.py imported anywhere in the process is the one every later import config gets.

Windows and macOS default to case-insensitive filesystems, so module.py and Module.py name the same file there. Python’s import system still matches names case-sensitively by default, so import module fails to find a file saved as Module.py, even on a filesystem that treats the two names as one. PYTHONCASEOK turns that case check off, so import module finds Module.py again, bringing back the confusion the check exists to prevent. Keep a module’s file name and its import spelled with the same case, rather than relying on either behavior.

`PYTHONPATH`

Python searches sys.path, a list of directories it builds at startup. print(sys.path) shows it. Its first entry is the directory of the script you ran (the current directory when you use -m or the REPL), and that entry is why use_module.py can import module with no setup. The entries from PYTHONPATH come next, and installed packages sit further down. Running with -P drops that first entry, so a local random.py no longer shadows the standard library.

What if your module or package isn’t in the same directory as the Python file that imports it? The original solution was the PYTHONPATH environment variable, which tells Python where to look for modules and packages. PYTHONPATH takes multiple paths, which Python adds to sys.path in the order given. Python searches sys.path from the front and uses the first directory that holds your module or package. It raises a ModuleNotFoundError when no directory does.

PYTHONPATH still works, but today you install your package into the environment you use, and the install puts it on the search path with no environment variable. With uv (this book’s tool of choice), you run uv sync, or uv pip install -e . for an editable install. The package resolves by name from anywhere, and edits to its source take effect immediately, without reinstalling.

Lazy Imports

Every import so far runs the target module’s top-level code immediately, and that is why importing a_package.module1 prints its message as it loads. For a large program that imports many modules but uses only some of them on any given run, that eager work slows startup.

Python 3.15 (PEP 810) adds the lazy soft keyword: a keyword only inside an import statement, the way match is a keyword only inside a match statement. A lazy import defers loading the module until the first time you use the imported name, so a run pays only for the modules it uses, while all imports stay at the top of the file:

# lazy_imports.py
lazy import json
lazy from pathlib import Path

print(json.dumps({"a": 1}))
#: {"a": 1}
print(Path("report/data.txt").suffix)
#: .txt

A lazy import looks like an ordinary one, with lazy in front, and once loaded the names behave like eagerly imported ones. json and pathlib load at the json.dumps() and Path(...) calls. The output is the same either way, so this listing cannot show the deferral. lazy_noisy.py, in Watching the Deferral, does.

Deferring an Import Before 3.15

Before 3.15, deferring a costly import meant moving it inside the function that needed it. That works, but it hides the dependency: nothing at the top of the file mentions the module, tools that read imports miss it, and the import statement re-runs its sys.modules lookup on every call. lazy import keeps the declaration at the top where a reader and a tool can see it, and pays the loading cost once, at first use.

Packages deferred submodule loading before lazy import existed, by giving __init__.py a module-level __getattr__() (PEP 562) that imports and returns a submodule the first time a caller asks for it by name. NumPy and SciPy use that technique to keep import time low. lazy import needs no hand-written __getattr__(), and defers any imported name, not only a package’s submodules. The __getattr__() pattern still matters for code that must run on a Python older than 3.15.

Watching the Deferral

You can watch lazy defer the load by importing a module whose body prints when it runs:

# noisy.py

print("noisy module loaded")

def announce():
    print("noisy.announce() called")

# lazy_noisy.py
lazy import noisy

print("before first use")
#: before first use
noisy.announce()
#: noisy module loaded
#: noisy.announce() called
print("after first use")
#: after first use

Nothing loads at the lazy import line. The body of noisy runs at noisy.announce(), the first access, so noisy module loaded prints after before first use. If a lazily imported module is missing or broken, the error surfaces at that first use rather than at the import line.

sys.lazy_modules holds the names still waiting to load, but the standard-library modules CPython loads at startup put lazy imports of their own into that set before your code runs:

# lazy_modules_check.py
import sys
lazy import noisy

print(len(sys.lazy_modules) > 1)
#: True
print("noisy" in sys.lazy_modules)
#: True
noisy.announce()
#: noisy module loaded
#: noisy.announce() called
print("noisy" in sys.lazy_modules)
#: False

This script marks one import lazy and the set holds more than one name, so sys.lazy_modules is not a clean “what my program deferred” list. noisy leaves the set once noisy.announce() loads it, so the set tracks names still waiting, not names your program ever deferred. Check it for a specific name you marked lazy, rather than reading the whole set.

Limits of `lazy`

lazy works with both import and from ... import, but only at module scope. Using it inside a function, a class body, or a try block is a SyntaxError, and Python likewise rejects lazy from module import * and a lazy from __future__ import.

To change which imports are lazy for a whole run without editing source, run with -X lazy_imports=MODE or set PYTHON_LAZY_IMPORTS=MODE. Both accept one of two values. normal, the default, defers only the imports you marked lazy. all defers every module-level import the keyword can mark, so the imports the keyword cannot mark, such as one inside a try block, stay eager. The PEP also describes a third value, none, a global off switch. CPython removed it before the 3.15 release.

Don’t make an import lazy when you import a module for what its body does rather than for a name it defines. A module that registers a plugin class, installs a codec, or fills a table does that work as it loads. If nothing touches the lazily imported name, the module stays unloaded and its registration is missing. The failure is silent: no error, just a table with a row missing. That silent failure is also why all is an experiment to run rather than a setting to leave on, since it defers ordinary import statements too, including the ones whose only purpose is to run the module.

Exercises

This chapter’s solutions give a hint, usually the shape of the code, and a full answer for each exercise.

Add a fourth module, a_package/module5.py, with its own function5() and a top-level print() so the module announces itself when it loads. Import it three ways, using import a_package.module5, from a_package import module5, and from a_package.module5 import function5, and confirm the loading message prints once, however many of the three you use together.
Add a_package/b_package/module6.py with a function6() that calls function5() from module5. Import and call function6() from a script outside a_package, then rename b_package to bPackage (and rename it back afterward) and explain, from the rules in File Names, why that name is a poor choice although the import still works.
Write a small module noisy2.py whose top-level body prints a message, like noisy.py. In a new script, lazy import both noisy and noisy2, then use noisy2 before noisy. Confirm the two loading messages print in the order you used the modules, not the order you wrote the lazy import lines.
Rename module.py to Module.py, change use_module.py to import Module, and update its call to Module.useful_function(). Run it. Then change the import back to import module, leaving the file named Module.py, and run it again. Predict the result before you run it, then explain what you see, given that Windows and macOS open module.py and Module.py as the same file. Look up PYTHONCASEOK to confirm your explanation.
Change a_package/module4.py to the absolute import from a_package.module1 import function1 and confirm use_module4.py still works. Then run python a_package/module4.py directly, both before and after the change. Both fail, with different errors: explain each, and say why python -m a_package.module4 works either way.
Remove the __all__ line from exporting.py. Predict what star_import.py prints without it, run it to check, then restore the line.
Give a module a top-level list, plugins = [], and bring the list into a script with from ... import plugins. Append to the list through the module’s name, then print the script’s plugins. Rebind the module’s name to a new list, append to that one, and print both. Explain why the first change reaches the script’s name and the second does not.