docs: add architecture overview docs

AGENTS.md

@@ -102,6 +102,12 @@ Look at these before writing code:
 
 - Use single backticks around function and class names in markdown (e.g. `req_ctxvar_context()`, `WorkContext`), double backticks in .rst (reStructuredText)
 - Use Sphinx `versionadded`, `versionremoved`, `versionchanged` directives for user-facing changes (get next version from last git tag)
+- When code changes affect the architecture described in `docs/concepts/`, update the relevant doc:
+  - `architecture-overview.rst` — major subsystems, data flow, extension points, key data structures
+  - `bootstrapper-architecture.rst` — phase pipeline, class hierarchy, bootstrapper-phase interaction
+  - `resolver-architecture.rst` — resolution strategies, provider hierarchy, version filtering
+  - `hooks-and-overrides.rst` — override hook points, global hooks, plugin discovery
+  - `package-settings.rst` — settings loading flow, merge order, PackageBuildInfo facade
 
 ## Commit Messages
 

docs/concepts/architecture-overview.rst

@@ -0,0 +1,150 @@
+Architecture Overview
+=====================
+
+Fromager rebuilds complete dependency trees of Python wheels from
+source.  This document maps the major subsystems and how they connect.
+
+.. seealso::
+
+   :doc:`bootstrap-vs-build` explains the two operating modes.
+   :doc:`bootstrapper-architecture` details the bootstrap engine.
+   :doc:`resolver-architecture` covers version resolution.
+   :doc:`hooks-and-overrides` describes the two plugin systems.
+   :doc:`package-settings` explains the settings layering.
+
+Major Subsystems
+----------------
+
+.. code-block:: text
+
+   ┌───────────────────────────────────────────────────────┐
+   │                    CLI & Context                      │
+   │         command parsing, WorkContext, settings        │
+   └───────────────────────────┬───────────────────────────┘
+                               │
+                               ▼
+   ┌───────────────────────────────────────────────────────┐
+   │                  Bootstrap Engine                     │
+   │        iterative DFS loop over phase pipeline         │
+   │                                                       │
+   │   ┌─────────────┐  ┌─────────────┐  ┌─────────────┐   │
+   │   │ Resolution  │  │    Source   │  │     Build   │   │
+   │   │ PyPI, graph │  │ Acquisition │  │   System    │   │
+   │   │  git URLs   │  │  download,  │  │  isolated   │   │
+   │   │             │  │ patch, Rust │  │   envs,     │   │
+   │   │             │  │   vendor    │  │   wheels    │   │
+   │   └─────────────┘  └─────────────┘  └─────────────┘   │
+   └───────────────────────────┬───────────────────────────┘
+                               │
+              ┌────────────────┴────────────────┐
+              ▼                                 ▼
+   ┌──────────────────┐              ┌──────────────────┐
+   │  Per-Package     │              │   Global Hooks   │
+   │  Overrides       │              │ post_bootstrap,  │
+   │                  │              │    post_build    │
+   └──────────────────┘              └──────────────────┘
+
+The **bootstrap engine** orchestrates the other subsystems.  For each
+package it resolves a version, downloads the source, builds a wheel,
+and recurses into dependencies.  **Resolution**, **source acquisition**,
+and **build** are called as needed during each phase of the pipeline.
+
+**Extension points** intercept the pipeline at specific steps, allowing
+per-package overrides (e.g. custom download logic) and global hooks
+(e.g. post-build notifications).
+
+Data Flow
+---------
+
+A ``fromager bootstrap`` invocation flows through the subsystems in
+this order:
+
+.. code-block:: text
+
+   requirements.txt
+                               │
+                               ▼
+   ┌─ CLI & Context ───────────────────────────────────────┐
+   │  parse args, load settings, create WorkContext        │
+   └───────────────────────────┬───────────────────────────┘
+                               │
+                               ▼
+   ┌─ Bootstrap Engine (iterative DFS) ────────────────────┐
+   │                                                       │
+   │  ┌─────────────┐   ┌─────────────┐                    │
+   │  │   Resolve   │──►│  Download   │                    │
+   │  │   version   │   │   source    │                    │
+   │  └─────────────┘   └──────┬──────┘                    │
+   │                           │                           │
+   │                           ▼                           │
+   │  ┌─────────────┐   ┌─────────────┐                    │
+   │  │  Extract &  │◄──│    Build    │                    │
+   │  │  recurse    │   │    wheel    │                    │
+   │  │  into deps  │   │             │                    │
+   │  └─────────────┘   └─────────────┘                    │
+   │                                                       │
+   └───────────────────────────┬───────────────────────────┘
+                               │
+                               ▼
+                        ┌── Outputs ──┐
+                        │ graph.json  │
+                        │ build-order │
+                        │ wheels/     │
+                        │ constraints │
+                        └─────────────┘
+
+The ``build`` command uses the same source acquisition and build
+subsystems but skips resolution and recursion -- it compiles a
+single package given a name, version, and source URL.
+
+Extension Points
+----------------
+
+Fromager has two plugin systems:
+
+.. list-table::
+   :header-rows: 1
+   :widths: 25 35 40
+
+   * - System
+     - Scope
+     - When it fires
+   * - **Per-package overrides**
+     - One package
+     - At each pipeline step (download, build sdist, build wheel,
+       dependency extraction, etc.)
+   * - **Global hooks**
+     - All packages
+     - After specific events (``post_bootstrap``, ``post_build``,
+       ``prebuilt_wheel``)
+
+Per-package overrides replace the default implementation of a step for
+a specific package.  Global hooks run in addition to the default logic
+for every package.  See :doc:`hooks-and-overrides` for the full
+breakdown.
+
+Key Data Structures
+-------------------
+
+Four data structures flow between subsystems:
+
+.. list-table::
+   :header-rows: 1
+   :widths: 30 70
+
+   * - Structure
+     - Role
+   * - ``WorkContext``
+     - Central configuration and state: directory paths, constraints,
+       dependency graph, settings, and variant info.  Passed to every
+       subsystem.
+   * - ``DependencyGraph``
+     - In-memory directed graph of all resolved dependencies.
+       Serialized to ``graph.json``.  Thread-safe.
+   * - ``PackageBuildInfo``
+     - Per-package build configuration: environment variables, patches,
+       resolver settings, and build options.  Derived from settings
+       files.
+   * - ``BuildEnvironment``
+     - Isolated virtual environment for building one package.  Created
+       per source build, cleaned up after completion.

docs/concepts/bootstrapper-architecture.rst

@@ -0,0 +1,106 @@
+Bootstrapper Architecture
+=========================
+
+The bootstrap command uses an iterative depth-first loop to resolve and
+build an entire dependency tree.  This document describes the phase
+pipeline, class hierarchy, and interaction model at a high level.
+
+.. seealso::
+
+   :doc:`architecture-overview` maps the major subsystems.
+   :doc:`bootstrap-vs-build` covers the difference between bootstrap and
+   build modes.
+
+Bootstrap Phase Flow
+--------------------
+
+Every package passes through a sequence of phases.  Source packages
+traverse the full pipeline; prebuilt wheels skip the build phases.
+
+.. code-block:: text
+
+   RESOLVE ──► START ──► PREPARE_SOURCE ─┬─► PREPARE_BUILD ──► BUILD ─┐
+                  │                      │                            │
+                  │            (prebuilt)└────────────────────────────┤
+                  │                                                   │
+                  │                  ┌────────────────────────────────┘
+                  │                  ▼
+                  │          PROCESS_INSTALL_DEPS ──► COMPLETE
+                  │
+             (already seen) ──► drop
+
+Each phase returns a list of new items to push onto the work stack.
+Dependency-discovery phases (``PREPARE_SOURCE``, ``PREPARE_BUILD``,
+``PROCESS_INSTALL_DEPS``) also emit ``RESOLVE`` items for newly
+discovered dependencies, which drives the recursive traversal.
+
+Phase Class Hierarchy
+---------------------
+
+All phases inherit from the ``Phase`` abstract base class, which defines
+the contract every phase must satisfy:
+
+- ``run(bt)`` — execute the phase logic; return new items for the stack.
+- ``background_work(bt)`` — optionally return a callable for background
+  I/O (used by ``Resolve`` and ``PrepareSource``).
+- ``requires_exclusive_run`` — return ``True`` to drain the thread pool
+  before running (used by ``Build`` for exclusive builds).
+
+.. list-table::
+   :header-rows: 1
+   :widths: 25 75
+
+   * - Phase
+     - Role
+   * - ``Resolve``
+     - Resolve available versions; fan out one ``Start`` per version
+   * - ``Start``
+     - Add to dependency graph; deduplicate already-seen packages
+   * - ``PrepareSource``
+     - Download source or prebuilt wheel; read build-system deps
+   * - ``PrepareBuild``
+     - Install build-system deps; discover backend and sdist deps
+   * - ``Build``
+     - Build sdist and/or wheel; update the local mirror
+   * - ``ProcessInstallDeps``
+     - Run post-bootstrap hooks; extract install deps; record build order
+   * - ``Complete``
+     - Clean up build directories (terminal phase)
+
+Each phase wraps a ``WorkItem`` dataclass that accumulates per-package
+state (resolved version, build environment, build result, etc.) as it
+moves through the pipeline.
+
+Bootstrapper and Phase Interaction
+----------------------------------
+
+The ``Bootstrapper`` class owns the work stack and thread pool.  It runs
+an iterative DFS loop:
+
+.. code-block:: text
+
+   ┌─────────────────────────────────────────────────────┐
+   │                  Bootstrapper Loop                  │
+   │                                                     │
+   │  while stack:                                       │
+   │      item = stack.pop()                             │
+   │                                                     │
+   │      if item.requires_exclusive_run:                │
+   │          drain thread pool                          │
+   │                                                     │
+   │      new_items = item.run(self)                     │
+   │                                                     │
+   │      for each new_item in new_items:                │
+   │          stack.push(new_item)                       │
+   └─────────────────────────────────────────────────────┘
+
+The ``Bootstrapper`` provides services that phases call back into during
+``run()``: dependency graph updates, seen-set tracking, build-order
+recording, and work-item creation for discovered dependencies.
+
+Some phases (``Resolve`` and ``PrepareSource``) perform background I/O
+in a thread pool.  When new items are pushed onto the stack, the
+``Bootstrapper`` submits their ``background_work()`` immediately so I/O
+overlaps with the main thread processing other items.  Each phase is
+responsible for blocking on its own ``bg_future`` inside ``run()`` when
+it needs the result.  Other phases run entirely on the main thread.

docs/concepts/hooks-and-overrides.rst

@@ -0,0 +1,84 @@
+Hooks and Overrides
+===================
+
+Fromager has two plugin systems that serve different purposes:
+**per-package overrides** replace default behavior for a specific
+package, while **global hooks** broadcast notifications after events
+for every package.
+
+.. seealso::
+
+   :doc:`architecture-overview` maps the major subsystems.
+   :doc:`/reference/hooks` documents each override hook's arguments and
+   return values.  :doc:`/customization` covers global hooks with code
+   examples.
+
+Comparison
+----------
+
+.. list-table::
+   :header-rows: 1
+   :widths: 20 40 40
+
+   * - Aspect
+     - Per-Package Overrides
+     - Global Hooks
+   * - **Scope**
+     - One named package
+     - Every package
+   * - **Cardinality**
+     - At most one override module per package
+     - Multiple plugins per hook
+   * - **Semantics**
+     - Replaces the default implementation
+     - Runs in addition to the default
+   * - **Return value**
+     - Used by the caller
+     - None (fire-and-forget)
+   * - **Typical use**
+     - Custom download, build, or resolution logic
+     - Publishing wheels, validation, logging
+
+Per-Package Overrides
+---------------------
+
+An override module provides alternative implementations for specific
+build steps.  Fromager looks up the module by the canonicalized package
+name.  If the module defines the requested method, it is called instead
+of the default; otherwise the default runs.
+
+Override hooks cover resolution, source acquisition, building,
+dependency extraction, and build environment customization.  Third-party
+packages register overrides via the ``fromager.project_overrides``
+entry-point group in their ``pyproject.toml``, mapping a package name to
+a Python module.
+
+See :doc:`/reference/hooks` for the complete list of hooks and their
+arguments.
+
+Global Hooks
+------------
+
+Global hooks are event callbacks that fire for every package.  All
+registered plugins for a hook are called sequentially.  They cannot
+alter the build result — they are purely side-effecting notifications.
+
+.. list-table::
+   :header-rows: 1
+   :widths: 25 75
+
+   * - Hook
+     - When it fires
+   * - ``post_build``
+     - After a wheel is built from source (not for prebuilt wheels)
+   * - ``prebuilt_wheel``
+     - After a prebuilt wheel is downloaded (not built from source)
+   * - ``post_bootstrap``
+     - After a package is bootstrapped, before its install dependencies
+       are processed
+
+Third-party packages register hooks via the ``fromager.hooks``
+entry-point group in their ``pyproject.toml``, mapping a hook name to
+a callable.
+
+See :doc:`/customization` for examples and argument details.

docs/concepts/index.rst

@@ -8,13 +8,23 @@ Key Topics
 
 These guides explain the fundamental concepts and design principles behind fromager:
 
+* **Architecture Overview** - Major subsystems, data flow, extension points, and key data structures
 * **Bootstrap vs Build Modes** - Understand the difference between recursive discovery (bootstrap) and single-package builds (build)
+* **Bootstrapper Architecture** - Phase pipeline, class hierarchy, and interaction model of the bootstrap engine
+* **Resolver Architecture** - Resolution strategies, provider hierarchy, version filtering, and per-package configuration
+* **Hooks and Overrides** - The two plugin systems: per-package overrides and global hooks
+* **Package Settings** - Settings loading, merge order, and the ``PackageBuildInfo`` facade
 * **Dependency Types** - Learn about build-system, build-backend, build-sdist, and install dependencies
 
 .. toctree::
    :maxdepth: 1
 
+   architecture-overview
    bootstrap-vs-build
+   bootstrapper-architecture
+   resolver-architecture
+   hooks-and-overrides
+   package-settings
    dependencies
 
 Related Practical Guides