4.0 migration guide + R-equivalents argument table (phase 4) (#759)

Author: igerber

CHANGELOG.md

@@ -7,6 +7,38 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 
 ## [Unreleased]
 
+### Added
+- **4.0 migration guide** (`docs/migration-4.0.md`, v4 program Phase 4; the
+  section-9 checklist item 3 obligation): every change that requires action at
+  the 4.0 cut, as a 21-row orientation table plus a 108-row per-symbol appendix
+  carrying the ledger's own `Old`/`New` locators and a one-line fix for each.
+  Also worked before/after examples for the three 3.9 merges, a codemod regex
+  table for the mechanical renames, and a separate "already shipped in 3.9"
+  section for the behaviour changes that are easy to mistake for 4.0 work. The
+  appendix is checked against `docs/v4-deprecations.yaml` in both directions by
+  `tests/test_v4_matrix.py`, so a rescheduled or added ledger row fails CI until
+  the guide is updated. Two hazards the guide calls out explicitly because they
+  are not derivable from the ledger: `robust=` translates to `vcov_type=`
+  differently on each of its four estimators (on `TripleDifference`,
+  `vcov_type="classical"` raises), and post-fit `aggregate()` still raises on
+  bootstrapped fits for five estimator families whose fit-time keyword 4.0
+  removes.
+- **R-equivalents argument mapping table** (`docs/r_comparison.rst`, the
+  section-8 rule-8 obligation): an explicit `yname`/`tname`/`idname`/`gname` ->
+  `outcome`/`time`/`unit`/`first_treat` table, extended across the `did`,
+  `HonestDiD` and `synthdid` mappings the page already evidences with paired
+  code blocks, including the non-1:1 cases (`aggte(type="dynamic")` ->
+  `aggregate(type="event_study")`; HonestDiD's coefficient/vcov inputs carried
+  by the results object). This table is the library's stated alternative to
+  shipping R-spelling parameter aliases, and is now gated by
+  `tests/test_docs_ia.py`.
+
+### Fixed
+- **`docs/r_comparison.rst` migration tips named a nonexistent results field**
+  (`.ci`); the canonical accessor is `.conf_int`. The `aggte()` comparison
+  comment also claimed aggregation is requested at fit time, which stopped being
+  the whole story when post-fit `results.aggregate()` shipped.
+
 ### Added
 - **ChangesInChanges serves both 2x2 distributional estimators** (v4 program
   Phase 3(c); ledger rows [M-015] shimmed, [M-143]):

DEFERRED.md

@@ -127,6 +127,7 @@ decisions (refactor waivers, perf trade-offs, test-infrastructure calls) are rec
 
 | Decision | Location | Verified |
 |----------|----------|----------|
+| **The 4.0 migration guide's code blocks are not snippet-executed.** `tests/test_doc_snippets.py` discovers a hardcoded list of `.rst` files and only `.. code-block:: python` / RST `::` bodies, so `docs/migration-4.0.md` gets no coverage. Deliberate: the guide is a MIXED document - most "after" examples (the renames, `results.att`) run on the current release, but the `field-flip` and `df-convention-flip` examples describe 4.0 behaviour that cannot run until 4.0, so a blanket execution lane would fail by construction. Closing the gap fully means a markdown-fence extractor plus a skip-marker convention for the future-API blocks - a harness change, out of scope for a docs PR. What IS gated: the appendix's ledger parity (`test_migration_guide_*`), which pins the row set and every mechanically checkable cell; and, since the first local review found all three merge examples carrying invalid keywords, `test_migration_guide_examples_bind_to_real_signatures`, which ast-parses the guide's python blocks and asserts every constructor/`fit()` keyword exists on the target signature. That is signature binding, NOT execution - it deliberately skips calls whose owner it cannot resolve (e.g. `results.aggregate(...)`), and it cannot catch a wrong VALUE or a wrong sequence of calls. The hand-written `Fix` prose remains unverifiable by any available means. | `docs/migration-4.0.md`, `tests/test_doc_snippets.py` | Phase 4 / 2026-08-09 |
 | **MultiPeriodDiD deprecation shim loses static constructor-arg checking (3.9 window).** The M-010 shim is `__init__(*args, **kwargs)` + an import-time `__signature__` mirror of DiD's constructor: runtime introspection (get_params/set_params, `inspect.signature`) and eager validation are fully preserved, but static type checkers / IDEs cannot check constructor arguments for the deprecated class until its 4.0 removal. Accepted: the alternative (hand-mirroring ~20 parameters) is a drift magnet on a class with one minor version of remaining life. | `diff_diff/estimators.py` | 3(a) / 2026-08-07 |
 | **DCDH `sklearn.base.clone` param-identity failure won't-fix.** `ChaisemartinDHaultfoeuille._validate_paths_of_interest` unconditionally canonicalizes `paths_of_interest` into a fresh `List[Tuple[int, ...]]`, so sklearn `clone()`'s post-construction `param1 is param2` identity check fails for configured instances - a pre-existing normalization the BaseEstimator mixin PR documented rather than changed (get_params/set_params signatures are clone-compatible; the dependency-free `cls(**est.get_params())` config-equality contract is the enforced one, `tests/test_base_estimator.py`). Fixing would mean returning the caller's raw object from a validator whose job is canonicalization. | `chaisemartin_dhaultfoeuille.py` | mixin PR / 2026-08-01 |
 | **scikit-learn stays out of dev deps; clone-identity tests remain importorskip-only.** The sklearn-`clone()` round-trip tests (`test_base_estimator.py`, had/rdd/cic suites) run only where scikit-learn happens to be installed - deliberate, matching the numpy/pandas/scipy-only dependency posture; the always-running contract is the dependency-free re-instantiation config-equality test. | `tests/test_base_estimator.py` | mixin PR / 2026-08-01 |

TODO.md

@@ -78,7 +78,7 @@ generic sparse-FE, QR+SVD rank-detection redundancy, `check_finite` bypass — m
 |-------|----------|--------|--------|----------|
 | Committed `fixest::feols` event-study golden for TWFE `event_study=True` (within + pooled specs, unbalanced + covariate panels, matched CR1 cluster convention, per-period effects + vcov block) - the in-suite gates are shared-core cross-checks (TWFE-within == MPD-absorb, pooled == MPD bit-exact), so a defect common to the shared core would pass; the live-R harness (`benchmarks/R/benchmark_multiperiod.R`, `feols(y ~ treated * time_f \| unit)`) validated the within design in `docs/benchmarks.rst` but is not a committed regression test - follow the `fixest_did_twfe_golden.json` committed-golden pattern (pytest.skip when absent) | `tests/test_fixest_did_twfe_parity.py`, `benchmarks/R/` | 3(a) R2 | Mid | Medium |
 | Type-blind `n_bootstrap` acceptance in already-validated estimators - HAD bool (`isinstance(..., int)` passes `True`, runs as 1 replicate), dCDH bool+float (its bare `< 0` check passes both `True` and `2.5`), TROP float (`2.5` passes the `>= 2` floor), SyntheticDiD float under all three variance methods + bool/negative under jackknife (its floor check is skipped there) - align these local checks with the `utils.validate_n_bootstrap` type guard (M-081 kept them out of the sweep: it scoped to previously-UNvalidated estimators only) | `diff_diff/had.py`, `diff_diff/chaisemartin_dhaultfoeuille.py`, `diff_diff/trop.py`, `diff_diff/synthetic_did.py` | 2(d) PR-B | Quick | Low |
-| M-020-era CS fit-time `aggregate=` teachings persist in troubleshooting.rst (:215/:241/:244) and choosing_estimator.rst (:243) - CS examples still fit with the deprecated kwarg; migrate to post-fit `results.aggregate('event_study')` (the two HAD examples in the same file were migrated with M-027) | `docs/troubleshooting.rst` | 2(b) PR-4 | Quick | Low |
+| Fit-time `aggregate=` teachings persist across the docs and tutorials (M-020 family, removed at 4.0); migrate to post-fit `results.aggregate(...)`. Re-scoped 2026-08-09 while shipping the migration guide - this is NOT a quick sweep. Narrative docs: `troubleshooting.rst:216`/`:245` both sit on the `n_bootstrap=999` fit at `:213`, and post-fit event-study aggregation **raises `NotImplementedError` on a bootstrapped fit** (`staggered_results.py:323`), so those two need a decision about what to teach before any edit; `choosing_estimator.rst:252` and `python_comparison.rst:416` use the analytical default and can migrate freely; `r_comparison.rst:119-127` needs its `results.event_study_effects`/`.group_effects` reads rebound to the `aggregate()` return values in the same edit (the fields stay `None` after post-fit aggregation). API pages: `docs/api/triple_diff.rst:56`, plus `business_report.rst:77` and `diagnostic_report.rst:59`, which **cannot** migrate today because both report consumers read the raw `event_study_effects` field; `had.rst:164` and `continuous_did.rst:137` are prose references only. Tutorials: **28 executable code-cell sites across 9 notebooks** (`02_staggered_did` 7, `09_real_world_examples` 6, `16_survey_did` 6, `26_composition_drift_calibration` 3, `21_had_pretest_workflow` 2, and one each in `08_triple_diff`, `16_wooldridge_etwfe`, `17_brand_awareness_survey`, `24_staggered_vs_collapsed_power`) - all nbmake-executed; `14_continuous_did` and `15_efficient_did` match only in markdown prose | `docs/troubleshooting.rst`, `docs/choosing_estimator.rst`, `docs/python_comparison.rst`, `docs/r_comparison.rst`, `docs/api/*.rst`, `docs/tutorials/*.ipynb` | 2(b) PR-4 | Heavy | Medium |
 | Evaluate adding the `BaseEstimator` param surface (get_params/set_params) to the exported classes that never had it - `PowerAnalysis`, `LinearRegression`, `BusinessReport`, `DiagnosticReport`, `TWFEWeightsResult` (a NEW public surface, deliberately out of the 2(c)-i pure-refactor scope; `LinearRegression` is the one `fit`-bearing class excluded from the contract suite's roster-completeness test). | `diff_diff/linalg.py`, `diff_diff/power.py` | mixin PR | Mid | Low |
 | Tighten the mypy suppressions that back the enforced-zero posture: burn down `prep_dgp`'s per-module `[index]` override (needs a None-vs-array restructure that preserves the seeded RNG stream), and evaluate re-enabling the globally disabled codes (`arg-type`, `return-value`, `var-annotated`, `assignment`) one at a time — `assignment` alone hid several real annotation drifts found during the 2026-07 triage. | `pyproject.toml` `[tool.mypy]`, `diff_diff/prep_dgp.py` | lint-CI | Mid | Low |
 | MMM interop follow-up: Meridian `roi_calibration_period` mask builder - accept the MMM's time index + channel order and emit the boolean `(n_media_times, n_media_channels)` mask so `.to_code()` scopes the prior to the experiment window automatically (today the caller passes a mask expression / `full_model_window=True`). | `diff_diff/mmm.py` | mmm-interop | Quick | Low |

docs/conf.py

@@ -34,20 +34,20 @@
     "sphinx_design",
 ]
 
-# MyST renders the two in-site methodology markdown pages (REGISTRY.md,
-# REPORTING.md) so cross-refs use :doc: instead of off-site blob/main URLs
-# (stable-docs readers otherwise land on a different revision than their
-# package version). dollarmath/amsmath cover the registry's LaTeX;
-# heading anchors to depth 4 make its GitHub-style #section links resolve.
+# MyST renders the three in-site markdown pages (methodology/REGISTRY.md,
+# methodology/REPORTING.md, migration-4.0.md) so cross-refs use :doc: instead
+# of off-site blob/main URLs (stable-docs readers otherwise land on a different
+# revision than their package version). dollarmath/amsmath cover the registry's
+# LaTeX; heading anchors to depth 4 make its GitHub-style #section links resolve.
 myst_enable_extensions = ["dollarmath", "amsmath"]
 myst_heading_anchors = 4
 
 
 templates_path = ["_templates"]
-# Only the two methodology pages are published; every other repo-internal
-# markdown under docs/ stays out of the build (performance/benchmark notes
-# are deliberately NOT on RTD — see the repo convention — and un-toctree'd
-# .md files would fail the -W build as orphans).
+# Only the two methodology pages and the 4.0 migration guide are published;
+# every other repo-internal markdown under docs/ stays out of the build
+# (performance/benchmark notes are deliberately NOT on RTD — see the repo
+# convention — and un-toctree'd .md files would fail the -W build as orphans).
 exclude_patterns = [
     "_build",
     "Thumbs.db",