Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
47 changes: 47 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,53 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
## [Unreleased]

### Added
- **ContinuousDiD post-fit `aggregate()` - a MIXED view/recompute adopter**
(v4 program 2(b) PR-3c; ledger row [M-025]). `ContinuousDiD.fit(aggregate=)`
is deprecated (`FutureWarning`; removed in 4.0, and the no-underscore
`"eventstudy"` spelling dies with it) in favor of post-fit
`ContinuousDiDResults.aggregate(type)` with the unified vocabulary plus
`"dose"` as this estimator's documented extra level. Unlike every prior
adopter the architecture is MIXED: the dose-response curves, the overall
binarized ATT (`ATT^{loc}` under PT; equals `ATT^{glob}` under SPT) and
`ACRT^{glob}` are ALWAYS computed by `fit()` (`aggregate="dose"` was
a fit-time no-op), so `aggregate('simple')` (2 rows, targets `att`/`acrt`
- the dual-estimand case; `n` = the disjoint treated+control unit total,
`n_kind='units'`) and `aggregate('dose')` (2N target-discriminated rows
over the dose grid) are pure VIEWS relaying stored inference verbatim and
work on ANY fit including bootstrapped ones (the stored FINITE overall
t-stat beside percentile p/CI relays through; the dose rows reproduce
`DoseResponseCurve.to_dataframe` exactly; only the df column is NaN under
bootstrap), while `aggregate('event_study')` RECOMPUTES the binarized
event study from a pruned per-cell influence payload retained on the
fit-built kit (per-(g,t) O(n_treated+n_control) IF-ingredient arrays +
unit-level arrays + the panel-level resolved survey design - on
replicate-weight designs the (n_obs x R) replicate matrix rides along;
the K-dimensional spline machinery is NOT retained, and no panel data
columns or raw unit identifiers are). Replicate-weight designs are
supported post-fit (IF-based replicate variance - no refit replay);
bootstrapped fits carry a scalars-only kit and the event-study route
fails closed (`NotImplementedError` - use the deprecated fit-time route
or re-fit with `n_bootstrap=0`; a seeded replay is a tracked follow-up).
The pre-existing fit-time value validation is retained (unknown strings
still raise `ValueError` after the warning). Post-fit-vs-fit-time
inertness is pinned at 1e-14 across plain/multi-cohort/anticipation/
covariates/survey-TSL/zero-dose-drop/replicate(healthy,dropped,
undefined-df)/discrete/lowest-dose/not-yet-treated/universal-base
designs. This ships the FIRST heterogeneous-`target`
`AggregationResult`, with a rendering amendment: `summary()` gains a
`target` column and a neutral `estimate` heading when targets are mixed,
and `to_dataframe()` orders heterogeneous-target rows by
first-appearance target blocks (att before acrt) with labels ascending
within each block under the existing sortability guard - uniform-target
producers render byte-identically (normative rule added to
`docs/v4-design.md` section 6). `compute_honest_did` /
`compute_pretrends_power` continue to reject ContinuousDiD containers BY
DESIGN (no joint event-study covariance; the binarized bins carry no
reference-period normalization) - both terminal messages now say so.
The event-study machinery moved verbatim to the new leaf module
`diff_diff/continuous_did_aggregation.py` (shared by fit and the
post-fit kit route); the M-092 per-row-df completion row now names
ContinuousDiD alongside EfficientDiD/ImputationDiD.
- **ImputationDiD + TwoStageDiD post-fit `aggregate()` via panel-backed
recompute kits** (v4 program 2(b) PR-3b; ledger rows [M-021]/[M-022]
flip to shimmed, new rows [M-118]/[M-119] claim the reserved balance_e
Expand Down
8 changes: 4 additions & 4 deletions METHODOLOGY_REVIEW.md
Original file line number Diff line number Diff line change
Expand Up @@ -705,7 +705,7 @@ and covariate-adjusted specifications.)

| Field | Value |
|-------|-------|
| Module | `continuous_did.py`, `continuous_did_bspline.py`, `continuous_did_results.py` |
| Module | `continuous_did.py`, `continuous_did_aggregation.py`, `continuous_did_bspline.py`, `continuous_did_results.py` |
| Primary Reference | Callaway, Goodman-Bacon & Sant'Anna (2024), *Difference-in-Differences with a Continuous Treatment*, NBER WP 32117 |
| R Reference | `contdid` v0.1.0 (CRAN) — two parity surfaces at relative tolerance: (a) **scalar overall ATT parity** with raw R `cont_did` / `pte_default` output at `< 0.01` (1%) on all 6 benchmarks; **scalar overall ACRT parity** with raw R `cont_did` at `< 0.01` (1%) on benchmarks 4-5; (b) **harmonized boundary-knot-normalized curve parity** with R-side ATT(d)/ACRT(d) reconstructed under `Boundary.knots = range(treated_doses)` (matching the library) at `< 0.01` max ATT(d) and `< 0.02` max ACRT(d) on benchmarks 1-3 via the benchmark harness (`_run_r_contdid` rebuilds the R-side basis under `Boundary.knots = range(treated_doses)` at `tests/test_methodology_continuous_did.py:333-367`; `_compare_with_r` orchestrates the Python-vs-R comparison at `:395-459`); benchmark 6 is event-study, scalar `overall_att` only (binarized ATT, no curve comparison and no ACRT in event-study mode). Surface (a) is direct raw-package parity; surface (b) is reconstructed-basis parity because raw `contdid` curves use `range(dvals)` instead of `range(dose)`. NOT bit-exact (`atol=1e-8`) like HAD because of the boundary-knots deviation documented below. See `tests/test_methodology_continuous_did.py::TestRBenchmark` |
| Status | **Complete** |
Expand All @@ -715,13 +715,13 @@ and covariate-adjusted specifications.)
- [x] **PT and SPT identification** (CGBS 2024 Assumptions 1-2) — two-level parallel trends with explicit untreated-and-doses conditioning; estimands `ATT(d|d)`, `ATT(d)`, `ACRT(d)`, `ATT^{loc}`, `ATT^{glob}`, `ACRT^{glob}` defined in `docs/methodology/continuous-did.md` § 4 + REGISTRY `## ContinuousDiD` Identification block. Hand-calc coverage: `tests/test_methodology_continuous_did.py::TestLinearDoseResponse` (4 tests at `atol=1e-10` / `atol=1e-6` on no-noise linear DGP — locks the `ATT^{glob}` binarization formula `E[ΔY | D > 0] − E[ΔY | D = 0]`, the `ACRT^{glob}` plug-in average, and the `ATT(d) = 2d`, `ACRT(d) = 2` closed forms).
- [x] **B-spline basis matching `splines2::bSpline`** (cubic and linear degrees, `num_knots=0` default; global boundary knots from the training-dose range, NOT per-cell) — `tests/test_methodology_continuous_did.py::TestQuadraticWithCubicBasis::test_quadratic_recovery` recovers `ATT(d) = d²` at `atol=1e-6` via a degree-3 basis (cubic spline can represent quadratic exactly). The matching basis algorithm lives in `diff_diff/continuous_did_bspline.py` (216 LoC); the boundary-knots deviation from R `contdid` is documented in the Deviations block below.
- [x] **Multi-period (g,t) cell iteration with base period selection** — `TestMultiPeriodAggregation::test_multiple_groups` and `test_gt_cell_count` exercise the cohort iteration on 2-cohort staggered panels; cell counts agree with the R `ptetools`-style convention. Scalar parity with raw R `cont_did` at 1% relative further locks the staggered-aggregation surface via `TestRBenchmark::test_benchmark_4_staggered_dose` and `test_benchmark_5_not_yet_treated` (both assert overall ATT AND overall ACRT at `< 0.01`).
- [x] **Dose-response (`aggregate="dose"`) and event-study (`aggregate="eventstudy"`) aggregation** with group-proportional weights (`n_treated/n_total` per group, divided among post-treatment cells; matches R `ptetools` convention). Two R-side surfaces are exercised: (a) **scalar `overall_att`** via `TestRBenchmark::test_benchmark_1_basic_cubic` / `_2_linear` / `_3_interior_knots` / `_4_staggered_dose` / `_5_not_yet_treated` (dose mode) and `_6_event_study` (event-study mode — binarized ATT only; benchmark 6 validates the event-study code path through the scalar surface, NOT per-horizon `event_study_effects`); (b) **harmonized boundary-knot-normalized ATT(d) / ACRT(d) curves** on benchmarks 1-3 via the benchmark harness — `_run_r_contdid` at `tests/test_methodology_continuous_did.py:333-367` rebuilds the R-side basis under `Boundary.knots = range(treated_doses)` (raw `contdid` curves use `range(dvals)`, so this is reconstructed-basis parity not raw-package parity), and `_compare_with_r` orchestrates the comparison at `:395-459`. Per-benchmark tolerances: all 6 assert overall ATT at `< 0.01` (1%); benchmarks 1-3 additionally assert max ATT(d) at `< 0.01` and max ACRT(d) at `< 0.02` via the helper; benchmarks 4-5 assert overall ACRT at `< 0.01` inline. Per-horizon `event_study_effects` estimates and inference are exercised by Python-side tests at `tests/test_continuous_did.py:557-690` and `:1500-1528` (no R cross-language comparison on the per-horizon surface). Skipped if R / `contdid` not installed via `_check_r_contdid()`; benchmarks use R's `dvals` for exact evaluation-grid alignment between Python and R outputs (boundary knots are harmonized separately under surface (b) — see the `_run_r_contdid` helper's `Boundary.knots = range(treated_doses)` block at `tests/test_methodology_continuous_did.py:333-367`).
- [x] **Dose-response (always computed by `fit()`) and event-study (post-fit `results.aggregate("event_study")`; the fit-time `aggregate="eventstudy"` kwarg is deprecated, row M-025) aggregation** with group-proportional weights (`n_treated/n_total` per group, divided among post-treatment cells; matches R `ptetools` convention). Two R-side surfaces are exercised: (a) **scalar `overall_att`** via `TestRBenchmark::test_benchmark_1_basic_cubic` / `_2_linear` / `_3_interior_knots` / `_4_staggered_dose` / `_5_not_yet_treated` (dose mode) and `_6_event_study` (event-study mode — binarized ATT only; benchmark 6 validates the event-study code path through the scalar surface, NOT per-horizon `event_study_effects`); (b) **harmonized boundary-knot-normalized ATT(d) / ACRT(d) curves** on benchmarks 1-3 via the benchmark harness — `_run_r_contdid` at `tests/test_methodology_continuous_did.py:333-367` rebuilds the R-side basis under `Boundary.knots = range(treated_doses)` (raw `contdid` curves use `range(dvals)`, so this is reconstructed-basis parity not raw-package parity), and `_compare_with_r` orchestrates the comparison at `:395-459`. Per-benchmark tolerances: all 6 assert overall ATT at `< 0.01` (1%); benchmarks 1-3 additionally assert max ATT(d) at `< 0.01` and max ACRT(d) at `< 0.02` via the helper; benchmarks 4-5 assert overall ACRT at `< 0.01` inline. Per-horizon `event_study_effects` estimates and inference are exercised by Python-side tests at `tests/test_continuous_did.py:557-690` and `:1500-1528` (no R cross-language comparison on the per-horizon surface). Skipped if R / `contdid` not installed via `_check_r_contdid()`; benchmarks use R's `dvals` for exact evaluation-grid alignment between Python and R outputs (boundary knots are harmonized separately under surface (b) — see the `_run_r_contdid` helper's `Boundary.knots = range(treated_doses)` block at `tests/test_methodology_continuous_did.py:333-367`).
- [x] **Multiplier bootstrap for inference** (PSU-level multiplier weights on the survey path per Phase 6) — implementation in `diff_diff/continuous_did.py`; bootstrap SE invariant on rank-deficient cells locked in `TestEdgeCasesMethodology::test_all_same_dose` (verifies `dose_response_att.se` is finite on a heterogeneous-outcome / identical-dose DGP); 80 unit tests in `tests/test_continuous_did.py` exercise the rest of the bootstrap path.
- [x] **Analytical SEs via influence functions** (NOT delta method; corrected post-v3.0.0, see Corrections Made) — IF-based variance with `safe_inference()` joint-NaN consistency on all six estimand fields (`overall_att`, `overall_acrt`, dose-response, event-study).
- [x] **Survey support**: weighted B-spline OLS, two-stage linearization (TSL) on influence functions, bootstrap + survey via PSU-level multiplier weights (Phase 3 + Phase 6). Boxed in REGISTRY `## ContinuousDiD` → Implementation Checklist → "Survey design support (Phase 3)" item.
- [x] **`+inf` → `0` never-treated recoding** with `UserWarning` reporting the affected row count (axis-E silent-coercion fix per Phase 2 audit) — the R-style convention of `first_treat = +inf` is normalized internally but no longer absorbed silently. **Any negative `first_treat` value (including `-inf`) raises `ValueError`** with the affected row count. Locked in `tests/test_continuous_did.py`.
- [x] **Zero-`first_treat` rows with nonzero `dose` force-zeroed** with `UserWarning` reporting the affected row count (axis-E silent-coercion fix per Phase 2 audit) — never-treated cells must have `D=0` for internal consistency; the previous silent zeroing is now signaled. Locked in `tests/test_continuous_did.py`.
- [x] **`bspline_derivative_design_matrix` derivative-construction failure warning** (Phase 2 axis-C #12 silent-failures audit fix) — aggregates failed basis indices into a single `UserWarning` naming them, instead of swallowing `scipy.interpolate.BSpline.ValueError` and leaving silently zeroed derivative columns. Both ACRT point estimates AND analytical/bootstrap inference read the same `dPsi` matrix (`continuous_did.py:1026-1046` and the bootstrap ACRT path at `continuous_did.py:1524-1561`), so both are biased on partial-derivative failure — the warning wording makes that explicit. The all-identical-knot degenerate case (single dose value) remains silently handled because derivatives are mathematically zero there. Locked in `tests/test_continuous_did.py::TestBSplineDerivativeDegenerateBasis` (3 tests: `test_single_dose_is_silent`, `test_valueerror_from_bspline_emits_aggregate_warning`, `test_clean_knots_emit_no_warning`); source-level aggregate-warning block at `diff_diff/continuous_did_bspline.py:150-187`.
- [x] **`bspline_derivative_design_matrix` derivative-construction failure warning** (Phase 2 axis-C #12 silent-failures audit fix) — aggregates failed basis indices into a single `UserWarning` naming them, instead of swallowing `scipy.interpolate.BSpline.ValueError` and leaving silently zeroed derivative columns. Both ACRT point estimates AND analytical/bootstrap inference read the same `dPsi` matrix (`_compute_dose_response_gt` - the `acrt_d = dPsi_eval @ beta_pred` point estimate at `continuous_did.py:1806` and the covariate-path IFs at `:1485-1526` - and the bootstrap ACRT path in `_bootstrap_gt_cell` at `continuous_did.py:2274-2340`), so both are biased on partial-derivative failure — the warning wording makes that explicit. The all-identical-knot degenerate case (single dose value) remains silently handled because derivatives are mathematically zero there. Locked in `tests/test_continuous_did.py::TestBSplineDerivativeDegenerateBasis` (3 tests: `test_single_dose_is_silent`, `test_valueerror_from_bspline_emits_aggregate_warning`, `test_clean_knots_emit_no_warning`); source-level aggregate-warning block at `diff_diff/continuous_did_bspline.py:150-187`.
- [x] **Edge cases**: all-same-dose (rank-deficient design, recovers only intercept = `ATT^{glob}`, ACRT = 0 everywhere), single-treated-unit (insufficient for OLS, raises `ValueError` "No valid"), discrete-treatment (detected and warned, saturated regression deferred), rank-deficiency per cell (cell skipped under `rank_deficient_action="silent"` / `"warn"`), balanced-panel-required (matches R `contdid` v0.1.0). Locked in `TestEdgeCasesMethodology` (2 methodology tests) + rank-deficient unit tests in `test_continuous_did.py`.
- [x] **Anticipation-aware not-yet-treated control mask**: when `anticipation > 0`, the not-yet-treated control mask uses `G > t + anticipation` (not just `G > t`) to exclude cohorts in the anticipation window from controls. When `anticipation=0` (default), behavior is unchanged. CHANGELOG `[3.0.x]`-era fix; locked in `test_continuous_did.py`.

Expand Down Expand Up @@ -1464,7 +1464,7 @@ Goldens at `benchmarks/data/r_conleyreg_conley_golden.json`; generator at `bench

**Outstanding Concerns — cross-estimator survey coverage boundary** (intentional, fail-closed `NotImplementedError` deferrals, not bugs; line refs current as of this review — re-grep before relying on them):
- **Conley + survey_design** (open methodological question — no canonical weighted spatial-HAC under probability sampling): `conley.py:298`, `linalg.py:1367` / `linalg.py:3501`, `spillover.py:3246`.
- **Replicate-weight designs** (use TSL strata/PSU/FPC, or `n_bootstrap=0`): `synthetic_did.py:437`, `continuous_did.py:1408`, `staggered_triple_diff.py:701`, `spillover.py:2400`, `bacon.py:526`, `had.py:1751` (+ HAD pretests), `wooldridge.py:89`, `trop.py:444`, `staggered.py:2228` (CS bootstrap), `efficient_did.py:1175`, `chaisemartin_dhaultfoeuille.py:2810`.
- **Replicate-weight designs** (use TSL strata/PSU/FPC, or `n_bootstrap=0`): `synthetic_did.py:437`, `continuous_did.py:2181`, `staggered_triple_diff.py:701`, `spillover.py:2400`, `bacon.py:526`, `had.py:1751` (+ HAD pretests), `wooldridge.py:89`, `trop.py:444`, `staggered.py:2228` (CS bootstrap), `efficient_did.py:1175`, `chaisemartin_dhaultfoeuille.py:2810`.
- **Survey + non-HC1 vcov** — HC2 / HC2-BM / classical **explicitly raise `NotImplementedError`** under `survey_design=` (a fail-closed guard: the survey TSL / replicate-refit variance would otherwise silently discard the requested sandwich family): `stacked_did.py:428`, `sun_abraham.py:751`, `wooldridge.py:702`, `twfe.py:252`.
- **Survey + user `cluster=`** — **explicitly raises** (a fail-closed guard, not a silent drop): the survey TSL / replicate-refit variance would otherwise ignore `cluster=`, so the combination is rejected at construction: `efficient_did.py:518`, `staggered.py:1719`, `imputation.py:314`, `two_stage.py:1439`, `triple_diff.py:674`.
- **SyntheticControl** — no survey support yet (`synthetic_control.py:335`).
Expand Down
2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -91,7 +91,7 @@ For rigorous DiD analysis, follow these 8 steps. Skipping diagnostic steps produ
4. **Choose estimator** - staggered adoption -> CS/SA/BJS (NOT plain TWFE); few treated units -> SDiD; factor confounding -> TROP; simple 2x2 -> DiD. Run `BaconDecomposition` to diagnose TWFE bias.
5. **Estimate** - `estimator.fit(data, ...)`. Always print the cluster count first and choose inference method based on the result (cluster-robust if >= 50 clusters, wild bootstrap if fewer).
6. **Sensitivity analysis** - `compute_honest_did(results)` for bounds under PT violations (MultiPeriodDiD, CS, or dCDH natively; a StackedDiD `results.aggregate('event_study')` container also admits - needs `kappa_pre >= 2`), `run_all_placebo_tests()` for 2x2 falsification, specification comparisons for staggered designs.
7. **Heterogeneity** - CS: `results.aggregate('group')`/`'event_study'` (post-fit, no refit); SA: `results.event_study_effects` / `to_dataframe(level='cohort')`; Stacked: `results.aggregate('event_study')`/`'simple'` post-fit views (surface always computed since 3.9); EDiD: `results.aggregate(...)` post-fit from retained EIFs (3.9); ImputationDiD/TwoStageDiD: `results.aggregate(...)` post-fit from panel-backed kits (3.9); subgroup re-estimation.
7. **Heterogeneity** - CS: `results.aggregate('group')`/`'event_study'` (post-fit, no refit); SA: `results.event_study_effects` / `to_dataframe(level='cohort')`; Stacked: `results.aggregate('event_study')`/`'simple'` post-fit views (surface always computed since 3.9); EDiD: `results.aggregate(...)` post-fit from retained EIFs (3.9); ImputationDiD/TwoStageDiD: `results.aggregate(...)` post-fit from panel-backed kits (3.9); ContinuousDiD: `results.aggregate('dose'/'simple'/'event_study')` post-fit (3.9; dose/simple are views, event_study recomputes); subgroup re-estimation.
8. **Robustness** - compare 2-3 estimators (CS vs SA vs BJS), report with and without covariates (shows whether conditioning drives identification), present pre-trends and sensitivity bounds.

Full guide: `diff_diff.get_llm_guide("practitioner")`.
Expand Down
Loading
Loading