Fail the suite on causalml's own arg-order FutureWarning (#995)

[jeongyoonlee]

Proposed changes

Closes #995.

The suite has no warnings-as-errors lane. There is no filterwarnings setting in pyproject.toml, setup.cfg, tests/conftest.py or any workflow, so a FutureWarning raised during a test run is printed and the run stays green.

That matters for the #854 shim specifically. It ships in 0.18.0 and stays live through 0.19.0 and 0.20.0 before the flip in v1.0 — about nine months. Both internal-positional-call defects so far (#988, #989) were found by running the suite by hand with -W error::FutureWarning, and each was then pinned by a bespoke "does not warn" test written after the fact. Coverage is therefore by enumeration: it protects the three paths someone thought to protect.

This adds a [tool.pytest.ini_options] section with one filter, so every test in the suite becomes a detector instead:

filterwarnings = [
    'error:Passing .treatment. and/or .y. to \w+\(\) by position is deprecated:FutureWarning',
]

Scoped to this one message, not to FutureWarning generally. A blanket error::FutureWarning would also promote warnings from numpy, pandas and scikit-learn, and the build would go red on a dependency release that has nothing to do with this repo.

No workflow change is needed — every lane runs plain pytest with no -c override, so the ini filter applies to the source builds and the optional TF/torch/JAX lanes alike. pytest.warns installs its own filter, so the tests that assert the warning are unaffected.

Types of changes

What types of changes does your code introduce to CausalML?
Put an x in the boxes that apply

• Bugfix (non-breaking change which fixes an issue)
• New feature (non-breaking change which adds functionality)
• Breaking change (fix or feature that would cause existing functionality to not work as expected)
• Documentation Update (if none of the other choices apply)

Test/CI only — no library code changes.

Checklist

• I have read the CONTRIBUTING doc
• I have signed the CLA
• Lint and unit tests pass locally with my changes
• I have added tests that prove my fix is effective or that my feature works
• I have added necessary documentation (if appropriate) — the filter carries a comment explaining the scoping
• Any dependent changes have been merged and published in downstream modules — Stop the #854 shim warning on causalml's own internal fit calls #988 and Migrate tests and docs to keyword fit/predict calls (#982, #983) #989 are on master

Further comments

What this catches that the current tests do not

Injected a positional self.get_prediction(X, p, treatment, y) into SensitivitySelectionBias.causalsens — the same defect class as #989, on a path none of the three bespoke tests assert on — and ran tests/test_sensitivity.py both ways:

config	result
master today (-c /dev/null)	20 passed, exit 0 — warning printed, defect invisible
this PR	7 failed, 13 passed, exit 1 — incl. test_SensitivitySelectionBias

test_SensitivitySelectionBias and test_SensitivitySelectionBias_summary_ate already execute that line; they simply had no reason to assert on warnings. The filter turns them into regression tests for free.

The test

test_arg_order_warning_is_an_error_under_the_repo_pytest_config asserts a positional call raises under the repo's own config. The filter matches by message, so editing the warning text in _arg_order.py would silently stop it matching while CI stayed green — the failure mode this issue exists to prevent. Mutation-checked: with the filterwarnings entry removed, the test fails with DID NOT RAISE <class 'FutureWarning'>.

Verification

• Full suite: 445 passed, 10 skipped, exit 0 in 11:06, with zero arg-order warnings emitted (status read directly from $?, not through a pipe). 445 is the previous 444 plus the one test added here.
• tests/test_fit_arg_order.py: 58 → 59 tests, all passing.
• Reverting Stop the #854 shim warning on causalml's own internal fit calls #988's fix at causalforest.py:136 turns the suite red, which is the acceptance criterion on CI: no warnings-as-errors lane, so the #854 deprecation shim is untested against regressions #995. Worth noting that test_causal_forest_keyword_fit_does_not_warn already catches that one on its own — the filter's contribution is the untested paths above, not the three already pinned.
• black --check clean.

Not included

The issue also floated a separate CI job as an alternative. A single ini entry covers local runs and every CI lane at once, so a second job would add surface without adding coverage.

🤖 Generated with Claude Code