From 1298db255280f4799c4a30acae570984255d299c Mon Sep 17 00:00:00 2001 From: "Jonathan D.A. Jewell" <6759885+hyperpolymath@users.noreply.github.com> Date: Mon, 3 Aug 2026 14:31:31 +0100 Subject: [PATCH] fix(coverage): make bisect_ppx actually instrument the library MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The coverage job runs, passes, and measures nothing. $ dune runtest --force --instrument-with bisect_ppx 537 tests OK, exit 0 $ bisect-ppx-report summary Error: no *.coverage files found Zero .coverage files are written anywhere in the tree. == Root cause lib/dune declares `(preprocess (pps ...))` but no instrumentation backend. `--instrument-with bisect_ppx` only instruments libraries that opt in with an `(instrumentation (backend bisect_ppx))` stanza; without one, dune instruments nothing, the tests run uninstrumented, and no coverage data exists to report. == Why nobody noticed .github/workflows/ci.yml:213 sets `continue-on-error: true` on the step that calls bisect-ppx-report. So the report failed on every run, the failure was swallowed, and the coverage job went green while producing no data. The same hollow-gate shape as a test that passes without running. The `continue-on-error` is left in place — it is defensible for a visibility-only artefact step, and with this fix the step now has something to report. Worth revisiting separately if coverage ever becomes gating. == After the fix .coverage files: 0 -> 2 Coverage: 8103/16243 (49.89%) That number has never been measurable before, so it is a baseline rather than a regression or an improvement. == What it says needs attention Core paths are reasonably covered: parser.ml 75.64% 969/1281 quantity.ml 79.28% 241/304 typecheck.ml 68.37% 1042/1524 borrow.ml 68.03% 800/1176 resolve.ml 59.21% 405/684 lexer.ml 58.33% 49/84 The gaps are concentrated in backends and tooling, not the front end: wasm_gc.ml 0.00% 0/178 <- entirely unexercised wasm.ml 0.24% 1/423 types.ml 9.95% 19/191 lean_codegen 10.48% 13/124 value.ml 13.13% 26/198 c_codegen.ml 14.60% 67/459 lsp_server.ml 15.47% 43/278 wasm_gc.ml at 0/178 is the standout: 178 points, none reached by any of the 537 tests. One change only: lib/dune. --- lib/dune | 10 +++++++++- 1 file changed, 9 insertions(+), 1 deletion(-) diff --git a/lib/dune b/lib/dune index e85af42f..df737673 100644 --- a/lib/dune +++ b/lib/dune @@ -114,7 +114,15 @@ (flags (:standard -w -8-9)) (preprocess - (pps ppx_deriving.show ppx_deriving.eq ppx_deriving.ord sedlex.ppx))) + (pps ppx_deriving.show ppx_deriving.eq ppx_deriving.ord sedlex.ppx)) + ; Without this stanza `dune runtest --instrument-with bisect_ppx` has nothing + ; to attach to: dune instruments nothing, the tests run uninstrumented, and no + ; .coverage files are written. `bisect-ppx-report summary` then fails with + ; "no *.coverage files found" — and because the CI step that calls it carries + ; `continue-on-error: true`, that failure was swallowed and the coverage job + ; went green while measuring nothing. + (instrumentation + (backend bisect_ppx))) (menhir (modules parser)