Summary
On a genuinely fresh system (extension never previously installed), make test / make verify-results runs the main test suite's installcheck before install ever copies the extension's control/SQL files into place -- so every test in the main suite fails with "extension X is not available", purely due to ordering, not a real regression.
test-build (when enabled) is NOT affected, but only by luck of its own separate design, not because the general problem is fixed.
Root cause
In base.mk:
TEST_DEPS = testdeps
...
check-stale-expected: installcheck
TEST_DEPS += check-stale-expected
...
TEST_DEPS += test-build
...
TEST_DEPS += install installcheck
test: $(TEST_DEPS)
Textually, TEST_DEPS ends up as testdeps check-stale-expected test-build install installcheck.
check-stale-expected has its own direct dependency, check-stale-expected: installcheck -- so when Make resolves check-stale-expected (2nd in the list), it must first satisfy installcheck, which runs the main suite's pg_regress against a system where the extension was never installed. Only much later does Make reach the literal install installcheck pair at the end of the textual list (by which point installcheck is already considered satisfied from the earlier edge, so only install still runs -- too late for the main suite it already failed for).
test-build happens to dodge this because it declares its own separate dependency, test-build: install, which forces install to run before test-build's own nested installcheck. That protects test-build specifically; it does nothing for the main suite's installcheck, which was already resolved earlier via check-stale-expected's edge.
Reproduction (any pgxntool project, PGXNTOOL_ENABLE_CHECK_STALE_EXPECTED=yes, the default)
# Simulate a genuinely fresh system: remove any previously-installed
# copy of the extension from the system extension directory first.
rm -f "$(pg_config --sharedir)/extension/<your_extension>"*
make clean
make test # or: make verify-results
Every test in the main suite fails with something like:
ERROR: extension "<your_extension>" is not available
DETAIL: Could not open extension control file "..." No such file or directory.
even though nothing is actually broken -- a second run (or an explicit make install first) succeeds.
Confirmed on Postgres-Extensions/test_factory, master (pgxntool 2.2.0), reproduced 3/3 times with a cleared extension directory, and confirmed fixed 3/3 times by running make install as a separate invocation before make test/make verify-results.
Note: an earlier session (test_factory PR #23) hit the same underlying symptom and worked around it with the same "explicit make install first" fix, but attributed the cause to "ambient parallel make" (this container's pg-build-test helper exports MAKEFLAGS=-j $(nproc)). That diagnosis was incomplete -- reproduced here with a plain serial make -n dry run (no -j anywhere), confirming the true cause is this TEST_DEPS ordering issue, independent of parallelism.
Suggested fix
Give installcheck (and check-stale-expected, transitively) a real dependency on install, the same way test-build: install already does -- e.g. installcheck: install (careful: PGXS itself may already define installcheck without that dependency, so this needs the same "add a prerequisite without clobbering PGXS's own recipe" pattern test-build already uses), or reorder TEST_DEPS so install is guaranteed to resolve before anything that (directly or transitively) needs installcheck.
Summary
On a genuinely fresh system (extension never previously installed),
make test/make verify-resultsruns the main test suite'sinstallcheckbeforeinstallever copies the extension's control/SQL files into place -- so every test in the main suite fails with "extension X is not available", purely due to ordering, not a real regression.test-build(when enabled) is NOT affected, but only by luck of its own separate design, not because the general problem is fixed.Root cause
In
base.mk:Textually,
TEST_DEPSends up astestdeps check-stale-expected test-build install installcheck.check-stale-expectedhas its own direct dependency,check-stale-expected: installcheck-- so when Make resolvescheck-stale-expected(2nd in the list), it must first satisfyinstallcheck, which runs the main suite'spg_regressagainst a system where the extension was never installed. Only much later does Make reach the literalinstall installcheckpair at the end of the textual list (by which pointinstallcheckis already considered satisfied from the earlier edge, so onlyinstallstill runs -- too late for the main suite it already failed for).test-buildhappens to dodge this because it declares its own separate dependency,test-build: install, which forcesinstallto run before test-build's own nestedinstallcheck. That protects test-build specifically; it does nothing for the main suite'sinstallcheck, which was already resolved earlier viacheck-stale-expected's edge.Reproduction (any pgxntool project,
PGXNTOOL_ENABLE_CHECK_STALE_EXPECTED=yes, the default)Every test in the main suite fails with something like:
even though nothing is actually broken -- a second run (or an explicit
make installfirst) succeeds.Confirmed on
Postgres-Extensions/test_factory,master(pgxntool 2.2.0), reproduced 3/3 times with a cleared extension directory, and confirmed fixed 3/3 times by runningmake installas a separate invocation beforemake test/make verify-results.Note: an earlier session (test_factory PR #23) hit the same underlying symptom and worked around it with the same "explicit
make installfirst" fix, but attributed the cause to "ambient parallel make" (this container'spg-build-testhelper exportsMAKEFLAGS=-j $(nproc)). That diagnosis was incomplete -- reproduced here with a plain serialmake -ndry run (no-janywhere), confirming the true cause is thisTEST_DEPSordering issue, independent of parallelism.Suggested fix
Give
installcheck(andcheck-stale-expected, transitively) a real dependency oninstall, the same waytest-build: installalready does -- e.g.installcheck: install(careful: PGXS itself may already defineinstallcheckwithout that dependency, so this needs the same "add a prerequisite without clobbering PGXS's own recipe" patterntest-buildalready uses), or reorderTEST_DEPSsoinstallis guaranteed to resolve before anything that (directly or transitively) needsinstallcheck.