Update pgxntool to 2.3.0

Makefile

@@ -2,12 +2,3 @@ include pgxntool/base.mk
 
 # Temporary hack
 testdeps: $(wildcard test/*/*.sql) $(wildcard test/*.sql) # Be careful not to include directories in this
-
-# Install the outgoing "current" version's full install script now that
-# default_version is 'stable' (see count_nulls.control and RELEASE.md), so
-# CREATE EXTENSION count_nulls VERSION '1.0.0' keeps working. Not covered by
-# base.mk's DATA wildcard, which only picks up upgrade scripts
-# (sql/*--*--*.sql) and the CURRENT version file (now
-# sql/count_nulls--stable.sql) - a pgxntool bug, filed as
-# Postgres-Extensions/pgxntool#48.
-DATA += sql/count_nulls--1.0.0.sql

pgxntool/CLAUDE.md

@@ -91,18 +91,24 @@ include pgxntool/base.mk
 
 ### Phase 1: Meta Generation (`build_meta.sh`)
 - Processes `META.in.json` (template with placeholders/empty values)
-- Strips out X_comment fields and empty values
+- Strips lines with empty string values (this incidentally removes some, but not all, `X_comment` placeholder lines)
 - Produces clean `META.json`
 
-### Phase 2: Variable Extraction (`meta.mk.sh`)
-- Parses `META.json` using `JSON.sh` (a bash-based JSON parser)
+### Phase 2: Distribution Variable Extraction (`meta.mk.sh`)
+- Parses `META.json` (PGXN distribution metadata)
 - Generates `meta.mk` with Make variables:
   - `PGXN` - distribution name
   - `PGXNVERSION` - version number
+- `base.mk` includes `meta.mk` via `-include`
+
+### Phase 3: Extension Variable Extraction (`control.mk.sh`)
+- Parses each `.control` file (what PostgreSQL actually uses), not `META.json`
+- Generates `control.mk` with Make variables and rules:
   - `EXTENSIONS` - list of extensions provided
-  - `EXTENSION_*_VERSION` - per-extension versions
+  - `EXTENSION_*_VERSION` - per-extension versions, from each `.control` file's `default_version`
   - `EXTENSION__CURRENT_VERSION__FILES` - the current/most-recent auto-generated versioned SQL file for each extension (not all version files)
-- `base.mk` includes `meta.mk` via `-include`
+  - Rules for generating versioned SQL files
+- `base.mk` includes `control.mk` via `-include`
 
 ### The Magic of base.mk
 
@@ -157,10 +163,11 @@ make pgxntool-sync     # Update to latest pgxntool via git subtree pull
 ### Critical Testing Rules
 
 **NEVER use `make installcheck` directly**. Always use `make test` instead. The `make test` target ensures:
-- Clean builds before testing
-- Proper test isolation
-- Correct test dependency installation
-- Proper cleanup and result comparison
+- Correct test dependency installation (`testdeps`, and `test-build` when enabled)
+- Extension is installed before tests run (`install`)
+- Test comparison via `installcheck`, with diffs shown on failure
+
+Note: `make test` intentionally does *not* depend on `clean` — depending on `clean` caused problems with incremental/watch-based builds (see `base.mk`). If your tests need a clean build to pass, that's a sign of a missing dependency elsewhere, not something to fix by adding `clean` back.
 
 **Database Connection Requirement**: PostgreSQL must be running before executing `make test`. If you get connection errors (e.g., "could not connect to server"), stop and ask the user to start PostgreSQL.
 
@@ -204,7 +211,7 @@ When tests fail, examine the diff output carefully. The actual test output in `t
 
 ### Distribution Packaging
 - `make dist` creates `../PGXN-VERSION.zip`
-- Always creates git tag matching version
+- Creates a git tag matching version, unless that tag already exists and points at HEAD (in which case tagging is skipped)
 - Uses `git archive` to package
 - Validates repo is clean before tagging
 
@@ -219,7 +226,7 @@ When tests fail, examine the diff output carefully. The actual test output in `t
 
 pgxntool can generate pg_tle (Trusted Language Extensions) registration SQL for deploying extensions in AWS RDS/Aurora without filesystem access.
 
-**Usage:** `make pgtle` or `make pgtle PGTLE_VERSION=1.5.0+`
+**Usage:** `make pgtle` or `make pgtle PGXNTOOL_PGTLE_VERSION=1.5.0+`
 
 **Output:** `pg_tle/{version_range}/{extension}.sql`
 
@@ -273,11 +280,11 @@ Generated files depend on:
 
 ## Scripts
 
-- **setup.sh** - Initializes pgxntool in a new extension project (copies templates, creates directories)
-- **build_meta.sh** - Strips empty fields from META.in.json to create META.json
-- **meta.mk.sh** - Parses META.json via JSON.sh and generates meta.mk with Make variables
-- **JSON.sh** - Third-party bash JSON parser (MIT licensed)
-- **safesed** - Utility for safe sed operations
+Each script at the top level of this repository (pgxntool/) begins with a shebang followed by a short
+header comment describing its purpose within the first ~10 lines — read that header
+rather than relying on a hand-maintained list here, which would just go stale the same
+way this section itself once did. Exception: trivial third-party or one-line utilities
+may lack such a header.
 
 ## Related Repositories
 

pgxntool/HISTORY.asc

@@ -1,3 +1,30 @@
+2.3.0
+-----
+== Rename `PGTLE_VERSION` to `PGXNTOOL_PGTLE_VERSION`
+`make pgtle`'s version-limiting variable was named `PGTLE_VERSION`, which
+make auto-imports from an identically-named environment variable. That name
+is also a natural choice for a CI job's "which pg_tle to test against" env
+var -- when one was set, `make pgtle`/`make run-pgtle` silently misbehaved
+instead of erroring. Renamed to `PGXNTOOL_PGTLE_VERSION` to avoid the
+collision. If you invoke `make pgtle PGTLE_VERSION=...` directly, update it
+to `PGXNTOOL_PGTLE_VERSION`.
+
+== Fix `DATA` wildcard skipping historical full-install version scripts
+`base.mk`'s `DATA` wildcard (`sql/*--*--*.sql`) only matched upgrade scripts
+(two `--` separators), silently skipping historical full-install scripts
+like `sql/ext--0.9.6.sql` (one `--`) even though these are meant to be
+committed to git. `make install` never placed them in the extension
+directory, so `CREATE EXTENSION ext VERSION 'x.y.z'` failed for any version
+older than current. Widened to `sql/*--*.sql`, which matches both.
+
+== Fix `make test` always exiting 0, even when every test fails
+`.IGNORE: installcheck` made `make` treat `installcheck` as always successful
+regardless of `pg_regress`'s exit status, and `test`'s recipe only `cat`ed
+`regression.diffs` for visibility without checking it. `make test` now exits
+non-zero when `regression.diffs` is non-empty, after printing it.
+
+Issues fixed in this release: #35, #49, #78, #79, #48
+
 2.2.0
 -----
 == Add `check-stale-expected` to catch orphaned test/expected files

pgxntool/README.asc

@@ -51,6 +51,8 @@ Whether `test-build` runs is controlled by the `PGXNTOOL_ENABLE_TEST_BUILD` vari
 
 NOTE: `test` intentionally does *not* depend on `clean` — that caused problems with incremental/watch-based builds. If your tests need a clean build to pass, that's a sign of a missing dependency elsewhere rather than something to fix by adding `clean` back.
 
+NOTE: `test` exits non-zero (after printing `regression.diffs`) if any test fails. Previously it always exited 0 regardless of test results, silently masking failures from CI and other automation that relies on the exit code.
+
 NOTE: While you can still run `make installcheck` or any other valid PGXS make target directly, it's recommended to use `make test` when using pgxntool. The `test` target ensures proper test isolation and correct dependency installation.
 
 === test-build
@@ -270,7 +272,7 @@ WARNING: If your project has a `.gitattributes` file, `make dist`/`make dist-onl
 `make forcedist` is a shortcut for `forcetag dist`: it force-recreates the tag (see `forcetag` above) before rebuilding the distribution .zip.
 
 === pgxntool-sync
-This rule will pull down the latest released version of PGXNtool via `git subtree pull` and then reconcile the files `setup.sh` copied into your project (`.gitignore`, `test/deps.sql`) with a 3-way merge.
+This rule will pull down the latest released version of PGXNtool via `git subtree pull` and then reconcile the files `setup.sh` copied into your project (`.gitignore`, `test/deps.sql`) with a 3-way merge (it also verifies the `test/pgxntool` symlink, recreating it if missing).
 
 NOTE: Your repository must be clean (no modified files) in order to run this. Running this command will produce a git commit of the merge.
 
@@ -308,10 +310,10 @@ Generates pg_tle (Trusted Language Extensions) registration SQL files for deploy
 
 `make pgtle` generates SQL files in `pg_tle/` subdirectories organized by pg_tle version ranges. For version range details, see `pgtle_versions.md`.
 
-Set `PGTLE_VERSION` on the command line to limit generation to the single version range that value falls into, instead of every known range:
+Set `PGXNTOOL_PGTLE_VERSION` on the command line to limit generation to the single version range that value falls into, instead of every known range:
 
 ----
-make pgtle PGTLE_VERSION=1.5.0
+make pgtle PGXNTOOL_PGTLE_VERSION=1.5.0
 ----
 
 === check-pgtle
@@ -457,6 +459,21 @@ If you need to support multiple versions of your extension:
 
 The version file for the current version (specified in the `.control` file's `default_version`) will be automatically regenerated when you run `make`, but other version files you create manually will be preserved.
 
+=== Gotchas
+
+==== Generated SQL Files Can Be Missed on a Clean Build
+
+`DATA` (the PGXS variable controlling what `make install` actually installs) is seeded by pgxntool with `$(wildcard sql/*--*.sql)`, so PGXS installs whatever version-specific and upgrade files exist under `sql/` when `DATA` is evaluated.
+
+If your extension *generates* its versioned SQL from another source (e.g. a `.sql.in` template expanded by a custom rule) and gitignores the generated `.sql`, a clean `make install` (or `make test`, `make all install` in one invocation) can silently omit files that were only just generated during that same run. This isn't a pattern-matching problem — the glob is correct — it's a GNU Make quirk: `wildcard` caches a directory's contents internally for the life of one `make` invocation, and that cache isn't invalidated just because a recipe created new files earlier in the same run. So even though `DATA` is a recursively-expanded variable (re-evaluated each time it's referenced), re-referencing it later in the same invocation can still see the stale, pre-generation listing.
+
+Workarounds:
+
+- Run generation and install as separate `make` invocations (e.g. `make && make install`) so the second invocation's parse-time wildcard scan sees files the first invocation created.
+- List the generated files in `DATA` explicitly, keyed off stable names derived from your `.sql.in` sources (known at parse time) rather than relying on `wildcard` to discover the generated `.sql` output.
+
+Tracked as https://github.com/Postgres-Extensions/pgxntool/issues/44[issue #44] — pgxntool doesn't yet have first-class support for generated versioned SQL; the ideas under discussion there (e.g. a dedicated generated-output directory pgxntool builds before installing) are a bigger design change than a one-line fix.
+
 == Document Handling
 PGXNtool supports generation and installation of document files. There are several variables and rules that control this behavior.
 
@@ -665,9 +682,9 @@ make dist PGXN_REMOTE=upstream
 
 Default: auto-detected, the first of `asciidoctor` or `asciidoc` found on `PATH`. Path to the Asciidoc processor used to build `.html` files from `$(ASCIIDOC_EXTS)` source files. Override if the processor you want isn't first on `PATH`, or isn't on `PATH` at all. See <<_document_handling>>.
 
-=== PGTLE_VERSION
+=== PGXNTOOL_PGTLE_VERSION
 
-Default: unset (generates every known pg_tle version range). Set on the command line to limit `make pgtle` to the single version range this value falls into. See <<_pgtle>>.
+Default: unset (generates every known pg_tle version range). Set on the command line to limit `make pgtle` to the single version range this value falls into. Not named `PGTLE_VERSION`: make auto-imports same-named environment variables, and that name collided silently with CI jobs that set a `PGTLE_VERSION` env var for an unrelated purpose (which pg_tle to test against). See <<_pgtle>>.
 
 === PG_CONFIG
 

pgxntool/README.html

@@ -475,6 +475,7 @@ <h1>PGXNtool</h1>
 <li><a href="#_alternative_ignoring_all_version_files">5.5. Alternative: Ignoring All Version Files</a></li>
 <li><a href="#_distribution_inclusion">5.6. Distribution Inclusion</a></li>
 <li><a href="#_multiple_versions">5.7. Multiple Versions</a></li>
+<li><a href="#_gotchas">5.8. Gotchas</a></li>
 </ul>
 </li>
 <li><a href="#_document_handling">6. Document Handling</a>
@@ -499,7 +500,7 @@ <h1>PGXNtool</h1>
 <ul class="sectlevel2">
 <li><a href="#_pgxn_remote">8.1. PGXN_REMOTE</a></li>
 <li><a href="#_asciidoc">8.2. ASCIIDOC</a></li>
-<li><a href="#_pgtle_version">8.3. PGTLE_VERSION</a></li>
+<li><a href="#_pgxntool_pgtle_version">8.3. PGXNTOOL_PGTLE_VERSION</a></li>
 <li><a href="#_pg_config">8.4. PG_CONFIG</a></li>
 <li><a href="#_testdir">8.5. TESTDIR</a></li>
 <li><a href="#_testout">8.6. TESTOUT</a></li>
@@ -1198,11 +1199,11 @@ <h3 id="_pgtle"><a class="anchor" href="#_pgtle"></a><a class="link" href="#_pgt
 <p><code>make pgtle</code> generates SQL files in <code>pg_tle/</code> subdirectories organized by pg_tle version ranges. For version range details, see <code>pgtle_versions.md</code>.</p>
 </div>
 <div class="paragraph">
-<p>Set <code>PGTLE_VERSION</code> on the command line to limit generation to the single version range that value falls into, instead of every known range:</p>
+<p>Set <code>PGXNTOOL_PGTLE_VERSION</code> on the command line to limit generation to the single version range that value falls into, instead of every known range:</p>
 </div>
 <div class="listingblock">
 <div class="content">
-<pre>make pgtle PGTLE_VERSION=1.5.0</pre>
+<pre>make pgtle PGXNTOOL_PGTLE_VERSION=1.5.0</pre>
 </div>
 </div>
 </div>
@@ -1481,6 +1482,34 @@ <h3 id="_multiple_versions"><a class="anchor" href="#_multiple_versions"></a><a
 <p>The version file for the current version (specified in the <code>.control</code> file&#8217;s <code>default_version</code>) will be automatically regenerated when you run <code>make</code>, but other version files you create manually will be preserved.</p>
 </div>
 </div>
+<div class="sect2">
+<h3 id="_gotchas"><a class="anchor" href="#_gotchas"></a><a class="link" href="#_gotchas">5.8. Gotchas</a></h3>
+<div class="sect3">
+<h4 id="_generated_sql_files_can_be_missed_on_a_clean_build"><a class="anchor" href="#_generated_sql_files_can_be_missed_on_a_clean_build"></a><a class="link" href="#_generated_sql_files_can_be_missed_on_a_clean_build">5.8.1. Generated SQL Files Can Be Missed on a Clean Build</a></h4>
+<div class="paragraph">
+<p><code>DATA</code> (the PGXS variable controlling what <code>make install</code> actually installs) is seeded by pgxntool with <code>$(wildcard sql/<strong>--</strong>.sql)</code>, so PGXS installs whatever version-specific and upgrade files exist under <code>sql/</code> when <code>DATA</code> is evaluated.</p>
+</div>
+<div class="paragraph">
+<p>If your extension <strong>generates</strong> its versioned SQL from another source (e.g. a <code>.sql.in</code> template expanded by a custom rule) and gitignores the generated <code>.sql</code>, a clean <code>make install</code> (or <code>make test</code>, <code>make all install</code> in one invocation) can silently omit files that were only just generated during that same run. This isn&#8217;t a pattern-matching problem — the glob is correct — it&#8217;s a GNU Make quirk: <code>wildcard</code> caches a directory&#8217;s contents internally for the life of one <code>make</code> invocation, and that cache isn&#8217;t invalidated just because a recipe created new files earlier in the same run. So even though <code>DATA</code> is a recursively-expanded variable (re-evaluated each time it&#8217;s referenced), re-referencing it later in the same invocation can still see the stale, pre-generation listing.</p>
+</div>
+<div class="paragraph">
+<p>Workarounds:</p>
+</div>
+<div class="ulist">
+<ul>
+<li>
+<p>Run generation and install as separate <code>make</code> invocations (e.g. <code>make &amp;&amp; make install</code>) so the second invocation&#8217;s parse-time wildcard scan sees files the first invocation created.</p>
+</li>
+<li>
+<p>List the generated files in <code>DATA</code> explicitly, keyed off stable names derived from your <code>.sql.in</code> sources (known at parse time) rather than relying on <code>wildcard</code> to discover the generated <code>.sql</code> output.</p>
+</li>
+</ul>
+</div>
+<div class="paragraph">
+<p>Tracked as <a href="https://github.com/Postgres-Extensions/pgxntool/issues/44">issue #44</a> — pgxntool doesn&#8217;t yet have first-class support for generated versioned SQL; the ideas under discussion there (e.g. a dedicated generated-output directory pgxntool builds before installing) are a bigger design change than a one-line fix.</p>
+</div>
+</div>
+</div>
 </div>
 </div>
 <div class="sect1">
@@ -1858,9 +1887,9 @@ <h3 id="_asciidoc"><a class="anchor" href="#_asciidoc"></a><a class="link" href=
 </div>
 </div>
 <div class="sect2">
-<h3 id="_pgtle_version"><a class="anchor" href="#_pgtle_version"></a><a class="link" href="#_pgtle_version">8.3. PGTLE_VERSION</a></h3>
+<h3 id="_pgxntool_pgtle_version"><a class="anchor" href="#_pgxntool_pgtle_version"></a><a class="link" href="#_pgxntool_pgtle_version">8.3. PGXNTOOL_PGTLE_VERSION</a></h3>
 <div class="paragraph">
-<p>Default: unset (generates every known pg_tle version range). Set on the command line to limit <code>make pgtle</code> to the single version range this value falls into. See <a href="#_pgtle">pgtle</a>.</p>
+<p>Default: unset (generates every known pg_tle version range). Set on the command line to limit <code>make pgtle</code> to the single version range this value falls into. Not named <code>PGTLE_VERSION</code>: make auto-imports same-named environment variables, and that name collided silently with CI jobs that set a <code>PGTLE_VERSION</code> env var for an unrelated purpose (which pg_tle to test against). See <a href="#_pgtle">pgtle</a>.</p>
 </div>
 </div>
 <div class="sect2">
@@ -1954,7 +1983,7 @@ <h2 id="_copyright"><a class="anchor" href="#_copyright"></a><a class="link" hre
 </div>
 <div id="footer">
 <div id="footer-text">
-Last updated 2026-07-29 14:55:58 -0500
+Last updated 2026-07-31 16:13:06 -0500
 </div>
 </div>
 </body>