Fix DATA wildcard skipping historical full-install version scripts (#48)

HISTORY.asc

@@ -1,5 +1,13 @@
 STABLE
 ------
+== Fix `DATA` wildcard skipping historical full-install version scripts
+`base.mk`'s `DATA` wildcard (`sql/*--*--*.sql`) only matched upgrade scripts
+(two `--` separators), silently skipping historical full-install scripts
+like `sql/ext--0.9.6.sql` (one `--`) even though these are meant to be
+committed to git. `make install` never placed them in the extension
+directory, so `CREATE EXTENSION ext VERSION 'x.y.z'` failed for any version
+older than current. Widened to `sql/*--*.sql`, which matches both.
+
 == Rename `PGTLE_VERSION` to `PGXNTOOL_PGTLE_VERSION`
 `make pgtle`'s version-limiting variable was named `PGTLE_VERSION`, which
 make auto-imports from an identically-named environment variable. That name
@@ -15,7 +23,7 @@ regardless of `pg_regress`'s exit status, and `test`'s recipe only `cat`ed
 `regression.diffs` for visibility without checking it. `make test` now exits
 non-zero when `regression.diffs` is non-empty, after printing it.
 
-Issues fixed in this release: #35, #49, #78, #79
+Issues fixed in this release: #35, #49, #78, #79, #48
 
 2.2.0
 -----

README.asc

@@ -457,6 +457,21 @@ If you need to support multiple versions of your extension:
 
 The version file for the current version (specified in the `.control` file's `default_version`) will be automatically regenerated when you run `make`, but other version files you create manually will be preserved.
 
+=== Gotchas
+
+==== Generated SQL Files Can Be Missed on a Clean Build
+
+`DATA` (the PGXS variable controlling what `make install` actually installs) is seeded by pgxntool with `$(wildcard sql/*--*.sql)`, so PGXS installs whatever version-specific and upgrade files exist under `sql/` when `DATA` is evaluated.
+
+If your extension *generates* its versioned SQL from another source (e.g. a `.sql.in` template expanded by a custom rule) and gitignores the generated `.sql`, a clean `make install` (or `make test`, `make all install` in one invocation) can silently omit files that were only just generated during that same run. This isn't a pattern-matching problem — the glob is correct — it's a GNU Make quirk: `wildcard` caches a directory's contents internally for the life of one `make` invocation, and that cache isn't invalidated just because a recipe created new files earlier in the same run. So even though `DATA` is a recursively-expanded variable (re-evaluated each time it's referenced), re-referencing it later in the same invocation can still see the stale, pre-generation listing.
+
+Workarounds:
+
+- Run generation and install as separate `make` invocations (e.g. `make && make install`) so the second invocation's parse-time wildcard scan sees files the first invocation created.
+- List the generated files in `DATA` explicitly, keyed off stable names derived from your `.sql.in` sources (known at parse time) rather than relying on `wildcard` to discover the generated `.sql` output.
+
+Tracked as https://github.com/Postgres-Extensions/pgxntool/issues/44[issue #44] — pgxntool doesn't yet have first-class support for generated versioned SQL; the ideas under discussion there (e.g. a dedicated generated-output directory pgxntool builds before installing) are a bigger design change than a one-line fix.
+
 == Document Handling
 PGXNtool supports generation and installation of document files. There are several variables and rules that control this behavior.
 

README.html

@@ -475,6 +475,7 @@ <h1>PGXNtool</h1>
 <li><a href="#_alternative_ignoring_all_version_files">5.5. Alternative: Ignoring All Version Files</a></li>
 <li><a href="#_distribution_inclusion">5.6. Distribution Inclusion</a></li>
 <li><a href="#_multiple_versions">5.7. Multiple Versions</a></li>
+<li><a href="#_gotchas">5.8. Gotchas</a></li>
 </ul>
 </li>
 <li><a href="#_document_handling">6. Document Handling</a>
@@ -1481,6 +1482,34 @@ <h3 id="_multiple_versions"><a class="anchor" href="#_multiple_versions"></a><a
 <p>The version file for the current version (specified in the <code>.control</code> file&#8217;s <code>default_version</code>) will be automatically regenerated when you run <code>make</code>, but other version files you create manually will be preserved.</p>
 </div>
 </div>
+<div class="sect2">
+<h3 id="_gotchas"><a class="anchor" href="#_gotchas"></a><a class="link" href="#_gotchas">5.8. Gotchas</a></h3>
+<div class="sect3">
+<h4 id="_generated_sql_files_can_be_missed_on_a_clean_build"><a class="anchor" href="#_generated_sql_files_can_be_missed_on_a_clean_build"></a><a class="link" href="#_generated_sql_files_can_be_missed_on_a_clean_build">5.8.1. Generated SQL Files Can Be Missed on a Clean Build</a></h4>
+<div class="paragraph">
+<p><code>DATA</code> (the PGXS variable controlling what <code>make install</code> actually installs) is seeded by pgxntool with <code>$(wildcard sql/<strong>--</strong>.sql)</code>, so PGXS installs whatever version-specific and upgrade files exist under <code>sql/</code> when <code>DATA</code> is evaluated.</p>
+</div>
+<div class="paragraph">
+<p>If your extension <strong>generates</strong> its versioned SQL from another source (e.g. a <code>.sql.in</code> template expanded by a custom rule) and gitignores the generated <code>.sql</code>, a clean <code>make install</code> (or <code>make test</code>, <code>make all install</code> in one invocation) can silently omit files that were only just generated during that same run. This isn&#8217;t a pattern-matching problem — the glob is correct — it&#8217;s a GNU Make quirk: <code>wildcard</code> caches a directory&#8217;s contents internally for the life of one <code>make</code> invocation, and that cache isn&#8217;t invalidated just because a recipe created new files earlier in the same run. So even though <code>DATA</code> is a recursively-expanded variable (re-evaluated each time it&#8217;s referenced), re-referencing it later in the same invocation can still see the stale, pre-generation listing.</p>
+</div>
+<div class="paragraph">
+<p>Workarounds:</p>
+</div>
+<div class="ulist">
+<ul>
+<li>
+<p>Run generation and install as separate <code>make</code> invocations (e.g. <code>make &amp;&amp; make install</code>) so the second invocation&#8217;s parse-time wildcard scan sees files the first invocation created.</p>
+</li>
+<li>
+<p>List the generated files in <code>DATA</code> explicitly, keyed off stable names derived from your <code>.sql.in</code> sources (known at parse time) rather than relying on <code>wildcard</code> to discover the generated <code>.sql</code> output.</p>
+</li>
+</ul>
+</div>
+<div class="paragraph">
+<p>Tracked as <a href="https://github.com/Postgres-Extensions/pgxntool/issues/44">issue #44</a> — pgxntool doesn&#8217;t yet have first-class support for generated versioned SQL; the ideas under discussion there (e.g. a dedicated generated-output directory pgxntool builds before installing) are a bigger design change than a one-line fix.</p>
+</div>
+</div>
+</div>
 </div>
 </div>
 <div class="sect1">
@@ -1954,7 +1983,7 @@ <h2 id="_copyright"><a class="anchor" href="#_copyright"></a><a class="link" hre
 </div>
 <div id="footer">
 <div id="footer-text">
-Last updated 2026-07-31 15:09:00 -0500
+Last updated 2026-07-31 16:13:06 -0500
 </div>
 </div>
 </body>

base.mk

@@ -46,7 +46,14 @@ control.mk: $(PGXNTOOL_CONTROL_FILES) Makefile $(PGXNTOOL_DIR)/base.mk $(PGXNTOO
 
 -include control.mk
 
-DATA         = $(EXTENSION__CURRENT_VERSION__FILES) $(wildcard sql/*--*--*.sql)
+# $(wildcard sql/*--*.sql) covers both upgrade scripts (ext--a--b.sql) and
+# historical full-install scripts (ext--a.sql); EXTENSION__CURRENT_VERSION__FILES
+# is also matched by it but is listed explicitly since it's a generated file
+# that must exist as a prerequisite before the wildcard can see it. $(sort)
+# dedupes the resulting overlap -- without it, `install` is invoked with the
+# current-version file listed twice and refuses to overwrite the copy it
+# just created, failing `make install` outright.
+DATA         = $(sort $(EXTENSION__CURRENT_VERSION__FILES) $(wildcard sql/*--*.sql))
 DOC_DIRS	+= doc
 # NOTE: if this is empty it gets forcibly defined to NUL before including PGXS
 DOCS		+= $(foreach dir,$(DOC_DIRS),$(wildcard $(dir)/*))