docs+install: 0.20.4 sweep, JOIN fan-out (#195), lifted alias restriction, and a JDK bootstrap on Windows - #232
Merged
Merged
Conversation
…sufficient
Two changes to the Windows install path, and one consequence of the second
that must not ship without it.
1. install.cmd downloads install.ps1 when it is not beside it.
Previously it errored and told the user to fetch two files. Now one file is
enough. A LOCAL install.ps1 still wins, so a downloaded pair — or a released
bundle — stays self-consistent instead of being silently mixed with main.
Written as `goto` plus a separate errorlevel test rather than
`if ... call ... || exit`: how cmd binds `||` inside an `if` body is ambiguous,
and this is not a platform where a subtlety can be settled by running it. The
download lives in a subroutine for the same reason the launcher's two FOR loops
are on separate lines — cmd expands every %VAR% in a parenthesised block in ONE
parse pass, so `set "PS1=..."` followed by a use of %PS1% inside that block
would read the OLD value.
2. install.ps1 installs a JDK instead of aborting.
Resolution order, and JAVA_HOME is the thing actually tested when it is set —
not the PATH `java`, which is frequently a different and older JVM:
%JAVA_HOME%\bin\java.exe -> `java` on PATH -> neither, or too old
In the last case the installer downloads a portable Temurin 17 JDK (an Adoptium
zip, never an MSI, so no administrator rights) and unpacks it to <install>\jdk.
17 satisfies BOTH floors — 11 for ES 6/7/8, 17 for ES 9 — so there is one
download to reason about rather than two.
The archive unpacks as jdk-17.x.y+z\, a version-stamped directory. It is staged
and MOVED one level up so the final JAVA_HOME is the fixed path <install>\jdk:
the launcher hard-codes %BASE_DIR%\jdk\bin and must not glob for a name that
changes with every Temurin build.
Machine state is left alone. JAVA_HOME and PATH are set for the installer's own
SESSION only — deliberately not SetEnvironmentVariable(...,"User"), which would
silently repoint every other tool on the box. Nothing is needed in later
sessions because of (3).
Get-JavaMajorVersion is split: Get-JavaMajorFromExe takes an explicit exe path,
so JAVA_HOME and PATH are probed by the same code. Resolve-Java runs AFTER the
-ListVersions early exit (listing versions must never download a JDK) and BEFORE
bundle selection, which reads the resolved major. Check-Prerequisites now
asserts that outcome rather than re-deriving it — still exactly ONE parse of
`java -version` per run, so a localised or multi-line output cannot be read two
different ways.
3. The generated launchers resolve Java the SAME way.
This is the half that makes (2) real rather than a one-shot trick: both
launchers invoked bare `java`, so a bootstrapped JDK would have been invisible
the moment the user opened a new terminal — the install would look broken for
exactly the users the feature is for. bin\softclient4es.bat and
bin\softclient4es.ps1 now try <install>\jdk, then %JAVA_HOME%, then PATH.
They PREPEND to PATH rather than call an absolute exe: every `java` below stays
unchanged, and there is no quoting of a path that contains spaces (the default
target is under %USERPROFILE%). `setlocal` scopes the edit to the launcher.
The summary and VERSION now report which JVM was chosen, and the printed tree
shows jdk\ when one was bootstrapped.
Verified on macOS with pwsh 7 — there is no Windows host here, so cmd.exe
executing install.cmd and Expand-Archive on a real Temurin zip are NOT covered:
* install.ps1 parses clean (Language.Parser.ParseFile, 0 errors)
* 16/16 checks in a harness that loads the REAL function bodies out of
install.ps1's own AST — not a transcription — and exercises Resolve-Java:
version parsing (17.0.11 -> 17, 1.8.0_292 -> 8, missing exe -> 0);
JAVA_HOME above the floor is used and nothing is bootstrapped; JAVA_HOME
BELOW the floor bootstraps exactly once and repoints session JAVA_HOME/PATH;
a JAVA_HOME pointing nowhere is not silently trusted; a failed bootstrap
returns false rather than passing silently
* -ListVersions still exits 0 against live JFrog and never reaches Java
resolution; -Help exits 0; -EsVersion 5 still exits 1
* install.cmd stays ASCII + CRLF, and its only goto target exists
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…ias restriction (arrow#137)
The elasticsql half of the post-release sweep. The softclient4es-web half is a
separate PR in that repo.
VERSIONS. Resolved against the published JFrog listings, not against commit
titles — the currently published set really is engine 0.20.3, drivers /
arrow-extensions 0.2.4, community-extensions 0.2.3, bundle line 0.20.1..0.20.3,
so every bump here is exactly one release:
engine / bundle examples 0.20.3 -> 0.20.4 README, repl.md, install.{sh,ps1}
jdbc / adbc / flight-sql 0.2.4 -> 0.2.5 jdbc.md, adbc_driver.md,
arrow_flight_sql.md,
download_analytics.md, README
arrow-extensions 0.2.4 -> 0.2.5 README
community-extensions 0.2.3 -> 0.2.4 README
Deliberately NOT bumped — dml_statements.md:450 says local JSON / JSON_ARRAY
"were additionally moved off Hadoop entirely in 0.20.3". That is a historical
fact about which release changed the behaviour, not a current-version reference;
bumping it would make the sentence false.
The repl.md example listing gains 0.20.4 and its total is corrected 2 -> 4. It
claimed "Total: 2 version(s)" under three bullets; a live -ListVersions run
prints "Total: 3 version(s)" for the three published bundles, so the count was
simply wrong.
JOINS — arrow#137, fixed. The "Two ORDER BY gotchas" callout documented a
restriction that no longer exists. Rewritten against the shipped planner code
(JoinPlanner.processPostJoinIds / isSelectAlias / referencesNoColumn) rather
than from the issue text, so the boundaries are the real ones: SELECT aliases
and ordinals resolve post-join in GROUP BY, HAVING and ORDER BY, matching is
case-insensitive — but an alias is still NOT legal in SELECT, ON or WHERE, and
it must be bare, because `ORDER BY d.cnt` qualifies a name no table owns and
dies inside DuckDB. Version boundary stated as "since 0.2.5", never "up to and
including", which is how this file class rots.
The second gotcha — ORDER BY a column present on BOTH sides — is untouched.
Nothing in this train changed it, so nothing new is asserted about it.
JOINS — #195, fan-out on a non-unique key. New section next to that callout,
with the issue's verified 2x3x2 = 12 example. One correction to the issue's own
suggested wording, because it matters: it groups "AVG/SUM" together as not
visibly breaking. SUM is inflated exactly like COUNT. What is preserved by
uniform duplication is AVG, MIN and MAX — which is precisely the trap, since
three columns of four then corroborate a count that is 6x off. Also notes that
AVG *is* wrong when the fan-out factor varies within a group.
Two further corrections against the source rather than the issue:
* the issue calls a truncated fan-out "a wrong answer that returns
successfully". The engine reports it — SQLWarning 01004, and an
x-result-truncated header on Flight SQL, per this file's own Row truncation
section, which the new text links to instead of contradicting.
* cap figures taken from the meters table in this file: Community 10,000 /
Pro 1,000,000.
WINDOWS INSTALL. repl.md documents the new Java bootstrap (probe order, Temurin
17 covering both floors, no admin, session-scoped env, uninstall removes it) and
the single-file install.cmd path; README's cmd.exe line becomes one command; the
install tree gains jdk\.
Every SQL example added or touched here was parse-probed through the real
Parser.apply — 6/6 parse AND round-trip (the rendered AST re-parses). That
covers COUNT(DISTINCT ...), which the new remedy advice depends on and which
would have been a guess otherwise.
⚠️ NOT YET SATISFIABLE, and a merge gate: the sweep convention is to verify each
version on JFrog BEFORE writing it, because a version with no -all bundle falls
back silently to the plain artifact and the docs end up demonstrating the path
that has no cross-index JOIN. The 0.20.4 / 0.2.5 artifacts are not published at
authoring time. Before merging, HEAD-check: softclient4es{6,7,8,9}-cli-all_2.13
0.20.4, softclient4es{6,7,8,9}-{jdbc,adbc}-driver 0.2.5,
softclient4es{6,7,8,9}-arrow-flight-sql 0.2.5, softclient4es-arrow-extensions
0.2.5, softclient4es-community-extensions 0.2.4.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This was referenced Aug 16, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Two commits, off the merged 0.20.4
main: the Windows installer changes, then the elasticsql half of the post-release documentation sweep. Thesoftclient4es-webhalf is a separate PR in that repo.1.
feat(install)— bootstrap a JDK on Windows, and makeinstall.cmdself-sufficientinstall.cmddownloadsinstall.ps1when it is not beside it, so one file is a complete install. A localinstall.ps1still wins, so a downloaded pair — or a released bundle — is never silently mixed withmain. Written asgoto+ a separate errorlevel test rather thanif … call … || exit, because howcmdbinds||inside anifbody is ambiguous and this is not a platform where a subtlety can be settled by running it.install.ps1installs a JDK instead of aborting. Resolution order — andJAVA_HOMEis the thing actually tested when it is set, not the PATHjava, which is frequently a different and older JVM:In the last case it downloads a portable Temurin 17 JDK (Adoptium zip, never an MSI, so no administrator rights) into
<install>\jdk. 17 satisfies both floors (11 for ES 6/7/8, 17 for ES 9), so there is one download to reason about rather than two.jdk-17.x.y+z\; it is staged and moved one level up so the final JAVA_HOME is the fixed path<install>\jdk— the launcher hard-codes%BASE_DIR%\jdk\binand must not glob for a name that changes with every Temurin build.JAVA_HOME/PATHare set for the installer's own session only — deliberately notSetEnvironmentVariable(…,"User"), which would silently repoint every other tool on the box and dangle after uninstall.Resolve-Javaruns after the-ListVersionsearly exit (listing versions must never download a JDK) and before bundle selection, which reads the resolved major.Both generated launchers resolve Java the same way —
<install>\jdk→%JAVA_HOME%→ PATH. This is the half that makes the bootstrap real: they invoked barejavabefore, so a bootstrapped JDK would have been invisible the moment the user opened a new terminal, and the install would look broken for exactly the users the feature is for. They prepend to PATH rather than call an absolute exe, so everyjavabelow stays unchanged and nothing has to quote a path containing spaces.2.
docs— versions, JOIN fan-out (#195), lifted alias restriction (arrow#137)Versions, resolved against the published JFrog listings rather than commit titles — the published set really is engine
0.20.3, drivers/arrow-extensions0.2.4, community-extensions0.2.3, bundle line0.20.1..0.20.3— so every bump is exactly one release:Deliberately not bumped:
dml_statements.md:450— "moved off Hadoop entirely in 0.20.3" is a historical fact about which release changed the behaviour. Bumping it would make the sentence false.The
repl.mdexample listing gains0.20.4and its total is corrected 2 → 4: it claimedTotal: 2 version(s)under three bullets, and a live-ListVersionsrun printsTotal: 3 version(s)for the three published bundles.arrow#137 — the restriction is gone. The "Two ORDER BY gotchas" callout documented behaviour that no longer exists. Rewritten against the shipped planner code (
JoinPlanner.processPostJoinIds/isSelectAlias/referencesNoColumn) rather than from the issue text, so the boundaries are the real ones: SELECT aliases and ordinals resolve post-join inGROUP BY/HAVING/ORDER BY, case-insensitively — but an alias is still not legal inSELECT/ON/WHERE, and it must be bare, sinceORDER BY d.cntqualifies a name no table owns and dies inside DuckDB. Stated as "since 0.2.5", never "up to and including" — that phrasing is how this file class rots. The second gotcha (a column present on both sides) is untouched: nothing in this train changed it.#195 — fan-out on a non-unique key. New section with the issue's verified 2 × 3 × 2 = 12 example. Three corrections made against the source rather than transcribing the issue:
SUMis inflated exactly likeCOUNT. What uniform duplication preserves isAVG,MIN,MAX— which is precisely the trap: three columns of four corroborate a count that is 6× off. Also noted thatAVGis wrong when the fan-out factor varies within a group.SQLWarning 01004,x-result-truncatedon Flight SQL) per this file's own truncation section, which the new text links to instead of contradicting.Windows install docs.
repl.mddocuments the Java bootstrap (probe order, Temurin 17 covering both floors, no admin, session-scoped env, uninstall removes it) and the single-fileinstall.cmdpath; README's cmd.exe line becomes one command; the install tree gainsjdk\.Verification
install.ps1parses clean; 16/16 checks in a harness that loads the real function bodies out ofinstall.ps1's own AST — not a transcription — and drivesResolve-Java: version parsing (17.0.11→17,1.8.0_292→8, missing→0); JAVA_HOME above the floor is used with no bootstrap; JAVA_HOME below the floor bootstraps exactly once and repoints sessionJAVA_HOME/PATH; a JAVA_HOME pointing nowhere is not silently trusted; a failed bootstrap returns false rather than passing silently.-ListVersionsstill exits 0 against live JFrog and never reaches Java resolution;-Helpexits 0;-EsVersion 5still exits 1.install.cmdstays ASCII + CRLF.Parser.apply— 6/6 parse and round-trip. That coversCOUNT(DISTINCT …), which the new remedy advice depends on and would otherwise have been a guess.cmd.exeexecutinginstall.cmdandExpand-Archiveon a real Temurin zip are not covered.Version gate — verified on JFrog
Every coordinate this PR writes was HEAD-checked against the live repository, per the sweep rule that versions are verified before they are written:
softclient4es8-java-client0.20.4ScrollConfig.retainDocumentId, #226)softclient4es-community-extensions0.2.4ElasticsearchCapability.watcherAvailability, #50)softclient4es-arrow-extensions0.2.5softclient4es{6,7,8,9}-jdbc-driver0.2.5softclient4es-jdbc-driver_2.130.2.5softclient4es{6,7,8,9}-adbc-driver0.2.5softclient4es{6,7,8,9}-arrow-flight-sql0.2.5softclient4es{6,7,8,9}-cli-all_2.130.20.4The es8 fat driver was also opened and inspected, since the entire reason jdbc pins arrow is that the fix ships inside that jar: it carries
DuckDBSpill(arrow-join 0.2.5, #141),JoinDetectorwith theCrossIndexJoinclassification underarrow/join/routing/, and core 0.20.4'sScrollConfig.retainDocumentIdplus thescrollRowsrename from #224.✅ Gate cleared — the REPL bundle is published and verified. The es8 bundle was opened:
softclient4es-bundle-info.propertiesreports exactly this train (engine.version=0.20.4,community.extensions.version=0.2.4,arrow.extensions.version=0.2.5,java.floor=11);Cli.classappears exactly once;licenses/+NOTICEare present; and the mergedMETA-INF/services/app.softnetwork.elastic.client.ExtensionSpilists all five extensions includingJoinExtension— the one a badfirst-merge silently drops, which would ship a bundle with no cross-index JOIN.The documented
-ListVersionsexample was also checked against a live run rather than left as prose: the installer now prints0.20.1 / 0.20.2 / 0.20.3 / 0.20.4andTotal: 4 version(s), exactly what this PR writes — which also confirms theTotal: 2it replaced was wrong.Closes #195
🤖 Generated with Claude Code