[Packaging] Fix #28497: Preserve non-ASCII output from Windows launchers in isolated mode - #33935
[Packaging] Fix #28497: Preserve non-ASCII output from Windows launchers in isolated mode#33935hasu (jun2077681) wants to merge 2 commits into
Conversation
python -I isolated mode ignores PYTHONUTF8/PYTHONIOENCODING, so non-ASCII output (e.g. Korean, Chinese) gets mangled on Windows when the console's ANSI codepage isn't UTF-8. Adding -X utf8 forces PEP 540 UTF-8 mode for stdio without weakening isolation (-I still ignores PYTHONPATH, site-packages, and script-dir sys.path insertion). Fixes Azure#28497
|
Hi hasu (@jun2077681), |
|
Azure Pipelines: There may be pipelines that require an authorized user to comment /azp run to run. |
|
@microsoft-github-policy-service agree |
There was a problem hiding this comment.
Pull request overview
This PR addresses Windows console encoding corruption for non-ASCII Azure CLI output when the MSI/ZIP launchers run the bundled Python in isolated mode (-I). It does so by explicitly enabling UTF-8 mode via the interpreter flag -X utf8, which preserves isolation while fixing stdout/stderr encoding behavior on non-UTF-8 legacy codepages.
Changes:
- Add
-X utf8to the ZIP launcher (az_zip.cmd) when invokingpython.exe -IBm azure.cli. - Add
-X utf8to the MSI launcher (az_msi.cmd) when invokingpython.exe -IBm azure.cli.
Reviewed changes
Copilot reviewed 2 out of 2 changed files in this pull request and generated 1 comment.
| File | Description |
|---|---|
| build_scripts/windows/scripts/az_zip.cmd | Adds -X utf8 to force UTF-8 stdio in the ZIP launcher while keeping isolated mode. |
| build_scripts/windows/scripts/az_msi.cmd | Adds -X utf8 to force UTF-8 stdio in the MSI launcher while keeping isolated mode. |
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
| @IF EXIST "%~dp0\..\python.exe" ( | ||
| SET AZ_INSTALLER=MSI | ||
| "%~dp0\..\python.exe" -IBm azure.cli %* | ||
| "%~dp0\..\python.exe" -X utf8 -IBm azure.cli %* |
There was a problem hiding this comment.
Good catch — added the same -X utf8 flag to az and azps.ps1 as well (they hit the identical -IBm isolated-mode encoding issue).
Both invoke the bundled python.exe with -IBm as well, so they hit the same non-ASCII output corruption as the .cmd launchers.
|
Windows az launcher |
Cooper Cox (coopercox-ms)
added
the
Azure Client Tools Agent Requested
Live test skipped⏭️ Skipping the live test for this revision because no new test file was added ( The live-test pipeline runs only the test files a PR changes, so there is nothing to execute for this commit. This is informational — a regression test is encouraged where it makes sense, but not required. If a test file is added in a later commit, the live test will run automatically. |
azure-client-tools-agent
Bot
added
the
azure-client-tools-agent
azure-client-tools-agent
Bot
removed
the
Azure Client Tools Agent Requested
|
/azp run |
|
Azure Pipelines: Successfully started running 3 pipeline(s). |
Cooper Cox (coopercox-ms)
changed the title
|
/azp run |
|
Azure Pipelines: Successfully started running 3 pipeline(s). |
Cooper Cox (coopercox-ms)
added
Azure Client Tools Agent Requested
![azure-client-tools-agent[bot]](https://avatars.githubusercontent.com/in/4205534?s=60&v=4){kind=small}
There was a problem hiding this comment.
Automated Review — Pass
All CI checks passed (65/65) and no blocking human review requests are outstanding.
- Live test: skipped — this PR changes no
tests/**/test_*.pyfiles (packaging/launcher fix only), so there is nothing for the live-test pipeline to exercise. - CI: 65/65 checks completed successfully, 0 failed, 0 pending.
- Regression coverage: not applicable — no Azure CLI command-module production code was changed.
No further automated action is required from this pass. A human maintainer should still confirm the packaging fix ([Packaging] Fix #28497) behaves as expected on Windows before merging.
azure-client-tools-agent
Bot
added
azure-client-tools-agent
🤖 PR Validation — ️✔️ All clear
Summary
az_msi.cmd/az_zip.cmdinvoke the bundled Python with-I(isolated mode), which drops-Ebehavior and ignoresPYTHONUTF8/PYTHONIOENCODING.Unable to encode the output with cp949/cp1252 encoding. Unsupported characters are discarded.).-X utf8(PEP 540) is an independent interpreter flag from-I— it only forces UTF-8 mode for stdio, and does not restore any of-I's ignored env vars or site-packages behavior. Adding it keeps the isolation guarantees intact while fixing output encoding.Change
Two one-line edits:
in both
build_scripts/windows/scripts/az_msi.cmdandaz_zip.cmd.Verification
Reproduced locally against an installed 2.89.1 MSI build (
C:\Program Files\Microsoft SDKs\Azure\CLI2\wbin\az.cmd):az repos pr show --id <id> --query title -o tsvon a PR with a Korean title printed mojibake (?? ??-style output).-X utf8, the same command prints the title correctly.PYTHONPATHbefore/after the patch has no effect on CLI behavior in either case (env var still ignored).Fixes #28497