Add azd ai dataset extension for Foundry datasets by m7md7sien · Pull Request #9499 · Azure/azure-dev

Mohamed Hessien (m7md7sien) · 2026-08-10T00:13:43Z

Adds azd ai dataset ΓÇö a standalone extension for registering and versioning Foundry datasets.

Why its own extension

A dataset is a general Foundry asset. Fine-tuning and other scenarios need one as much as evaluation does, so the CRUD group does not belong inside the evaluation extension. This extension has its own Go module, lint workflow and release pipeline, and no dependency on azure.ai.evaluations.

The line drawn with the evaluation extension: CRUD moved, generation stayed. generate writes the datasets: entry into the evaluation configuration, so moving it would strand a config write across an extension boundary ΓÇö a generated dataset registered with the service and absent from configuration, which azd up would not reconcile. create/update/list/show/delete/versions need no configuration at all, which is why they could move.

Surface

azd ai dataset create <name>      register a dataset, publishing version 1
azd ai dataset update <name>      publish a new version
azd ai dataset list               list the project's datasets
azd ai dataset show <name>        show a version
azd ai dataset delete <name>      delete a version
azd ai dataset versions list <name>

Notable

Every user-facing string lives in one file (internal/messages/messages.go) so the whole voice of the CLI can be reviewed in one sitting. 69 messages.
TYPE, not FORMAT. The API accepts format on upload and never returns it, so the previous column was blank for every dataset ever listed.
Identity fixed. Carved from the evaluation extension, it had inherited its User-Agent (azd-ext-azure-ai-evaluations) and debug log name (azd-ai-eval-*.log), so a dataset request could not be told from an eval one in a service log or a local one.
Auth errors say what to do. A credential that cannot mint a token fails before any 401 and surfaced as a bare SDK error. It now suggests retrying first and logging in second ΓÇö observed failing transiently against a login that was valid.
A name the service will reject is refused locally. A dataset name with a space came back as a 400 wrapped in four levels of nested JSON; it is now checked against the accepted character set before the request.
A UTF-8 BOM no longer corrupts an upload. Windows editors and PowerShell write one by default, and it was being uploaded into the first row's first key rather than skipped.
Nil-guarded the endpoint resolution cascade and closed the azd client on the error paths that abandoned it.

Verified

go build, go vet -tags live,hero, go test ./... and gofmt clean. Exercised live against a Foundry project: create ΓåÆ 1.0, update ΓåÆ 2.0, versions list, show, list, and published to a local registry for a bug bash through real azd.

Not included, deliberately

No registry.json entry and no microsoft.foundry bundle dependency. Both need published artifacts; listing an unpublished extension in the bundle breaks azd extension install microsoft.foundry for everyone. Those land after the first release.

Open question for reviewers

azd ai dataset versions list <unknown-name> errors and exits 1 here, while the same command in the evaluation extension (#9500) lists nothing and exits 0. I tried making the eval side match this one and it broke two tests whose names are the argument against it ΓÇö "an unknown name lists nothing rather than failing" and "deleting an unregistered dataset is idempotent, not an error", the second because delete idempotence is verified by listing what is left.

A list is a filter rather than a lookup, so empty-and-successful is defensible and -o json callers range over the array instead of branching on an error. By that reasoning this extension's versions list is the one that should move. dataset show on an unknown name is not in question: that one is a lookup and should fail, as it does.

Draft ΓÇö not requesting reviewers yet.

Fixes #9550

Datasets are a general Foundry asset that fine-tuning and other scenarios need as much as evaluation does, so the CRUD group ships as its own extension rather than inside azure.ai.evaluations. Generation stays with the evaluation extension, because it writes that extension's config file. Standalone: no dependency on azure.ai.evaluations, its own go module, lint workflow and release pipeline.

Adds internal/messages, holding the extension's errors, warnings and status lines behind one named function each, grouped under Datasets, Config, Output and Talking to the service. Wording is unchanged; the call sites now name the situation instead of spelling out the sentence. Cobra help, wire constants and table headers stay where they are.

Three things carried over from the extension this was carved out of. The User-Agent announced azure-ai-evaluations and the debug log was written to azd-ai-eval-<date>.log, so a dataset request could not be told from an eval one in either a service log or a local one. The FORMAT column was worse than wrong: format is a field the API accepts on upload and never sends back, so the column was blank for every dataset ever listed. It shows type, which the service does return.

Carved from the same code, so it had the same two: a nil Environment or GetValue response panicking inside the endpoint cascade, and an azd client abandoned when endpoint or credential setup failed.

Same credential failure and same 401/403 path as the eval extension, which this was carved from.

Same intermittent credential failure as the eval extension.

Same three as the eval extension, which this was carved from.

azure-pipelines · 2026-08-10T00:13:56Z

Azure Pipelines: 7 pipeline(s) were filtered out due to trigger conditions. There may be pipelines that require an authorized user to comment /azp run to run.

A BOM at the head of a .jsonl is uploaded as part of the first row's first key, so every consumer of the dataset sees one malformed record -- and nothing fails until something tries to read that row. Notepad, VS Code on Windows and PowerShell all write one. A file holding nothing but a BOM is still empty, and is still refused as such.

Found by bug-bashing the off-path cases. A missing dataset reported "has no versions", which is not what happened -- a dataset cannot exist with no versions, and the service answers an unknown name with an empty list rather than a 404. Listing its versions was worse: it said "No datasets found", as though the project had none at all. Both now say the name is not a dataset and point at the listing, matching how the evaluations extension already answers the same question. A name with a space took a file upload and then a 400 carrying four levels of nested JSON, with the sentence that matters at the bottom. The rule is simple and documented, so it is checked before the round trip.

A mistyped path surfaced as 'GetFileAttributesEx <path>: The system cannot find the file specified.' The syscall name is noise to the person who mistyped it.

--from-file <file> resolved to the file's DIRECTORY, and the upload helper then took whichever .jsonl sorted first. Pointing at one dataset in a folder holding several registered a different one under that name. A test pinned the old behaviour: 'a file resolves to the directory the upload scans'.

The unit tests cover the client layer and the helpers; nothing exercised the command surface a user touches - flag parsing, exit codes, the rendered table, and whether -o json emits something a script can consume. 18 tests behind the live tag, gated on AZURE_AI_DATASET_E2E_LIVE, type-checked by ci-test.ps1 like the eval extension's suites.

azure-pipelines · 2026-08-10T04:26:08Z

Azure Pipelines: 7 pipeline(s) were filtered out due to trigger conditions. There may be pipelines that require an authorized user to comment /azp run to run.

Copilot

Pull request overview

Adds a standalone Foundry dataset extension with CRUD/versioning commands, API clients, tests, documentation, and release automation.

Changes:

Adds dataset creation, updates, listing, inspection, deletion, and version listing.
Adds endpoint resolution, validation, structured errors, and JSON/table output.
Adds unit/live tests plus build, lint, and release infrastructure.

Reviewed changes

Copilot reviewed 49 out of 50 changed files in this pull request and generated 19 comments.

Show a summary per file

File	Description
`eng/pipelines/release-ext-azure-ai-dataset.yml`	Adds extension release pipeline.
`cli/azd/extensions/azure.ai.dataset/version.txt`	Sets initial beta version.
`cli/azd/extensions/azure.ai.dataset/tests/cli/harness_test.go`	Adds live CLI harness.
`cli/azd/extensions/azure.ai.dataset/tests/cli/dataset_test.go`	Tests dataset CLI workflows.
`cli/azd/extensions/azure.ai.dataset/README.md`	Documents installation and commands.
`cli/azd/extensions/azure.ai.dataset/main.go`	Adds extension entry point.
`cli/azd/extensions/azure.ai.dataset/internal/version/version.go`	Defines build metadata.
`cli/azd/extensions/azure.ai.dataset/internal/pkg/dataset_api/version_test.go`	Tests version handling.
`cli/azd/extensions/azure.ai.dataset/internal/pkg/dataset_api/uri_test.go`	Tests URI resolution.
`cli/azd/extensions/azure.ai.dataset/internal/pkg/dataset_api/upload_version_test.go`	Tests conflict-aware uploads.
`cli/azd/extensions/azure.ai.dataset/internal/pkg/dataset_api/operations.go`	Implements dataset and storage operations.
`cli/azd/extensions/azure.ai.dataset/internal/pkg/dataset_api/operations_wire_test.go`	Tests API request contracts.
`cli/azd/extensions/azure.ai.dataset/internal/pkg/dataset_api/models.go`	Defines API models and file handling.
`cli/azd/extensions/azure.ai.dataset/internal/pkg/dataset_api/list.go`	Implements listing and version ordering.
`cli/azd/extensions/azure.ai.dataset/internal/pkg/dataset_api/download_wire_test.go`	Tests download HTTP behavior.
`cli/azd/extensions/azure.ai.dataset/internal/pkg/dataset_api/download_test.go`	Tests blob selection logic.
`cli/azd/extensions/azure.ai.dataset/internal/pkg/dataset_api/bom_test.go`	Tests BOM and file handling.
`cli/azd/extensions/azure.ai.dataset/internal/messages/messages.go`	Centralizes user-facing messages.
`cli/azd/extensions/azure.ai.dataset/internal/foundry/projectctx/validator.go`	Validates Foundry endpoints.
`cli/azd/extensions/azure.ai.dataset/internal/foundry/projectctx/types.go`	Defines endpoint-resolution types.
`cli/azd/extensions/azure.ai.dataset/internal/foundry/projectctx/store.go`	Reads shared project context.
`cli/azd/extensions/azure.ai.dataset/internal/foundry/projectctx/resolver.go`	Implements endpoint precedence.
`cli/azd/extensions/azure.ai.dataset/internal/foundry/projectctx/resolver_test.go`	Tests endpoint resolution.
`cli/azd/extensions/azure.ai.dataset/internal/exterrors/errors.go`	Adds structured error helpers.
`cli/azd/extensions/azure.ai.dataset/internal/exterrors/codes.go`	Defines structured error codes.
`cli/azd/extensions/azure.ai.dataset/internal/cmd/table_test.go`	Tests table rendering.
`cli/azd/extensions/azure.ai.dataset/internal/cmd/surface_test.go`	Tests command surface and flags.
`cli/azd/extensions/azure.ai.dataset/internal/cmd/root.go`	Builds the command tree.
`cli/azd/extensions/azure.ai.dataset/internal/cmd/output.go`	Implements JSON and text output.
`cli/azd/extensions/azure.ai.dataset/internal/cmd/output_test.go`	Tests output formatting.
`cli/azd/extensions/azure.ai.dataset/internal/cmd/names.go`	Validates dataset names.
`cli/azd/extensions/azure.ai.dataset/internal/cmd/names_test.go`	Tests naming constraints.
`cli/azd/extensions/azure.ai.dataset/internal/cmd/helpers_test.go`	Tests command helpers.
`cli/azd/extensions/azure.ai.dataset/internal/cmd/debug.go`	Configures debug logging.
`cli/azd/extensions/azure.ai.dataset/internal/cmd/dataset.go`	Implements dataset commands.
`cli/azd/extensions/azure.ai.dataset/internal/cmd/context.go`	Creates command service context.
`cli/azd/extensions/azure.ai.dataset/internal/cmd/artifacts.go`	Defines persistence helpers.
`cli/azd/extensions/azure.ai.dataset/internal/cmd/apiversions.go`	Defines service API versions.
`cli/azd/extensions/azure.ai.dataset/go.mod`	Defines the Go module.
`cli/azd/extensions/azure.ai.dataset/extension.yaml`	Declares extension metadata.
`cli/azd/extensions/azure.ai.dataset/cspell.yaml`	Configures spell checking.
`cli/azd/extensions/azure.ai.dataset/ci-test.ps1`	Adds CI test execution.
`cli/azd/extensions/azure.ai.dataset/ci-build.ps1`	Adds CI binary builds.
`cli/azd/extensions/azure.ai.dataset/CHANGELOG.md`	Records the initial release.
`cli/azd/extensions/azure.ai.dataset/build.sh`	Adds Unix cross-platform builds.
`cli/azd/extensions/azure.ai.dataset/build.ps1`	Adds Windows cross-platform builds.
`cli/azd/extensions/azure.ai.dataset/.golangci.yaml`	Configures Go linting.
`cli/azd/extensions/azure.ai.dataset/.gitignore`	Ignores generated artifacts.
`.github/workflows/lint-ext-azure-ai-dataset.yml`	Adds GitHub lint workflow.

💡 Configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

+    "-X 'azureaieval/internal/version.Version=$Version' " +
+    "-X 'azureaieval/internal/version.Commit=$SourceVersion' " +
+    "-X 'azureaieval/internal/version.BuildDate=$(Get-Date -Format o)' "


+    )
+fi
+
+VERSION_PATH="azureaieval/internal/version"


+    )
+}
+
+$VERSION_PATH = "azureaieval/internal/version"


+	q.Set("comp", "list")
+	u.RawQuery = q.Encode()
+
+	log.Printf("[dataset_api] listing blobs: %s", u.Redacted())


+func DatasetVersionNotFoundWithHint(dataset, version string) error {
+	return fmt.Errorf(
+		"no dataset %q at version %q in this project; "+
+			"`azd ai dataset list` shows the ones there are", dataset, version)
+}


+	var respErr *azcore.ResponseError
+	if !errors.As(err, &respErr) {


+	var respErr *azcore.ResponseError
+	if errors.As(err, &respErr) {


+	cmd.Flags().StringVar(&version, "version", "",
+		"Current version to increment from. Omit to increment from the latest registered version.")


+- The CRUD groups moved here from `azure.ai.evaluations`. Generation stayed
+  there: `dataset generate` writes the `datasets:` entry in `evals/eval.yaml`,
+  and that file belongs to the evaluation extension.


Two lines of ci-test.ps1 assumed PowerShell 6 or later. $IsWindows does not exist on Windows PowerShell 5.1, so the .exe suffix was never appended; and Join-Path there takes a single child path, so the three-argument call failed outright. The script then found no gotestsum, printed the fallback notice, and ran bare `go test` with no JUnit report -- while still exiting 0, which is why it went unnoticed.

Copilot

Pull request overview

Copilot reviewed 53 out of 54 changed files in this pull request and generated no new comments.

Suppressed comments (19)

cli/azd/extensions/azure.ai.dataset/README.md:28

This points users to a command that the evaluation extension does not expose. Its surface is azd ai eval generate, with --dataset selecting dataset generation; using the current text results in an unknown-command error.

Generation is `azd ai eval dataset generate`, in `azure.ai.evaluations`, and
stays there: it writes the `datasets:` entry in `evals/eval.yaml`, which is that

cli/azd/extensions/azure.ai.dataset/build.sh:36

The Unix extension build stamps azureaieval, which is not this module. Point the linker at azureaidataset/internal/version so locally packed binaries carry the requested version and build identity.

VERSION_PATH="azureaieval/internal/version"

cli/azd/extensions/azure.ai.dataset/.gitignore:5

The debug logger writes azd-ai-dataset-*.log, but this rule ignores the old evaluation filename. Dataset logs—including request diagnostics—will remain unignored and can be committed accidentally.

azd-ai-eval-*.log

cli/azd/extensions/azure.ai.dataset/ci-test.ps1:10

This names the evaluation extension's opt-in variable, but tests/cli/TestMain checks AZURE_AI_DATASET_E2E_LIVE. Following this instruction silently skips the live suite instead of running it.

# AZURE_AI_EVAL_E2E_LIVE and a project endpoint. They are still type-checked

cli/azd/extensions/azure.ai.dataset/README.md:41

The documented endpoint cascade omits the AZURE_AI_PROJECT_ENDPOINT fallback and the shared global project context that projectctx.Resolve checks before host variables. Users may unnecessarily set a lower-precedence value because this order does not match runtime behavior.

1. `--project-endpoint`
2. `FOUNDRY_PROJECT_ENDPOINT` in the active azd environment
3. the host environment variable of the same name

cli/azd/extensions/azure.ai.dataset/ci-build.ps1:45

These linker symbols still target the evaluation module. The release template invokes this script for every artifact, so azureaidataset/internal/version.Version, Commit, and BuildDate are never stamped and the dataset User-Agent remains at its dev default.

    "-X 'azureaieval/internal/version.Version=$Version' " +
    "-X 'azureaieval/internal/version.Commit=$SourceVersion' " +
    "-X 'azureaieval/internal/version.BuildDate=$(Get-Date -Format o)' "

cli/azd/extensions/azure.ai.dataset/build.ps1:44

The PowerShell extension build stamps azureaieval, which is not this module. Point the linker at azureaidataset/internal/version so locally packed binaries carry the requested version and build identity.

$VERSION_PATH = "azureaieval/internal/version"

cli/azd/extensions/azure.ai.dataset/internal/pkg/dataset_api/operations.go:459

url.URL.Redacted() only hides user-info passwords; it does not remove query parameters. This logs the full container SAS, including sig, into the debug file. Log only the scheme/host/path and keep the authenticated URL solely on the request.
cli/azd/extensions/azure.ai.dataset/internal/pkg/dataset_api/list.go:19
Both list methods discard NextLink, so projects with a paged response silently lose datasets/versions. This also makes show and update-version discovery select an older version. Follow all pages and validate each continuation URL is same-origin before requesting it.
cli/azd/extensions/azure.ai.dataset/internal/cmd/dataset.go:79
A list failure is currently converted into exists == false. update then reports that the dataset does not exist for authentication, transport, and service failures, while create proceeds toward an upload. Propagate the list error; only a successful empty response means absent.

			existing, err := ec.datasetClient.ListDatasetVersions(
				ctx, name, ProjectEndpointAPIVersion,
			)
			exists := err == nil && existing != nil && len(existing.Value) > 0

cli/azd/extensions/azure.ai.dataset/internal/cmd/dataset.go:88

The consistency probes ignore every error as if the version were absent. If either point read fails with auth, transport, or a 5xx response, update returns the misleading “does not exist” error. Continue only on 404 and propagate other failures.

					if _, err := ec.datasetClient.GetDataset(
						ctx, name, v, ProjectEndpointAPIVersion,
					); err == nil {

cli/azd/extensions/azure.ai.dataset/internal/cmd/dataset.go:105

The PR states that CRUD needs no configuration, but every create/update mutates the active environment by writing one global EVAL_DATASET_VERSION. That key cannot represent multiple datasets and has no reader in this extension, so the command introduces an unrelated persistent side effect. Remove this write and its dead cache helpers.

			if err := ec.setEnvValue(ctx, envKeyDatasetVersion, ds.Version); err != nil {

cli/azd/extensions/azure.ai.dataset/internal/pkg/dataset_api/operations.go:157

This collapses every listing error into “no registered version” and then attempts to publish 1.0. A transient list failure can therefore create an old version or turn the real service error into a conflict. Return the error separately; reserve the empty value for a successful empty listing.
cli/azd/extensions/azure.ai.dataset/internal/exterrors/errors.go:94
Use errors.AsType[*azcore.ResponseError](err) here. This repository's Go 1.26 modernization rules prohibit the target-variable errors.As pattern, and CI runs go fix -diff.

	var respErr *azcore.ResponseError
	if errors.As(err, &respErr) {

cli/azd/extensions/azure.ai.dataset/internal/pkg/dataset_api/operations.go:177

Use the Go 1.26 errors.AsType form required by this repository; the current target-variable pattern is subject to the enforced go fix modernization check.
cli/azd/extensions/azure.ai.dataset/internal/pkg/dataset_api/operations.go:168
This module targets Go 1.26, and the repository requires errors.AsType for typed error matching. The go fix -diff workflow will reject this older target-variable pattern.
cli/azd/extensions/azure.ai.dataset/README.md:6
This quickstart cannot currently succeed: the same README notes that no registry entry exists, so azd extension install azure.ai.dataset cannot resolve. Document the local pack/publish flow until the registry entry lands, or add the registry entry before presenting this command.

$ azd extension install azure.ai.dataset

cli/azd/extensions/azure.ai.dataset/README.md:22

The documented delete command omits the required --version flag, so copying it always fails with --version is required. Include the mandatory flag in the command table.

| `azd ai dataset delete <name>` | Delete a dataset version |

cli/azd/extensions/azure.ai.dataset/internal/cmd/dataset.go:304

The code already documents that this version listing can return empty for a few seconds after creation, but show treats that stale response as proof the dataset does not exist. An immediate create → show can therefore fail falsely; use a bounded consistency fallback before returning not-found.

				if len(list.Value) == 0 {
					return messages.DatasetNotFound(name)
				}

cspell: - Correct "neighbouring" to "neighboring" in the changelog. - Add the identifiers cspell cannot infer to the extension dictionary: azureaidataset, projectctx, projectctx's, exterrors, CLI's. These mirror the entries the azure.ai.evaluations extension already carries. golangci-lint: - Delete datasetContext.getEnvValue and the surface_test find helper. Both had no callers, so they were dead rather than suppressible. - Rename the storageServer test field credentialPath to uriPath. It holds a URL path, never a secret; the old name is what tripped gosec G101, so the rename removes the finding instead of hiding it behind a nolint. - Annotate the three filepath.WalkDir ReadFile calls in surface_test with //nolint:gosec and the reason: these walk this package's own source, not caller input. No behaviour change. go build, go vet, gofmt and go test ./... all clean.

Seven review comments flagged this and were all marked outdated as line numbers moved, so none of them had actually been applied. `url.URL.Redacted()` masks a userinfo password only; it leaves the query string intact. A storage SAS carries its credential in the query as `sig`, so both `log.Printf(... u.Redacted())` sites wrote a live, still-valid storage credential into `azd-ai-dataset-*.log` whenever --debug was on. Separately, `http.Client.Do` returns a *url.Error whose message embeds the full request URL. For the three SAS-backed calls (blob upload, container listing, blob download) a DNS/TLS/timeout/cancellation failure therefore printed the SAS to the user. - Add logSafeURL, which drops the query and fragment before formatting. - Add redactURLError, which rebuilds a *url.Error with a redacted URL and without mutating the original. Uses errors.AsType per cli/azd/AGENTS.md. - Apply to both log sites and all three SAS-backed Do() calls. Tests assert the premise as well as the fix: one case pins that Redacted() alone does leak the signature, so the unsafe call cannot quietly come back. The request itself still uses the unmodified URL. gofmt, go vet, go build and go test ./... are clean.

Copilot

Pull request overview

Copilot reviewed 54 out of 55 changed files in this pull request and generated 3 comments.

Suppressed comments (20)

cli/azd/extensions/azure.ai.dataset/internal/pkg/dataset_api/operations.go:427

A failed direct download returns a raw *url.Error, which includes the full SAS-bearing request URL and can expose the token in user-facing output or the fallback debug log. Redact the query before wrapping this error. — azd-code-reviewer
cli/azd/extensions/azure.ai.dataset/internal/pkg/dataset_api/operations.go:469
A container-list transport error includes the full SAS URL in *url.Error.Error(), so returning it through ListingContainerBlobs can expose the token. Sanitize the query before wrapping the error. — azd-code-reviewer
cli/azd/extensions/azure.ai.dataset/internal/pkg/dataset_api/operations.go:507
A failed blob download returns a raw *url.Error containing the full SAS-bearing URL. That error is rendered to users, so redact its query before wrapping it. — azd-code-reviewer
cli/azd/extensions/azure.ai.dataset/internal/pkg/dataset_api/list.go:19
NextLink is captured but never followed, so list, versions list, latest-version resolution, and existence checks only see the first page. The methods promise all datasets/versions and the JSON renderer deliberately drops the paging link, leaving callers no way to fetch omitted pages. Follow pages inside the client before returning. — azd-code-reviewer
cli/azd/extensions/azure.ai.dataset/internal/cmd/dataset.go:105
Every dataset publish overwrites the same project-wide EVAL_DATASET_VERSION, even though the command can manage many independently named datasets. Publishing dataset B can therefore leave consumers treating B's version as A's, and this extension never reads the key back. Remove this mutation or persist a dataset-keyed value with a documented consumer contract. — azd-code-reviewer

			if err := ec.setEnvValue(ctx, envKeyDatasetVersion, ds.Version); err != nil {

cli/azd/extensions/azure.ai.dataset/ci-build.ps1:45

These linker targets still use the evaluation module path. Because this module is azureaidataset, the release build leaves Version, Commit, and BuildDate at their defaults, including the dev version used in the User-Agent. Point all three targets at this module. — azd-code-reviewer

    "-X 'azureaieval/internal/version.Version=$Version' " +
    "-X 'azureaieval/internal/version.Commit=$SourceVersion' " +
    "-X 'azureaieval/internal/version.BuildDate=$(Get-Date -Format o)' "

cli/azd/extensions/azure.ai.dataset/build.sh:36

This copied module path prevents local cross-platform builds from stamping the dataset extension's version metadata; the linker target must match the azureaidataset module. — azd-code-reviewer

VERSION_PATH="azureaieval/internal/version"

cli/azd/extensions/azure.ai.dataset/build.ps1:44

This copied module path prevents the PowerShell build from stamping this extension's version metadata; target the azureaidataset module instead. — azd-code-reviewer

$VERSION_PATH = "azureaieval/internal/version"

cli/azd/extensions/azure.ai.dataset/internal/cmd/dataset.go:96

This turns every list failure—401/403, timeout, or 5xx—into exists == false, then also ignores non-404 point-read failures. update consequently reports “does not exist,” while create may attempt an upload without establishing that the name is free. Return list errors and only ignore a point-read error when it is actually not-found. — azd-code-reviewer

			exists := err == nil && existing != nil && len(existing.Value) > 0

cli/azd/extensions/azure.ai.dataset/internal/cmd/dataset.go:304

This treats an empty version list as authoritative even though the new upload path explicitly documents that this listing is eventually consistent. A show immediately after a successful create can therefore report that the dataset does not exist. Resolve the just-published version through a consistent point read or retry the empty listing before returning not-found. — azd-code-reviewer

				if len(list.Value) == 0 {
					return messages.DatasetNotFound(name)
				}

cli/azd/extensions/azure.ai.dataset/.gitignore:5

The generated debug file is named azd-ai-dataset-<date>.log (internal/cmd/debug.go:31), so this copied evaluation pattern does not ignore it and makes credential-bearing diagnostics easy to commit accidentally. — azd-code-reviewer

azd-ai-eval-*.log

cli/azd/extensions/azure.ai.dataset/.golangci.yaml:12

This disables the repository's established 125-character Go limit for the new module; the root configuration sets it at cli/azd/.golangci.yaml:27. Keeping 220 here means this extension's dedicated lint workflow accepts code the rest of azd rejects. — azd-code-reviewer

      line-length: 220

cli/azd/extensions/azure.ai.dataset/ci-test.ps1:10

The live harness actually requires AZURE_AI_DATASET_E2E_LIVE; this copied evaluation variable sends anyone following the CI instructions to a run that silently skips all tagged tests. — azd-code-reviewer

# AZURE_AI_EVAL_E2E_LIVE and a project endpoint. They are still type-checked

cli/azd/extensions/azure.ai.dataset/internal/cmd/dataset.go:366

The live test and stated behavior require delete to be idempotent, but a service that represents an already-absent version as 404 is converted into a failure here. Treat not-found as successful deletion and only return other errors. — azd-code-reviewer

				if dataset_api.IsNotFound(err) {
					return messages.DatasetVersionNotFound(name, version)

cli/azd/extensions/azure.ai.dataset/internal/cmd/dataset.go:238

This now makes an unknown name return an empty successful list, while the PR description still says this extension errors and presents the behavior as an open question. Update the description to record the decision implemented and tested here. — azd-code-reviewer

			// An unknown name lists nothing and succeeds; it is not an error.
			// `-o json` callers range over the array, and a delete is checked
			// for idempotence by listing what is left. The empty sentence names
			// the dataset, though: the project may hold plenty of others, so
			// "No datasets found." would be answering a different question.
			return renderDatasets(cmd, list, messages.NoDatasetVersions(name))

cli/azd/extensions/azure.ai.dataset/tests/cli/dataset_test.go:51

Cleanup silently returns on list/JSON failures and ignores every delete result, so the live suite can pass while leaving all created dataset versions behind. Make cleanup failures visible and require each delete to succeed. — azd-code-reviewer
cli/azd/extensions/azure.ai.dataset/README.md:6
This first-run command cannot work in this PR because the registry entry is deliberately omitted, as the TODO at lines 54–57 confirms. Replace it with complete local-registry setup instructions or defer the install example until the extension is resolvable. — azd-code-reviewer

$ azd extension install azure.ai.dataset

cli/azd/extensions/azure.ai.dataset/README.md:22

delete rejects calls without --version, but the command table presents this incomplete invocation. Include the required flag so the documented command succeeds. — azd-code-reviewer

| `azd ai dataset delete <name>` | Delete a dataset version |

cli/azd/extensions/azure.ai.dataset/README.md:41

This claims a three-level endpoint cascade, but the resolver also checks AZURE_AI_PROJECT_ENDPOINT at the azd/host levels and the shared global project context between them. Document all levels so users can understand why an endpoint other than these three wins. — azd-code-reviewer

1. `--project-endpoint`
2. `FOUNDRY_PROJECT_ENDPOINT` in the active azd environment
3. the host environment variable of the same name

cli/azd/extensions/azure.ai.dataset/README.md:28

The companion evaluation extension exposes azd ai eval generate, not azd ai eval dataset generate, and its current configuration is azure.eval.yaml. As written, this sends users to a nonexistent command and names the wrong file. — azd-code-reviewer

Generation is `azd ai eval dataset generate`, in `azure.ai.evaluations`, and
stays there: it writes the `datasets:` entry in `evals/eval.yaml`, which is that

+	resp, err := httpClient.Do(req)
+	if err != nil {
+		return messages.UploadingBlobFailed(err)


+	q.Set("comp", "list")
+	u.RawQuery = q.Encode()
+
+	log.Printf("[dataset_api] listing blobs: %s", u.Redacted())


+	best := ""
+	bestOrder := -2.0
+	for _, d := range datasets {
+		if o := VersionOrder(d.Version); o > bestOrder {
+			bestOrder, best = o, d.Version
+		}
+	}
+	if best == "" && len(datasets) > 0 {
+		return datasets[len(datasets)-1].Version
+	}


The cspell job runs `--config ./.vscode/cspell.yaml`, so words have to be known to the repo-level config; the extension's own cspell.yaml serves the other job and the editor. - "unauthorised" -> "unauthorized" in a ServiceRefused comment. US spelling is what the rest of the repo uses. - Rename the redactURLError local from `uerr` to `urlError`. `uerr` is not a word cspell can resolve, and naming it properly removes the finding rather than adding a dictionary entry for an abbreviation. - Register CODEOWNERS for the extension README under `overrides`, matching how this config already scopes per-file words. Verified with the exact CI invocation: 577 files checked, 0 issues.

Copilot

Pull request overview

Copilot reviewed 55 out of 56 changed files in this pull request and generated 1 comment.

Suppressed comments (16)

cli/azd/extensions/azure.ai.dataset/internal/pkg/dataset_api/list.go:24

[azd-code-reviewer] Both list methods return only the first page even though DatasetList exposes NextLink. Projects with enough datasets or versions will omit entries; show can select the wrong latest version and update can derive a conflicting version. Follow same-origin pagination links until exhausted, as the repository's connection clients do.
cli/azd/extensions/azure.ai.dataset/internal/pkg/dataset_api/operations.go:157
[azd-code-reviewer] latestRegisteredVersion collapses every list error into an empty version. On an auth or service failure, UploadNextVersion then attempts 1.0 and can return a misleading conflict instead of the original failure. Return (string, error) and propagate errors from both call sites; an empty successful list already represents an unknown or not-yet-consistent dataset.
cli/azd/extensions/azure.ai.dataset/internal/cmd/dataset.go:105
[azd-code-reviewer] CRUD is described as configuration-free, but every create/update writes one global EVAL_DATASET_VERSION into the active azd environment. That value has no dataset name, and --project-endpoint may target a project unrelated to the active environment, so publishing any dataset can overwrite evaluation state with an ambiguous or foreign version. Remove this persistence from the standalone dataset extension.

			if err := ec.setEnvValue(ctx, envKeyDatasetVersion, ds.Version); err != nil {

cli/azd/extensions/azure.ai.dataset/internal/pkg/dataset_api/operations.go:194

[azd-code-reviewer] This module targets Go 1.26, and the repository requires errors.AsType rather than the mutable-target errors.As pattern. Use the typed helper here.
cli/azd/extensions/azure.ai.dataset/internal/pkg/dataset_api/operations.go:203
[azd-code-reviewer] Use the repository's Go 1.26 errors.AsType pattern here instead of declaring a mutable target for errors.As.
cli/azd/extensions/azure.ai.dataset/ci-build.ps1:45
[azd-code-reviewer] The release build stamps symbols in the evaluation module, which is not part of this binary. The linker therefore leaves version.Version as dev, so every published request still uses azd-ext-azure-ai-dataset/dev despite the identity fix. Stamp the azureaidataset module instead.

    "-X 'azureaieval/internal/version.Version=$Version' " +
    "-X 'azureaieval/internal/version.Commit=$SourceVersion' " +
    "-X 'azureaieval/internal/version.BuildDate=$(Get-Date -Format o)' "

cli/azd/extensions/azure.ai.dataset/internal/cmd/dataset.go:93

[azd-code-reviewer] Any listing failure is currently treated as “dataset absent.” A 401, 403, timeout, or 5xx therefore makes update report that the dataset does not exist and lets create continue toward an upload. Propagate list errors, and ignore only IsNotFound from the consistency probes.

			exists := err == nil && existing != nil && len(existing.Value) > 0

cli/azd/extensions/azure.ai.dataset/build.ps1:44

[azd-code-reviewer] This local PowerShell build also targets the evaluation module, so its version, commit, and date linker values are silently ignored. Use this extension's module path.

$VERSION_PATH = "azureaieval/internal/version"

cli/azd/extensions/azure.ai.dataset/build.sh:36

[azd-code-reviewer] This local shell build targets the evaluation module, leaving the dataset binary's build metadata at its development defaults. Use this extension's module path.

VERSION_PATH="azureaieval/internal/version"

cli/azd/extensions/azure.ai.dataset/README.md:41

[azd-code-reviewer] The documented cascade omits two implemented sources: the shared project context and the AZURE_AI_PROJECT_ENDPOINT fallback. This makes the stated precedence inaccurate and hides why a command may use an endpoint other than the listed values.

Every command resolves the Foundry project endpoint in this order:

1. `--project-endpoint`
2. `FOUNDRY_PROJECT_ENDPOINT` in the active azd environment
3. the host environment variable of the same name

cli/azd/extensions/azure.ai.dataset/README.md:29

[azd-code-reviewer] This points readers to a command that the companion evaluation extension does not expose (#9500 defines azd ai eval generate) and names the obsolete config path. A user following this section gets an unknown command before reaching dataset creation.

Generation is `azd ai eval dataset generate`, in `azure.ai.evaluations`, and
stays there: it writes the `datasets:` entry in `evals/eval.yaml`, which is that
extension's file. Splitting the two would leave a generated dataset registered

cli/azd/extensions/azure.ai.dataset/ci-test.ps1:10

[azd-code-reviewer] The live harness checks AZURE_AI_DATASET_E2E_LIVE, not this evaluation variable. Following this CI documentation causes the suite to print its skip message and exit successfully instead of running.

# AZURE_AI_EVAL_E2E_LIVE and a project endpoint. They are still type-checked

cli/azd/extensions/azure.ai.dataset/.gitignore:5

[azd-code-reviewer] Debug logging now writes azd-ai-dataset-*.log, so this inherited evaluation pattern does not ignore the generated files. Running with --debug leaves untracked logs in the extension directory.

azd-ai-eval-*.log

cli/azd/extensions/azure.ai.dataset/internal/exterrors/errors.go:95

[azd-code-reviewer] Use errors.AsType[*azcore.ResponseError] here; the repository's Go 1.26 modernization rules prohibit the mutable-target errors.As form.
cli/azd/extensions/azure.ai.dataset/tests/cli/dataset_test.go:51
[azd-code-reviewer] Cleanup silently returns on list/JSON failures and ignores every delete result, so a green live run can leave dataset versions in the shared Foundry project. Make cleanup failures fail the test and show the command output instead of hiding leaked remote state.
cli/azd/extensions/azure.ai.dataset/CHANGELOG.md:11
[azd-code-reviewer] The companion extension exposes azd ai eval generate, not dataset generate. The release notes currently document a command users cannot run.

- The CRUD groups moved here from `azure.ai.evaluations`. Generation stayed
  there: `dataset generate` writes the `datasets:` entry in `evals/eval.yaml`,
  and that file belongs to the evaluation extension.

+	httpClient := blobHTTPClient()
+	resp, err := httpClient.Do(req.Raw())
+	if err != nil {
+		return nil, messages.DownloadingDatasetBlob(err)


Three review findings, all still live in the code despite being marked outdated when line numbers moved. latestRegisteredVersion swallowed every listing error and returned "", which is indistinguishable from a brand-new dataset. A 403, throttle, timeout or cancellation therefore restarted an existing dataset at 1.0 and issued a pending upload against it. It now returns (string, error): a 404 and an empty listing still report "" with no error, everything else is propagated. The retry loop keeps best-effort semantics, because the version it just had refused is already a correct next step. LatestVersion documented a fallback to the service's last entry when nothing is orderable, but bestOrder started at -2.0 and VersionOrder returns -1 for an unorderable version, so the *first* one always won and the fallback was unreachable. Starting at -1.0 makes the documented behaviour real. Also finishes the errors.As -> errors.AsType migration required by cli/azd/AGENTS.md; no errors.As calls remain in the extension. Tests cover all three, and were checked by reverting each fix: - a refused listing must surface and must not reach startPendingUpload - an unknown dataset (404) still starts at 1.0 - an all-unorderable list returns the last entry, not the first

The paged response modelled nextLink but neither list method followed it, so a project with more than one page listed incompletely and latestRegisteredVersion could pick a version from a stale first page. followPages walks the links and returns one merged list. Two guards come with it: - Same-origin check. nextLink is service-supplied and the pipeline attaches the caller's token, so a link off the endpoint's origin is refused rather than followed; otherwise a spoofed or redirected response could hand that token to another host. - Repeat and count cap. A service that keeps returning the same link stops the walk instead of spinning; that is the service misbehaving, not a reason to fail the user's command. Also corrects the envKeyDatasetVersion comment. It claimed the key caches a version "so a later read does not have to list every version" — no code in either extension reads it, so the comment described an optimization that does not exist. Tests: pages merge in order, an off-origin link fails the call and the other host is never contacted, and a repeated link terminates.

Copilot

Pull request overview

Copilot reviewed 57 out of 58 changed files in this pull request and generated 1 comment.

Suppressed comments (17)

cli/azd/extensions/azure.ai.dataset/internal/pkg/dataset_api/list.go:19

Both list methods stop after the first response even though this model exposes NextLink. In a paged project, list is incomplete and LatestVersion can select an old version, causing update to target the wrong version. Follow all continuation links and validate that each link has the same origin as the configured endpoint.
cli/azd/extensions/azure.ai.dataset/internal/cmd/dataset.go:93
The list error is converted into exists == false. For update, a 401, 403, or 5xx is then swallowed along with both probe errors, and the user receives “does not exist” instead of the authentication/service failure. Propagate non-404 failures from both the listing and point probes.

			existing, err := ec.datasetClient.ListDatasetVersions(
				ctx, name, ProjectEndpointAPIVersion,
			)
			exists := err == nil && existing != nil && len(existing.Value) > 0
			if !exists {

cli/azd/extensions/azure.ai.dataset/internal/cmd/dataset.go:232

An unknown dataset can be represented by a 404—the same client already handles that case in latestRegisteredVersion. Returning every error here contradicts the documented/tested contract that versions list <unknown> succeeds with an empty result.

			list, err := ec.datasetClient.ListDatasetVersions(ctx, name, ProjectEndpointAPIVersion)
			if err != nil {
				return messages.ListingDatasetVersions(name, err)
			}

cli/azd/extensions/azure.ai.dataset/internal/cmd/dataset.go:298

ListDatasetVersions is documented elsewhere in this client as eventually consistent, but show reads it only once. Immediately after a successful create or update, an empty stale page therefore becomes a false “dataset not found” error. Add a bounded retry/backoff for an empty latest-version listing before concluding the dataset is absent.

			if version == "" {
				list, err := ec.datasetClient.ListDatasetVersions(ctx, name, ProjectEndpointAPIVersion)
				if err != nil {
					return messages.ResolvingLatestDatasetVersion(name, err)
				}

cli/azd/extensions/azure.ai.dataset/internal/cmd/dataset.go:366

The command converts a 404 into failure even though the CLI contract and TestCLIDeleteIsIdempotent require deleting an absent version to succeed. Ignore not-found responses and emit the normal idempotent success result; only other failures should be returned.

			); err != nil {
				if dataset_api.IsNotFound(err) {
					return messages.DatasetVersionNotFound(name, version)
				}
				return messages.DeletingDatasetVersion(name, version, err)

cli/azd/extensions/azure.ai.dataset/internal/pkg/dataset_api/models.go:51

These wire-format aliases are also serialized directly by every -o json command. A response using snake_case therefore exposes data_uri, while the same dataset from another endpoint exposes dataUri (and a mixed response can expose both), breaking the stable script-facing shape asserted by the CLI tests. Decode aliases internally, then map to a dedicated canonical output model before rendering JSON.
cli/azd/extensions/azure.ai.dataset/ci-build.ps1:45
The release pipeline invokes this script, but these linker paths still target the evaluation module. The dataset module’s version variables are never stamped, so released requests keep the azd-ext-azure-ai-dataset/dev User-Agent instead of the published version.

    "-X 'azureaieval/internal/version.Version=$Version' " +
    "-X 'azureaieval/internal/version.Commit=$SourceVersion' " +
    "-X 'azureaieval/internal/version.BuildDate=$(Get-Date -Format o)' "

cli/azd/extensions/azure.ai.dataset/build.sh:36

This copied linker path targets azureaieval, so locally packaged binaries do not stamp this module’s Version, Commit, or BuildDate. Point it at the module declared by this extension.

VERSION_PATH="azureaieval/internal/version"

cli/azd/extensions/azure.ai.dataset/build.ps1:44

This copied linker path targets azureaieval, so locally packaged binaries do not stamp this module’s Version, Commit, or BuildDate. Point it at the module declared by this extension.

$VERSION_PATH = "azureaieval/internal/version"

cli/azd/extensions/azure.ai.dataset/README.md:6

This quickstart cannot currently work: the same README states that no registry entry exists, so azd extension install azure.ai.dataset cannot resolve. Document the local pack/publish installation path here, or defer this install example until the registry entry lands.

$ azd extension install azure.ai.dataset

cli/azd/extensions/azure.ai.dataset/README.md:22

The documented delete invocation omits the required --version flag, so copying it always fails with “--version is required.”

| `azd ai dataset delete <name>` | Delete a dataset version |

cli/azd/extensions/azure.ai.dataset/README.md:28

The companion evaluation surface exposes azd ai eval generate, not azd ai eval dataset generate, and its default configuration is azure.eval.yaml (with compatibility for an existing eval.yaml), not evals/eval.yaml. These instructions currently lead users to a nonexistent command and file.

Generation is `azd ai eval dataset generate`, in `azure.ai.evaluations`, and
stays there: it writes the `datasets:` entry in `evals/eval.yaml`, which is that

cli/azd/extensions/azure.ai.dataset/README.md:41

This list omits two implemented resolution sources and the AZURE_AI_PROJECT_ENDPOINT fallback, so users may not understand why a different endpoint won. Document the full cascade implemented by projectctx.Resolve.

1. `--project-endpoint`
2. `FOUNDRY_PROJECT_ENDPOINT` in the active azd environment
3. the host environment variable of the same name

cli/azd/extensions/azure.ai.dataset/.gitignore:5

Debug logging creates azd-ai-dataset-<date>.log, but this copied pattern ignores only evaluation logs. Dataset debug logs will therefore appear as untracked files and can be committed accidentally.

azd-ai-eval-*.log

cli/azd/extensions/azure.ai.dataset/tests/cli/dataset_test.go:51

Cleanup silently returns when listing or JSON decoding fails, and delete failures are also ignored. A live test can therefore pass while leaving every created dataset version in the shared Foundry project. Fail/report these operations so leaked test assets are visible.
cli/azd/extensions/azure.ai.dataset/internal/messages/messages.go:195
This error is specifically about a missing version, but it sends the user to dataset list, which does not enumerate that dataset’s versions. Point to dataset versions list <name> so the suggested command can actually find a valid version.
cli/azd/extensions/azure.ai.dataset/ci-test.ps1:10
The live suite is gated by AZURE_AI_DATASET_E2E_LIVE, not the copied evaluation variable named here. Anyone following this CI documentation will set the wrong variable and the suite will exit without running.

# AZURE_AI_EVAL_E2E_LIVE and a project endpoint. They are still type-checked

Copilot

Pull request overview

Copilot reviewed 57 out of 58 changed files in this pull request and generated 1 comment.

Suppressed comments (12)

cli/azd/extensions/azure.ai.dataset/ci-build.ps1:45

These linker overrides still target the evaluation module, so the release pipeline silently leaves this binary's Version, Commit, and BuildDate at dev/none/unknown. Stamp azureaidataset/internal/version instead so the dataset User-Agent carries the released identity.

    "-X 'azureaieval/internal/version.Version=$Version' " +
    "-X 'azureaieval/internal/version.Commit=$SourceVersion' " +
    "-X 'azureaieval/internal/version.BuildDate=$(Get-Date -Format o)' "

cli/azd/extensions/azure.ai.dataset/build.sh:36

The local shell build stamps azureaieval, which is not this module, so dataset binaries keep the fallback version metadata. Point the linker path at this module.

VERSION_PATH="azureaieval/internal/version"

cli/azd/extensions/azure.ai.dataset/build.ps1:44

The local PowerShell build stamps azureaieval, which is not this module, so dataset binaries keep the fallback version metadata. Point the linker path at this module.

$VERSION_PATH = "azureaieval/internal/version"

cli/azd/extensions/azure.ai.dataset/.gitignore:5

Debug logging creates azd-ai-dataset-<date>.log, but this copied pattern ignores eval logs instead. Running the new extension with --debug therefore leaves its credential-sensitive diagnostic log visible as an untracked file.

azd-ai-eval-*.log

cli/azd/extensions/azure.ai.dataset/README.md:28

Both references are stale relative to the companion evaluation extension: its command surface is azd ai eval generate, and its canonical configuration is azure.eval.yaml (while preserving an existing eval.yaml), not evals/eval.yaml. As written, users are sent to a nonexistent command and file.

Generation is `azd ai eval dataset generate`, in `azure.ai.evaluations`, and
stays there: it writes the `datasets:` entry in `evals/eval.yaml`, which is that

cli/azd/extensions/azure.ai.dataset/README.md:41

This omits two resolution levels implemented by projectctx.Resolve: the shared global project context and AZURE_AI_PROJECT_ENDPOINT fallbacks in both azd and host environments. Document the full precedence order; otherwise users can neither predict which endpoint wins nor discover the supported fallback.

1. `--project-endpoint`
2. `FOUNDRY_PROJECT_ENDPOINT` in the active azd environment
3. the host environment variable of the same name

cli/azd/extensions/azure.ai.dataset/internal/pkg/dataset_api/operations.go:515

This direct-download path wraps the raw *url.Error, whose URL contains the SAS sig query. On a transport failure the credential can therefore appear both in the user-facing error and in the debug log at the fallback call site. Apply the same redactURLError used by the other blob operations before wrapping it.
cli/azd/extensions/azure.ai.dataset/README.md:6
This quick start cannot currently work: the PR deliberately omits the registry.json entry, and lines 54-57 state that azd extension install azure.ai.dataset cannot resolve yet. Replace this with the local pack/publish workflow for now, or add the install command only when the registry entry lands.

$ azd extension install azure.ai.dataset

cli/azd/extensions/azure.ai.dataset/README.md:22

The documented delete invocation omits --version, but the implementation rejects every delete without that flag. Include the required flag so copying this command does not immediately fail.

| `azd ai dataset delete <name>` | Delete a dataset version |

cli/azd/extensions/azure.ai.dataset/internal/cmd/dataset.go:26

This probe does not cover every first version this CLI can publish. Because create also exposes --version, create --version 4.0 publishes 5.0; during listing lag, the subsequent update probes only 1.0 and 1 and incorrectly reports that the dataset does not exist. Either remove --version from create or include NextVersion(version) in the fallback probes.

var firstDatasetVersions = []string{dataset_api.NextVersion(""), "1"}

cli/azd/extensions/azure.ai.dataset/internal/cmd/dataset.go:93

A listing failure is currently converted into exists == false, and every direct-probe error is also ignored. A 403, timeout, or token failure therefore becomes the misleading “does not exist” result for update, while create may continue toward an upload. Only 404 should mean absence; propagate other errors.

			existing, err := ec.datasetClient.ListDatasetVersions(
				ctx, name, ProjectEndpointAPIVersion,
			)
			exists := err == nil && existing != nil && len(existing.Value) > 0

cli/azd/extensions/azure.ai.dataset/internal/cmd/dataset.go:232

The comment promises that an unknown name returns an empty successful list, but a 404 from the versions endpoint is returned as an error here. The API client already treats 404 as the normal “no versions” case in latestRegisteredVersion; normalize it here too so the command's documented behavior is consistent across service responses.

			list, err := ec.datasetClient.ListDatasetVersions(ctx, name, ProjectEndpointAPIVersion)
			if err != nil {
				return messages.ListingDatasetVersions(name, err)
			}

Pagination, from the review of the nextLink commit: - A relative nextLink was refused. url.Parse leaves scheme and host empty, so the same-origin check rejected a legitimate link with a message naming no host, turning a working listing into a hard failure. The link is now resolved against the endpoint first, which keeps the origin guarantee intact. - A page answering 200 with no body failed the whole listing and discarded every page already collected. An empty page now ends the walk. - The page cap and repeat-link exits truncated silently. They now log, because a short list that nobody knows is short is how a stale latest-version gets chosen. - followPages appended into the first page's backing array, which the caller still owns. It copies now. Changelog: this is the extension's first release, so both changelogs describe what it does rather than accumulating fixes from our own pre-release cycles. --fail-on help now states the exit code a caller observes (1) without explaining why it is not the one the process exits with.

Copilot

Pull request overview

Copilot reviewed 58 out of 59 changed files in this pull request and generated no new comments.

Suppressed comments (13)

cli/azd/extensions/azure.ai.dataset/internal/cmd/dataset.go:89

[azd-code-reviewer] The point-read probe also treats every error as “not found.” If authentication expires or the service fails after an empty listing, update reports that the asset is absent. Ignore only 404 responses and surface other probe failures.

				for _, v := range firstDatasetVersions {
					if _, err := ec.datasetClient.GetDataset(
						ctx, name, v, ProjectEndpointAPIVersion,
					); err == nil {
						exists = true

cli/azd/extensions/azure.ai.dataset/ci-build.ps1:45

[azd-code-reviewer] These linker targets still use the evaluation module path. The release pipeline invokes this script, so the assignments miss azureaidataset/internal/version.Version and every published binary sends azd-ext-azure-ai-dataset/dev instead of its release version. Stamp the dataset module path here.

    "-X 'azureaieval/internal/version.Version=$Version' " +
    "-X 'azureaieval/internal/version.Commit=$SourceVersion' " +
    "-X 'azureaieval/internal/version.BuildDate=$(Get-Date -Format o)' "

cli/azd/extensions/azure.ai.dataset/build.sh:36

[azd-code-reviewer] The local cross-platform build also stamps the copied evaluation module path, so these binaries retain the dev user-agent version. Point VERSION_PATH at this module.

VERSION_PATH="azureaieval/internal/version"

cli/azd/extensions/azure.ai.dataset/build.ps1:44

[azd-code-reviewer] This PowerShell build stamps azureaieval, not the dataset module, leaving version.Version at dev. Use the module path declared by this extension.

$VERSION_PATH = "azureaieval/internal/version"

cli/azd/extensions/azure.ai.dataset/internal/cmd/dataset.go:80

[azd-code-reviewer] A listing failure is currently converted into exists == false. In particular, update turns a 403, timeout, or service failure into the misleading “dataset does not exist” error instead of preserving the real failure. Only a 404/empty result should enter the consistency probe.

This issue also appears on line 85 of the same file.

			existing, err := ec.datasetClient.ListDatasetVersions(
				ctx, name, ProjectEndpointAPIVersion,
			)
			exists := err == nil && existing != nil && len(existing.Value) > 0
			if !exists {

cli/azd/extensions/azure.ai.dataset/internal/cmd/dataset.go:366

[azd-code-reviewer] This contradicts the new idempotent-delete contract and its live test: whenever the service returns 404 rather than 204 for an absent version, cleanup fails. Treat not-found as successful deletion and only wrap other failures.

			if err := ec.datasetClient.DeleteDatasetVersion(
				ctx, name, version, ProjectEndpointAPIVersion,
			); err != nil {
				if dataset_api.IsNotFound(err) {
					return messages.DatasetVersionNotFound(name, version)

cli/azd/extensions/azure.ai.dataset/internal/cmd/root.go:52

[azd-code-reviewer] None of these runnable commands registers SDK flag options for inherited --output. As a result, arbitrary values such as -o yaml succeed and silently render a table, while generated metadata cannot advertise the supported values. Register json/table with a table default on each runnable command, following extensions/azure.ai.skills/internal/cmd/skill_list.go:51.

	rootCmd.AddCommand(
		newDatasetCreateCommand(),
		newDatasetUpdateCommand(),
		newDatasetListCommand(),
		newDatasetShowCommand(),
		newDatasetDeleteCommand(),
		newDatasetVersionsCommand(),
	)

cli/azd/extensions/azure.ai.dataset/internal/messages/messages.go:358

[azd-code-reviewer] This helper is also called when --no-prompt was not supplied, so dataset create <name> and dataset delete <name> falsely tell users they are running non-interactively. Since these commands do not prompt in either mode, report only that the flag is required.
cli/azd/extensions/azure.ai.dataset/README.md:31
[azd-code-reviewer] The companion extension exposes azd ai eval generate, not azd ai eval dataset generate, so this onboarding command fails with “unknown command.” The companion PR also names azure.eval.yaml; avoid pinning the stale evals/eval.yaml path here.

Generation is `azd ai eval dataset generate`, in `azure.ai.evaluations`, and
stays there: it writes the `datasets:` entry in `evals/eval.yaml`, which is that
extension's file. Splitting the two would leave a generated dataset registered
with the service but absent from the configuration, so `azd up` would not
reconcile it and no eval could name it.

cli/azd/extensions/azure.ai.dataset/README.md:41

[azd-code-reviewer] This documented cascade omits two implemented sources and therefore gives the wrong precedence. Global project context is checked before host variables, and AZURE_AI_PROJECT_ENDPOINT is a fallback at both environment levels.

Every command resolves the Foundry project endpoint in this order:

1. `--project-endpoint`
2. `FOUNDRY_PROJECT_ENDPOINT` in the active azd environment
3. the host environment variable of the same name

cli/azd/extensions/azure.ai.dataset/.gitignore:5

[azd-code-reviewer] Debug mode creates azd-ai-dataset-<date>.log, but this copied pattern ignores eval logs instead. Dataset debug logs will remain untracked files in the extension directory.

# Debug log written when --debug or AZD_EXT_DEBUG is set.
azd-ai-eval-*.log

cli/azd/extensions/azure.ai.dataset/tests/cli/dataset_test.go:51

[azd-code-reviewer] Cleanup silently returns when listing or JSON decoding fails, and it also ignores each delete result. A transient cleanup failure therefore leaves live Foundry datasets behind while the suite still passes. Assert every cleanup operation so leaked state is reported.
cli/azd/extensions/azure.ai.dataset/CHANGELOG.md:17
[azd-code-reviewer] No CLI command calls DownloadDatasetContent; only internal tests do. This release note therefore advertises a content-read capability users cannot access. Remove the bullet or add the corresponding public command before release.

- Reads dataset content back, whether the service hands out a blob URI or the
  container holding it.

The feed still serves beta.5, which predates the SAS redaction, the version-selection error propagation and pagination. Bumping so a bug bash tester can tell which build they have.

Copilot

Pull request overview

Copilot reviewed 58 out of 59 changed files in this pull request and generated no new comments.

Suppressed comments (15)

cli/azd/extensions/azure.ai.dataset/ci-build.ps1:45

These linker targets still use the evaluation module path, so the release pipeline's -X flags do not stamp this extension's Version, Commit, or BuildDate. In particular, the dataset User-Agent continues to report dev, defeating the identity/versioning change. Target azureaidataset/internal/version instead.

    "-X 'azureaieval/internal/version.Version=$Version' " +
    "-X 'azureaieval/internal/version.Commit=$SourceVersion' " +
    "-X 'azureaieval/internal/version.BuildDate=$(Get-Date -Format o)' "

cli/azd/extensions/azure.ai.dataset/build.ps1:44

The local PowerShell build also points its linker stamps at the evaluation module, so locally packaged dataset binaries retain the default dev/none/unknown values. Use this module's import path.

$VERSION_PATH = "azureaieval/internal/version"

cli/azd/extensions/azure.ai.dataset/build.sh:36

The shell build stamps azureaieval, which is not this module, so its generated binaries do not receive the dataset version metadata. Use the dataset module path.

VERSION_PATH="azureaieval/internal/version"

cli/azd/extensions/azure.ai.dataset/README.md:28

Both references are stale relative to the companion evaluation extension: its surface is azd ai eval generate, and its canonical configuration is azure.eval.yaml, not an eval dataset generate command writing evals/eval.yaml. As written, the documented command does not exist.

Generation is `azd ai eval dataset generate`, in `azure.ai.evaluations`, and
stays there: it writes the `datasets:` entry in `evals/eval.yaml`, which is that

cli/azd/extensions/azure.ai.dataset/README.md:41

This advertised resolution order omits two implemented sources: the shared project context is checked before host variables, and AZURE_AI_PROJECT_ENDPOINT is a fallback at both environment levels. Documenting only three levels makes endpoint selection surprising when the omitted higher-precedence value wins.

Every command resolves the Foundry project endpoint in this order:

1. `--project-endpoint`
2. `FOUNDRY_PROJECT_ENDPOINT` in the active azd environment
3. the host environment variable of the same name

cli/azd/extensions/azure.ai.dataset/.gitignore:5

Debug logging now writes azd-ai-dataset-*.log, but this copied evaluation pattern no longer matches it. Running with --debug from the extension directory leaves generated logs unignored.

azd-ai-eval-*.log

cli/azd/extensions/azure.ai.dataset/internal/cmd/dataset.go:80

A non-404 listing failure is discarded here. For example, a 403 or timeout makes update report that the dataset does not exist, while create proceeds toward an upload without establishing that the name is free. Only not-found/empty results should enter the eventual-consistency probe; propagate other listing failures.

			existing, err := ec.datasetClient.ListDatasetVersions(
				ctx, name, ProjectEndpointAPIVersion,
			)
			exists := err == nil && existing != nil && len(existing.Value) > 0
			if !exists {

cli/azd/extensions/azure.ai.dataset/internal/cmd/dataset.go:299

This treats the eventually consistent version listing as authoritative even though the upload path explicitly handles it lagging after a publish. Immediately after create, show can incorrectly return “no dataset”; immediately after update, it can show the previous version as latest. Resolve latest with point reads past the listed version (bounded, as the upload conflict walk is) before rendering or declaring the dataset absent.

				list, err := ec.datasetClient.ListDatasetVersions(ctx, name, ProjectEndpointAPIVersion)
				if err != nil {
					return messages.ResolvingLatestDatasetVersion(name, err)
				}
				// The service answers an unknown name with an empty list rather

cli/azd/extensions/azure.ai.dataset/internal/cmd/output.go:29

No leaf command registers azdext.RegisterFlagOptions, so unsupported values such as -o yaml are accepted and silently fall through to human output. Register the supported json/table values (and the intended default) on every runnable command so help, metadata, completion, and parse-time validation agree with the renderer.

	v, err := cmd.Flags().GetString("output")
	if err != nil {
		return ""
	}
	return strings.ToLower(v)

cli/azd/extensions/azure.ai.dataset/internal/pkg/dataset_api/models.go:44

The service compatibility fields leak into -o json, because commands encode Dataset directly. The same logical URI is therefore emitted as dataUri or data_uri depending on which service spelling arrived, giving scripts an unstable schema. Keep both fields for decoding, but render a normalized output DTO with one canonical field name.
cli/azd/extensions/azure.ai.dataset/README.md:6
This first-run command cannot work in this PR: the same README states that the registry entry is deliberately absent, so azd extension install azure.ai.dataset cannot resolve. Replace this with complete local pack/publish instructions until the registry entry lands, or defer the install snippet with that release change.

$ azd extension install azure.ai.dataset

cli/azd/extensions/azure.ai.dataset/README.md:22

The documented delete invocation omits the required --version flag, so copying it always fails with “--version is required.” Include the required argument in the command table.

| `azd ai dataset delete <name>` | Delete a dataset version |

cli/azd/extensions/azure.ai.dataset/tests/cli/dataset_test.go:51

Cleanup silently returns when listing/decoding fails and ignores every delete result, so failed live runs can leave datasets behind while still reporting successful cleanup. Make cleanup failures visible and require each delete to succeed.
cli/azd/extensions/azure.ai.dataset/internal/foundry/projectctx/resolver.go:139
These environment-variable reads are not fully reflected in the repository's environment-variable reference: FOUNDRY_PROJECT_ENDPOINT currently names only azure.ai.agents, and AZURE_AI_PROJECT_ENDPOINT has no entry. Update cli/azd/docs/environment-variables.md so both new extension inputs and their precedence are documented.
cli/azd/extensions/azure.ai.dataset/internal/cmd/root.go:21
The PR description says every user-facing string lives in internal/messages/messages.go, but this root help text and the command/flag help strings in dataset.go are defined inline. Either centralize those strings as claimed or narrow the PR description; the current “69 messages in one file” review guarantee is not true.

		Short: fmt.Sprintf(
			"Register and version Foundry datasets from your terminal. %s",
			color.YellowString("(Beta)"),

Both pass, so this is coverage rather than a fix, but neither was guarded and both are the kind that fail silently. A protocol-relative nextLink (//otherhost/path) inherits our scheme, so adding ResolveReference is precisely the change that could have opened a bypass; the test asserts the other host receives zero requests, since the pipeline attaches the caller's token. And a two-hop A-to-B-to-A cycle defeats any guard that only remembers the previous link, which is what the sibling eval paginator turned out to have.

Copilot

Pull request overview

Copilot reviewed 59 out of 60 changed files in this pull request and generated no new comments.

Suppressed comments (10)

cli/azd/extensions/azure.ai.dataset/internal/cmd/dataset.go:232

The API client explicitly treats a 404 from this endpoint as an unknown/versionless dataset, but this command returns that 404 before reaching the advertised empty-list behavior. Convert IsNotFound(err) to an empty DatasetList; otherwise versions list <unknown> still exits nonzero.

			list, err := ec.datasetClient.ListDatasetVersions(ctx, name, ProjectEndpointAPIVersion)
			if err != nil {
				return messages.ListingDatasetVersions(name, err)
			}

cli/azd/extensions/azure.ai.dataset/ci-build.ps1:45

These linker targets still use the evaluation module path. This module is azureaidataset, so the release build does not stamp its Version, Commit, or BuildDate variables and the dataset User-Agent remains .../dev.

    "-X 'azureaieval/internal/version.Version=$Version' " +
    "-X 'azureaieval/internal/version.Commit=$SourceVersion' " +
    "-X 'azureaieval/internal/version.BuildDate=$(Get-Date -Format o)' "

cli/azd/extensions/azure.ai.dataset/build.ps1:44

This copied evaluation module path means local PowerShell builds do not stamp azureaidataset/internal/version; the binary keeps the development version in its User-Agent. Point VERSION_PATH at this module.

$VERSION_PATH = "azureaieval/internal/version"

cli/azd/extensions/azure.ai.dataset/build.sh:36

This copied evaluation module path means shell builds do not stamp azureaidataset/internal/version; the binary keeps the development version in its User-Agent. Point VERSION_PATH at this module.

VERSION_PATH="azureaieval/internal/version"

cli/azd/extensions/azure.ai.dataset/.gitignore:5

Debug logging creates azd-ai-dataset-YYYY-MM-DD.log, but this pattern only ignores the old evaluation filename. Dataset debug logs will remain unignored and can be committed accidentally.

azd-ai-eval-*.log

cli/azd/extensions/azure.ai.dataset/ci-test.ps1:10

The live harness checks AZURE_AI_DATASET_E2E_LIVE, so this setup instruction names an environment variable that cannot enable the suite.

# AZURE_AI_EVAL_E2E_LIVE and a project endpoint. They are still type-checked

cli/azd/extensions/azure.ai.dataset/README.md:28

Both references are stale: the evaluation extension exposes azd ai eval generate, not azd ai eval dataset generate, and its configuration is azure.eval.yaml. As written, the documented command fails and points readers at the obsolete filename.

Generation is `azd ai eval dataset generate`, in `azure.ai.evaluations`, and
stays there: it writes the `datasets:` entry in `evals/eval.yaml`, which is that

cli/azd/extensions/azure.ai.dataset/README.md:41

This omits two implemented resolution levels and the AZURE_AI_PROJECT_ENDPOINT fallback. Users relying on shared project context or the legacy key cannot infer why a value wins; document the same four-level order implemented by projectctx.Resolve.

1. `--project-endpoint`
2. `FOUNDRY_PROJECT_ENDPOINT` in the active azd environment
3. the host environment variable of the same name

cli/azd/extensions/azure.ai.dataset/internal/cmd/dataset.go:93

Listing failures are currently indistinguishable from an absent dataset. A 403 or transport error makes exists false, and both point probes discard their errors too, so update reports “does not exist” while create may proceed despite being unable to verify existence. Propagate non-404 errors and only treat empty/404 responses as absence.

This issue also appears on line 229 of the same file.

			exists := err == nil && existing != nil && len(existing.Value) > 0

cli/azd/extensions/azure.ai.dataset/CHANGELOG.md:17

No command in this extension calls DownloadDatasetContent, so the first-release changelog promises a read-back capability that users cannot invoke. Remove this feature bullet or add the corresponding public command before release.

- Reads dataset content back, whether the service hands out a blob URI or the
  container holding it.

Mohamed Hessien (m7md7sien) added 7 commits August 9, 2026 17:24

Stop the same panic and leak here

93516a2

Carved from the same code, so it had the same two: a nil Environment or GetValue response panicking inside the endpoint cascade, and an azd client abandoned when endpoint or credential setup failed.

Say when the answer was 'log in again' here too

2e8f8fb

Same credential failure and same 401/403 path as the eval extension, which this was carved from.

Tell people to retry before telling them to log in

88ffc5f

Same intermittent credential failure as the eval extension.

Refuse an empty dataset early, bound the blob calls, guard the nil list

dd0b293

Same three as the eval extension, which this was carved from.

microsoft-github-policy-service Bot added the customer-reported identify a customer issue label Aug 10, 2026

Mohamed Hessien (m7md7sien) added 2 commits August 10, 2026 03:50

Mohamed Hessien (m7md7sien) mentioned this pull request Aug 10, 2026

Add azd ai eval extension for Foundry evaluations #9500

Open

Mohamed Hessien (m7md7sien) added 4 commits August 10, 2026 05:52

Format the Go sources with gofmt

9df2b57

Say a --from-file path is missing, not which syscall found out

be18334

A mistyped path surfaced as 'GetFileAttributesEx <path>: The system cannot find the file specified.' The syscall name is noise to the person who mistyped it.

Mohamed Hessien (m7md7sien) marked this pull request as ready for review August 10, 2026 04:25

Mohamed Hessien (m7md7sien) requested review from Daniel Jurek (danieljurek), tg-msft and Victor Vazquez (vhvb1989) as code owners August 10, 2026 04:26

Copilot AI balanced review requested due to automatic review settings August 10, 2026 04:26

Mohamed Hessien (m7md7sien) requested review from JeffreyCA, Rick Winter (RickWinter), Marina He (hemarina) and Richard Park (richardpark-msft) as code owners August 10, 2026 04:26

Copilot started reviewing on behalf of Mohamed Hessien (m7md7sien) August 10, 2026 04:26 View session

Copilot AI reviewed Aug 10, 2026

View reviewed changes

Copilot AI review requested due to automatic review settings August 10, 2026 05:42

Copilot AI reviewed Aug 12, 2026

View reviewed changes

Copilot AI review requested due to automatic review settings August 13, 2026 01:44

Copilot started reviewing on behalf of Mohamed Hessien (m7md7sien) August 13, 2026 01:45 View session

Copilot AI reviewed Aug 13, 2026

View reviewed changes

Copilot AI review requested due to automatic review settings August 13, 2026 01:54

Copilot started reviewing on behalf of Mohamed Hessien (m7md7sien) August 13, 2026 01:55 View session

Copilot AI reviewed Aug 13, 2026

View reviewed changes

Comment thread cli/azd/extensions/azure.ai.dataset/internal/pkg/dataset_api/operations.go Outdated

httpClient := blobHTTPClient()

resp, err := httpClient.Do(req.Raw())

if err != nil {

return nil, messages.DownloadingDatasetBlob(err)

Copilot AI review requested due to automatic review settings August 13, 2026 02:15

Copilot started reviewing on behalf of Mohamed Hessien (m7md7sien) August 13, 2026 02:15 View session

Copilot AI reviewed Aug 13, 2026

View reviewed changes

Comment thread cli/azd/extensions/azure.ai.dataset/internal/pkg/dataset_api/operations.go

Copilot AI review requested due to automatic review settings August 13, 2026 02:24

Copilot started reviewing on behalf of Mohamed Hessien (m7md7sien) August 13, 2026 02:25 View session

Copilot AI reviewed Aug 13, 2026

View reviewed changes

Comment thread cli/azd/extensions/azure.ai.dataset/internal/cmd/dataset.go

Copilot AI review requested due to automatic review settings August 13, 2026 13:47

Copilot started reviewing on behalf of Mohamed Hessien (m7md7sien) August 13, 2026 13:48 View session

Copilot AI reviewed Aug 13, 2026

View reviewed changes

Bump to 1.0.0-beta.6

3faf386

The feed still serves beta.5, which predates the SAS redaction, the version-selection error propagation and pagination. Bumping so a bug bash tester can tell which build they have.

Copilot AI review requested due to automatic review settings August 13, 2026 14:02

Copilot started reviewing on behalf of Mohamed Hessien (m7md7sien) August 13, 2026 14:02 View session

Copilot AI reviewed Aug 13, 2026

View reviewed changes

Copilot AI review requested due to automatic review settings August 13, 2026 19:43

Copilot started reviewing on behalf of Mohamed Hessien (m7md7sien) August 13, 2026 19:43 View session

Copilot AI reviewed Aug 13, 2026

View reviewed changes

Navigation Menu

Uh oh!

Add azd ai dataset extension for Foundry datasets - #9499

Add azd ai dataset extension for Foundry datasets#9499
Mohamed Hessien (m7md7sien) wants to merge 33 commits into
Azure:mainfrom
m7md7sien:feat/azure-ai-dataset

Mohamed Hessien (m7md7sien) commented Aug 10, 2026 •

edited

Loading

Uh oh!

azure-pipelines Bot commented Aug 10, 2026

Uh oh!

azure-pipelines Bot commented Aug 10, 2026

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

Copilot AI left a comment

Uh oh!

Copilot AI left a comment

Uh oh!

Copilot AI left a comment

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

Copilot AI left a comment

Uh oh!

Copilot AI left a comment

Uh oh!

Copilot AI left a comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

		var respErr *azcore.ResponseError
		if !errors.As(err, &respErr) {

		var respErr *azcore.ResponseError
		if errors.As(err, &respErr) {

		cmd.Flags().StringVar(&version, "version", "",
		"Current version to increment from. Omit to increment from the latest registered version.")

Uh oh!

Conversation

Mohamed Hessien (m7md7sien) commented Aug 10, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Why its own extension

Surface

Notable

Verified

Not included, deliberately

Open question for reviewers

Uh oh!

azure-pipelines Bot commented Aug 10, 2026

Uh oh!

azure-pipelines Bot commented Aug 10, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Mohamed Hessien (m7md7sien) commented Aug 10, 2026 •

edited

Loading