Add large-payload tombstone fetch and purge-result RPCs for blob auto-purge

[wangbill (YunchuWang)]

Summary

Adds the authoritative proto contract for the large-payload blob auto-purge feature. This is the source-of-truth definition that the other PRs vendor locally and re-sync from once this merges:

• SDK: microsoft/durabletask-dotnet #758
• DTS backend: AAPT-DTMB #16368738

The definitions here are byte-identical to the canonical contract shared with durabletask-dotnet #758.

Context

When an orchestration payload exceeds a threshold it is externalized to Azure Blob Storage and a token is persisted in the backend's SQL Payloads.Text column instead of the payload bytes. Purging the orchestration removes the SQL state, but the backend cannot delete the blob because it does not hold the customer's storage credentials, so the blob is orphaned.

Fix: the backend keeps the durable cleanup work and hands it to a worker that does have storage credentials. The backend records a tombstone for each externalized payload leaving live state; the worker fetches a bounded batch of due tombstones, deletes each blob, and reports the outcome of every attempt. The backend then resolves, reschedules, or quarantines each row.

Change

Edits only protos/orchestrator_service.proto. Nothing is added to the backend-facing service: the backend serves both services on one endpoint and the worker already reaches TaskHubSidecarService there, so authorization stays identical to the existing worker stream.

Opt-in on the existing handshake — google.protobuf.BoolValue large_payload_auto_purge_enabled = 12 on the existing GetWorkItemsRequest. GetWorkItems is already task-hub scoped and authenticated, so the setting binds to that task hub without a new RPC. This is a setting, not a WorkerCapability: WORKER_CAPABILITY_LARGE_PAYLOADS means a worker can resolve externalized payloads, not that the customer opted into deleting them. A repeated capability enum is presence-only and could never express an explicit false, so it could not turn the feature off.

Two bounded unary RPCs, placed immediately after GetWorkItems:

RPC	Purpose
GetLargePayloadTombstones	Hand the worker a bounded, deterministically ordered batch of due tombstones
ReportLargePayloadPurgeResults	Record the outcome of each attempt; the backend owns retry scheduling

Dispositions — exactly three: DELETED, RETRY, QUARANTINED. The split is by whether a failure can self-heal. There is deliberately no Discarded: it was success-shaped and destroyed evidence.

No reason or error-code fields - disposition is the only outcome field on LargePayloadPurgeResult. Earlier revisions of this PR also carried a LargePayloadPurgeReason enum and a storageErrorCode string; both were removed because they were write-only. The backend persists them, but no SELECT, WHERE, API, or alert reads either one, and every backend action branches on disposition alone. Failure detail already lives in the worker's telemetry at full exception fidelity, where a row is correlated by (partitionId, instanceKey, payloadId), so the fields were a lossy copy nobody queried. No reserved statements were added: these messages have never shipped in a release, so no deployed peer has ever seen field 6 or 7. disposition stays at field 5 rather than being renumbered to close the gap.

revision — carried on both the tombstone and the result as a compare-and-swap guard, so duplicate or stale reports are no-ops without a per-row lease.

Tokens — documented as blob:v2:{fullBlobUrl}. Legacy v1 tokens are never tombstoned: v1 carries a container name but not the storage account, so a delete cannot be verified against the configured account. The backend hard-deletes v1 payload rows instead, and a v1 token reaching the worker is an invariant violation that is quarantined rather than discarded.

Supersedes the earlier shape

This replaces the previous draft contract, which is fully removed rather than extended:

• GetTombstonedPayloads / AckPurgedPayloads → renamed and reshaped. The old pair was a fetch plus a bare identity-only ack, which cannot distinguish success from an unfixable failure and left retry scheduling ambiguous.
• Messages TombstonedPayload, PayloadPurgeAck, GetTombstonedPayloadsRequest, GetTombstonedPayloadsResponse, AckPurgedPayloadsRequest, AckPurgedPayloadsResponse → removed.
• protos/backend_service.proto → reverted; it is now byte-identical to main.

Because none of the related PRs has merged, this is not a compatibility constraint.

Verification

• git diff origin/main -- protos/backend_service.proto is empty.
• Field 12 was previously unused in GetWorkItemsRequest (which used 1, 2, 3, 10, 11); no field number collides.
• google/protobuf/wrappers.proto was already imported; no duplicate import added.
• All three protos compile with protoc (no errors; the only warnings are pre-existing unused imports in backend_service.proto). Descriptor introspection confirms the field numbers, RPC wiring, and enum values, and that no removed symbol survives.

No language stubs were regenerated: this is a proto-only repo (SDKs consume the protos via git submodule and generate code at build time into gitignored build/ dirs; no generated stubs are checked in).

[wangbill (YunchuWang)]

no need for this?

[wangbill (YunchuWang)]

Good catch, and you're right that this needed justifying rather than just existing.

It can happen, and that's exactly why it's here. proto3 has no field presence for scalars, so if a client never sets disposition, the field still arrives — as 0. There's no way for the backend to tell "client omitted this" from "client meant the value at 0".

That makes the value at 0 load-bearing. If 0 were DELETED, then a client that forgot to set the field, or an older/buggy SDK build, would report every row as a success. The backend would delete those tombstones, and since the tombstone is the only durable record of the blob, the blobs would be orphaned permanently with no way to find them again. Making 0 a value the backend rejects turns that failure into a loud no-op instead of silent data loss.

It's also not optional: proto3 requires the first enum value to be zero. WorkerCapability in this same file follows the identical convention (WORKER_CAPABILITY_UNSPECIFIED = 0), so this is consistent with what's already here.

I've added that reasoning as a comment on the value itself so the next reader doesn't have to ask.

[wangbill (YunchuWang)]

update your comments to explain the fields without mentioning the design doc

[wangbill (YunchuWang)]

Done — removed every design-doc reference from this file.

You're right that they don't belong here. This is a public repo, and an external reader has no way to resolve §5.5. It also rendered as mojibake in the diff view, since the section sign was the only non-ASCII byte in the file.

Five sites were affected: the token field, DISPOSITION_DELETED, the reason enum header, BLOB_NOT_STORE_OWNED, and the old LEGACY_V1_TOKEN. Each is rewritten to state the actual reason inline instead of pointing elsewhere. For example, BLOB_NOT_STORE_OWNED now explains that the payload column is customer-writable, so matching token text isn't proof the store wrote the blob — which was the whole point the reference was standing in for.

The file now has zero non-ASCII bytes.

[wangbill (YunchuWang)]

why need unspecified when can that happen

[wangbill (YunchuWang)]

Same reasoning as the disposition UNSPECIFIED thread, and yes — it can happen.

proto3 gives scalars no field presence, so a client that never sets reason still sends it as 0. There's no wire-level way to distinguish "not set" from "set to the value at 0". Keeping 0 meaningless is what lets the backend tell a genuine report apart from one where the field was never populated — otherwise an unset field would silently masquerade as a real diagnostic and send whoever is debugging a stalled ledger down the wrong path.

It's also required: proto3 mandates that the first enum value be zero. WorkerCapability in this same file does the same thing.

Lower stakes than the disposition one, since nothing branches on reason — but for that exact reason a wrong value here is pure misinformation, which is worth guarding against.

While addressing this, I also cut the enum from 11 values to 7 based on your other comment.

[wangbill (YunchuWang)]

why do we need purgereason code?

[wangbill (YunchuWang)]

Fair challenge — I checked before defending it, and you were right that it was overbuilt. Cut from 11 values to 7.

What I verified first: the backend has zero branches on reason. LargePayloadPurgePolicy never references it, and LastFailureReason shows up only in tests and one SELECT. It's write-only. disposition alone drives every backend action — delete, reschedule, quarantine.

So what is it for? Diagnostics, and only that. The worker runs in the customer's process, so when a task hub stops reclaiming blobs we can't read worker logs to find out why. reason is the one signal that crosses that boundary. Without it, a ledger that isn't draining looks identical whether storage is throttling, the store can't delete at all, or tokens are corrupt — three very different operator responses.

But that also sets the right granularity: it should match the number of distinct operator responses, not the number of distinct causes. storageErrorCode already carries the specific storage status, so four of the eleven values were encoding the same fact twice. Collapsed:

• TRANSIENT_STORAGE_FAILURE + STORAGE_ACCOUNT_UNREACHABLE + STORAGE_AUTHORIZATION_FAILED → STORAGE_FAILURE. All three are reconfigurable or self-healing and lead to the same response; the specific status stays in storageErrorCode.
• MALFORMED_TOKEN + INVALID_STORAGE_REQUEST + LEGACY_V1_TOKEN + UNSUPPORTED_TOKEN_VERSION → TOKEN_NOT_PURGEABLE. Each means the token can't be acted on and no retry changes that. storageErrorCode separates the storage-rejected case (populated) from the parse cases (empty).

Two I deliberately kept:

• STORE_CANNOT_DELETE is not folded into STORAGE_FAILURE because storage is never contacted, so storageErrorCode is empty. It's also deployment-wide rather than per-row — every payload fails identically — which is a different response.
• BLOB_NOT_STORE_OWNED stays because without it a DELETED row can't distinguish "reclaimed the blob" from "left it alone, wasn't ours". Your PurgedCount decision depends on that staying countable, and folding it in would make the metric claim reclaimed bytes that were never reclaimed.

Dispositions are unchanged — still the same four values with the same numbers. Worth flagging one thing explicitly: UNSUPPORTED_TOKEN_VERSION used to be a Retry reason and now folds into TOKEN_NOT_PURGEABLE, which sits under Quarantined. No row's disposition actually flips as a result; only the reason narrows, and an unreadable token still can't be deleted either way.