Skip to content

Enforce microVersionId conflicts at the metadata write level#6235

Open
maeldonn wants to merge 3 commits into
development/9.4from
improvement/CLDSRV-952
Open

Enforce microVersionId conflicts at the metadata write level#6235
maeldonn wants to merge 3 commits into
development/9.4from
improvement/CLDSRV-952

Conversation

@maeldonn

@maeldonn maeldonn commented Jul 23, 2026

Copy link
Copy Markdown
Contributor

The JS pre-checks in putMetadata are not atomic: two concurrent CRR writes can
both pass them and let a stale revision overwrite a newer one. This attaches a
conditional put ($gt on microVersionId) so the write itself rejects stale
revisions with PreconditionFailed, mapped to the same conflict response as the
pre-checks. Enabled only on backends supporting conditional puts (mongodb).

Issue: CLDSRV-952

@maeldonn
maeldonn requested review from a team, SylvainSenechal and delthas July 23, 2026 14:04
@bert-e

bert-e commented Jul 23, 2026

Copy link
Copy Markdown
Contributor

Hello maeldonn,

My role is to assist you with the merge of this
pull request. Please type @bert-e help to get information
on this process, or consult the user documentation.

Available options
name description privileged authored
/after_pull_request Wait for the given pull request id to be merged before continuing with the current one.
/bypass_author_approval Bypass the pull request author's approval
/bypass_build_status Bypass the build and test status
/bypass_commit_size Bypass the check on the size of the changeset TBA
/bypass_incompatible_branch Bypass the check on the source branch prefix
/bypass_jira_check Bypass the Jira issue check
/bypass_peer_approval Bypass the pull request peers' approval
/bypass_leader_approval Bypass the pull request leaders' approval
/approve Instruct Bert-E that the author has approved the pull request. ✍️
/create_pull_requests Allow the creation of integration pull requests.
/create_integration_branches Allow the creation of integration branches.
/no_octopus Prevent Wall-E from doing any octopus merge and use multiple consecutive merge instead
/unanimity Change review acceptance criteria from one reviewer at least to all reviewers
/wait Instruct Bert-E not to run until further notice.
Available commands
name description privileged
/help Print Bert-E's manual in the pull request.
/status Print Bert-E's current status in the pull request TBA
/clear Remove all comments from Bert-E from the history TBA
/retry Re-start a fresh build TBA
/build Re-start a fresh build TBA
/force_reset Delete integration branches & pull requests, and restart merge process from the beginning.
/reset Try to remove integration branches unless there are commits on them which do not appear on the source branch.

Status report is not available.

@maeldonn
maeldonn requested review from benzekrimaha and removed request for delthas July 23, 2026 14:04
@scality scality deleted a comment from bert-e Jul 23, 2026
@bert-e

bert-e commented Jul 23, 2026

Copy link
Copy Markdown
Contributor

Waiting for approval

The following approvals are needed before I can proceed with the merge:

  • the author

  • 2 peers

Comment thread package.json
Comment thread lib/routes/routeBackbeat.js Outdated
@codecov

codecov Bot commented Jul 23, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 86.08%. Comparing base (0d08680) to head (9b5d259).
✅ All tests successful. No failed tests found.

Additional details and impacted files

Impacted file tree graph

Files with missing lines Coverage Δ
lib/routes/routeBackbeat.js 79.04% <100.00%> (+0.30%) ⬆️

... and 2 files with indirect coverage changes

@@                 Coverage Diff                 @@
##           development/9.4    #6235      +/-   ##
===================================================
- Coverage            86.10%   86.08%   -0.02%     
===================================================
  Files                  212      212              
  Lines                14483    14490       +7     
===================================================
+ Hits                 12470    12474       +4     
- Misses                2013     2016       +3     
Flag Coverage Δ
file-ft-tests 70.09% <42.85%> (-0.07%) ⬇️
file-ft-tests-null-compat 70.62% <42.85%> (-0.02%) ⬇️
kmip-ft-tests 27.92% <0.00%> (-0.02%) ⬇️
mongo-v0-ft-tests 71.19% <57.14%> (-0.04%) ⬇️
mongo-v1-ft-tests 71.25% <57.14%> (+0.04%) ⬆️
multiple-backend 36.14% <42.85%> (+<0.01%) ⬆️
s3c-ft-tests-v0 65.12% <42.85%> (-0.02%) ⬇️
s3c-ft-tests-v0-null-compat 65.18% <42.85%> (-0.02%) ⬇️
s3c-ft-tests-v1 65.10% <42.85%> (-0.02%) ⬇️
sur-tests 34.77% <0.00%> (-0.91%) ⬇️
sur-tests-inflights 37.52% <0.00%> (-0.05%) ⬇️
unit 73.12% <100.00%> (+0.11%) ⬆️
utapi-v2-tests 35.09% <0.00%> (-0.02%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

🚀 New features to boost your workflow:
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@maeldonn maeldonn changed the title Enforce microVersionId conflict checks at the metadata write level Enforce microVersionId conflicts at the metadata write level Jul 23, 2026
The JS pre-checks in putMetadata are not atomic: two concurrent writes
can both pass them and let a stale revision overwrite a newer one.
Pass a conditional put to the metadata backend so the write itself
rejects stale microVersionIds with PreconditionFailed, mapped to the
same CRR conflict response as the pre-checks. Enabled only on backends
advertising conditional put support (mongodb).

Issue: CLDSRV-952
@maeldonn
maeldonn force-pushed the improvement/CLDSRV-952 branch from b85feb8 to ff93206 Compare July 23, 2026 14:30
@maeldonn
maeldonn marked this pull request as ready for review July 23, 2026 14:30
void (await endPromise);

sinon.assert.notCalled(putObjectMDStub);
assert.strictEqual(mockResponse.statusCode, 409);

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Only asserting statusCode is 409 doesn't distinguish this from the StaleMicroVersionIdException 409 response. Assert the body code so a regression that returns the wrong 409 type fails the test.

Suggested change
assert.strictEqual(mockResponse.statusCode, 409);
assert.strictEqual(mockResponse.statusCode, 409);
assert.strictEqual(mockResponse.body.code, 'MicroVersionIdAlreadyStoredException');

routeBackbeat('127.0.0.1', mockRequest, mockResponse, log);
void (await endPromise);

assert.strictEqual(mockResponse.statusCode, 409);

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same here — assert the body code so this test doesn't pass if a different 409 path fires.

Suggested change
assert.strictEqual(mockResponse.statusCode, 409);
assert.strictEqual(mockResponse.statusCode, 409);
assert.strictEqual(mockResponse.body.code, 'StaleMicroVersionIdException');

@benzekrimaha benzekrimaha left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Logic LGTM.

Comment thread package.json
"@opentelemetry/instrumentation-mongodb": "~0.69.0",
"@smithy/node-http-handler": "^3.0.0",
"arsenal": "git+https://github.com/scality/arsenal#8.5.6",
"arsenal": "git+https://github.com/scality/arsenal#fa9abec316dc0a48b9511eec5b41f7b9a626d5f4",

@benzekrimaha benzekrimaha Jul 24, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

reminder to point to the released arsenal tag containing ARSN-619 before merging (/after_pull_request on scality/Arsenal#2678 if needed).


// atomic counterpart of the pre-checks above ($gt cannot match a missing microVersionId)
if (incomingMicroVersionId !== null && objMd?.microVersionId && metadata.supportsConditionalPutObjectMD?.()) {
options.conditions = { microVersionId: { $gt: incomingMicroVersionId } };

@benzekrimaha benzekrimaha Jul 24, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

could the condition also cover the missing-field case, e.g. $or: [{ microVersionId: { $exists: false } }, { microVersionId: { $gt: … } }]? otherwise this window stays unprotected , if translateConditions cannot express it, worth mentionning in the comment.

@bert-e

bert-e commented Jul 24, 2026

Copy link
Copy Markdown
Contributor

Conflict

There is a conflict between your branch improvement/CLDSRV-952 and the
destination branch development/9.4.

Please resolve the conflict on the feature branch (improvement/CLDSRV-952).

git fetch && \
git checkout origin/improvement/CLDSRV-952 && \
git merge origin/development/9.4

Resolve merge conflicts and commit

git push origin HEAD:improvement/CLDSRV-952

@benzekrimaha
benzekrimaha self-requested a review July 24, 2026 15:36
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants