Skip to content

fix: bound media search and backfill requests - #4273

Merged
marinofaggiana merged 4 commits into
masterfrom
mediaLimitCalc
Aug 24, 2026
Merged

fix: bound media search and backfill requests#4273
marinofaggiana merged 4 commits into
masterfrom
mediaLimitCalc

Conversation

@marinofaggiana

@marinofaggiana marinofaggiana commented Aug 23, 2026

Copy link
Copy Markdown
Member

Summary

Fixes #4255 by replacing effectively unbounded media WebDAV searches with bounded requests.

This PR introduces two changes:

  • Media verification now uses a fixed search limit of 1,000 results.
  • Media metadata backfill now processes bounded batches of up to 1,000 results, split into four pages of 250.

Backfill behavior

  • Persists the oldest successfully processed date as a Realm cursor.
  • Checkpoints progress after every completed page.
  • Resumes from the saved date during the next background task.
  • Restarts safely from the beginning when migrating an incomplete legacy offset.
  • Cancels the underlying HTTP request when the background task expires.
  • Preserves correct pagination when hidden files are present.
  • Detects a non-advancing date cursor and stops instead of looping indefinitely.
  • Bumps the Realm schema version to 414.

Known limitation

Nextcloud WebDAV SEARCH supports sorting by displayname, but does not support range comparisons such as lte for that field. Pagination therefore cannot use a composite date-and-filename cursor.

If at least 1,000 media items share the exact same cursor date, the backfill stops safely instead of entering an infinite loop. A future background task may encounter the same boundary again, but existing items remain protected by ocId deduplication.

馃 AI (if applicable)

  • The content of this PR was partly or fully generated using AI

Signed-off-by: Marino Faggiana <marino.faggiana@nextcloud.com>
Signed-off-by: Marino Faggiana <marino.faggiana@nextcloud.com>
Use a fixed 1,000-item cap because Nextcloud cannot range-filter display names for same-date results.

Signed-off-by: Marino Faggiana <marino.faggiana@nextcloud.com>
Signed-off-by: Marino Faggiana <marino.faggiana@nextcloud.com>
@marinofaggiana
marinofaggiana merged commit 95b54c4 into master Aug 24, 2026
4 checks passed
@marinofaggiana
marinofaggiana deleted the mediaLimitCalc branch August 24, 2026 06:04
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Media verification SEARCH uses nresults=1000000 and exhausts server memory on large accounts

1 participant