MDEV-39800: Assertion '!(mode & 2048U) || (mode & 512U) || is_supremum' failed

[mariadb-TafzeelShams]

• The Jira issue number for this PR is: MDEV-39800

Description

ISSUE:

Lock bypassing optimization allows an X-lock request to skip
waiting locks when the requesting transaction already holds an
S-lock on the same record. This optimization is designed for
regular B-tree record locks that use heap-number-based conflict
detection.

Spatial index predicate locks use different semantics. They
perform MBR (Minimum Bounding Rectangle) overlap checks for
conflict detection and must not participate in bypass
optimization.

The assertion failure occurred because predicate insert
intention locks is not considered inside lock bypass code,
which assumed all insert intention locks must be either gap
locks or on the supremum record (MDEV-34877).

ut_ad(!(insert_intention) || (gap) || is_supremum)

Additionally, predicate locks could incorrectly enable
bypass_mode because the existing checks did not explicitly
exclude LOCK_PREDICATE locks.

FIX:

• lock_t::is_predicate(): Add a helper to identify spatial
  index locks.

• lock_t::can_be_bypassed(): Return false for predicate
  locks.

• lock_rec_has_to_wait_in_queue(): Update the assertion to
  allow predicate insert intention locks. Also add a
  !is_predicate() check to bypass_mode calculation.

• lock_rec_queue_validate_bypass(): Add an early return to
  skip bypass validation for predicate locks.

Release Notes

How can this PR be tested?

Added MTR test case incurrently disabled innodb_gis.rtree_concurrent_srch

Basing the PR against the correct MariaDB version

• This is a new feature or a refactoring, and the PR is based against the main branch.
• This is a bug fix, and the PR is based against the earliest maintained branch in which the bug can be reproduced.

PR quality check

• I checked the CODING_STANDARDS.md file and my PR conforms to this where appropriate.
• For any trivial modifications to the PR, I am ok with the reviewer making the changes themselves.

[gemini-code-assist]

Code Review

This pull request addresses MDEV-39800 by ensuring that predicate locks do not participate in the lock bypass mechanism in InnoDB. It introduces an is_predicate() helper method to ib_lock_t and updates lock_rec_queue_validate_bypass and lock_rec_has_to_wait_in_queue to exclude predicate locks from bypass logic. Additionally, a concurrent search test case has been added to verify the fix. There are no review comments, so no further feedback is provided.

Important

The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.