[SPARK-58389][SQL][FOLLOWUP] Pin DSv2 table instances by state options during analysis

[yyanyy]

What changes were proposed in this pull request?

SPARK-58389 changed DSv2 relation caching to include all read options. This is required
to preserve each reference's complete option bag, but it also means references that differ
only in scan-specific options can independently call loadTable and receive different
concrete table versions within one query.

This PR separates relation reuse from table-state pinning:

• The existing relationCache remains keyed by all read options and reuses a finalized relation
  only when the complete option bags match.
• A new query-scoped tableCache is keyed by catalog, identifier, time travel, and table-state
  options. References with the same table-state key reuse one concrete Table, while retaining
  their own complete options in their relations.
• References with different table-state options do not share a table pin.
• sharedRelationCache and CacheManager reuse continue to require complete option matching.

This PR adds the evolving SupportsTableStateOptions catalog capability so a catalog can declare
which raw options may affect the table state selected by loadTable, such as a branch, tag,
snapshot, or version. Catalogs that do not implement the capability are handled conservatively:
all raw options are considered table-state-affecting.

While applying the same table-pinning model to cacheable V2TableReference resolution, this PR
also fixes two existing gaps in getOrLoadRelation:

• It previously called loadTable(identifier) without passing the options captured in the table
  reference. A table-cache miss now uses the options-aware catalog API with the reference's
  complete option bag.
• It previously did not consult sharedRelationCache. Temporary-view re-resolution now consults
  sharedRelationCache while establishing the initial table pin, allowing it to preserve a
  Table already pinned through CacheManager.

Transaction references still use the Table loaded through the transaction catalog and do not
consult sharedRelationCache. Write targets remain non-cacheable and bypass the query-scoped read
caches.

The resulting resolution flow is:

1. Check relationCache using the full-option relation key.
2. On a miss, check tableCache using the table-state key.
3. On a tableCache hit, construct a relation from the pinned Table and the current reference's
   complete options. Do not call loadTable or consult sharedRelationCache.
4. On a tableCache miss, load the current Table through the applicable options-aware path.
5. Where sharedRelationCache lookup applies, reuse its relation only when table identity and all
   options match. The shared cached Table establishes the initial pin on a match; otherwise, the
   newly loaded Table establishes it.
6. Store the pinned Table in tableCache and the finalized relation in relationCache.

Execution-time table refresh uses the same table-state option projection to preserve this
first-resolution-wins behavior.

Why are the changes needed?

A catalog may accept both table-state options and scan-specific options. Using the complete option
bag for relation reuse is necessary, but using it as the only level of caching can cause references
in the same table-state domain to load different concrete table versions during one query.

The new tableCache pins one concrete Table per state key without weakening full-option matching
for finalized relations, sharedRelationCache, or CacheManager.

Forwarding options from V2TableReference is also necessary because those options may select the
table state being reloaded. Temporary views additionally need the sharedRelationCache bridge to
preserve a CacheManager-pinned Table.

Different state domains, including different parsed time-travel specifications, continue to
resolve and pin independently.

Does this PR introduce any user-facing change?

Yes, for catalog implementors only.

This adds the evolving SupportsTableStateOptions catalog capability. There is no new SQL syntax
or configuration, and catalogs that do not implement it retain the conservative behavior where all
raw options are considered table-state-affecting.

The table-consistency fix otherwise addresses behavior introduced on the unreleased master branch
by SPARK-58389.

How was this patch tested?

Added regression coverage for table-state projection and pinning, conservative fallback,
sharedRelationCache and CacheManager matching, nested analysis, execution refresh, and
temporary-view, transaction, and write-target V2TableReference behavior.

Was this patch authored or co-authored using generative AI tooling?

Generated-by: OpenAI Codex

[yyanyy]

The other case here for now is TransactionContext that shouldn't use shared relation cache to avoid replacing the table earlier than the appropriate transaction check to decide if a cache reuse is safe (txn.registerScans?)

[uros-b]

@szehon-ho Could you PTAL at these changes?