place_sheet parallel rebuild: face/edge cone orientations unreconciled across the point SF — parallel FE on placed meshes assembles a wrong operator

[lmoresi]

Split out of the PR #518 review (comments there, 2026-08-10) so it cannot be lost: the composed acceptance found a parallel-only defect in the gather-first rebuild that blocks FE work on placed meshes at np>1. It survives #519 unchanged (measured on development at eb8d2d3), and #519's thin-volume embedding shares the same rebuild skeleton, so its parallel FE path inherits it.

Symptom

Plain Stokes — no fault machinery — on a placed-sheet mesh:

configuration	time
base box, serial	22 s
base box, np=2	13 s
placed mesh, serial	22 s
placed mesh, np=2	3594 s (converges — wrong, not singular)
placed mesh, np=2, after a FRESH DMPlexDistribute	2212 s

Eliminated by measurement

Element quality (radius-ratio distribution matches base); cell handedness (the #518 orientation census gate passes); duplicated/missing points (global owned v/e/f/c identical serial vs np=2: 1138/6444/9881/4574); vertex SF corruption (coordinate drift 0.0 exactly); the partition (fresh canonical redistribute does not cure it); the solver configuration (base box np=2 is fast); the fault contact machinery (plain Stokes grinds).

The true signal

DMPlexCheckFaces FAILS on the rebuilt mesh in parallel — np=2: both ranks (measured on development eb8d2d3); np=4: e.g. rank 3 with face closure: 1462 1456 1418 vs cell face: 1456 1462 1418 — same vertex set, swapped order: a cone-orientation mismatch. The plain distributed base box passes CheckFaces cleanly, so this is not a checker false-positive (unlike check_geometry, which does false-positive on healthy distributed meshes on this stack — see the gate scoping in 8ff3659).

Mechanism: the collective uninterpolate -> DMPlexInterpolate rebuild interpolates each rank's chart locally, and nothing afterwards reconciles leaf faces'/edges' cone order against their roots. Inconsistent cone orientations across the SF disorder the P2 edge/face DOF numbering in cross-seam assembly: the operator is assembled wrong. Serial cannot see it (no seam); redistribution faithfully migrates the bad cones.

Reproducer (np=2, ~2 min to the CheckFaces verdict)

import numpy as np
import underworld3 as uw
from underworld3.utilities.place_surface import place_sheet
from petsc4py import PETSc

base = uw.meshing.UnstructuredSimplexBox(
    minCoords=(0, 0, 0), maxCoords=(1, 1, 1), cellSize=0.11,
    regular=False, qdegree=2)
u = np.array([1.0, 0.0, 0.25]); u /= np.linalg.norm(u)
v = np.array([0.0, 1.0, 0.0]); s = np.linspace(-0.2, 0.2, 5)
pts = np.array([np.array([0.5, 0.5, 0.5]) + a*u + b*v
                for a in s for b in s])
tris, n = [], 5
rim = lambda k: (k // n in (0, n-1)) or (k % n in (0, n-1))
for i in range(n-1):
    for j in range(n-1):
        a, b = i*n+j, i*n+j+1
        c, d = (i+1)*n+j, (i+1)*n+j+1
        t1, t2 = (a, b, d), (a, d, c)
        if all(map(rim, t1)) or all(map(rim, t2)):
            t1, t2 = (a, b, c), (b, d, c)   # no all-rim triangles
        tris += [t1, t2]
bounds = base._boundaries_with("Rupture")
placed, info = place_sheet(base.dm, pts, np.array(tris, dtype=np.int64),
                           label="Rupture",
                           label_value=bounds["Rupture"].value)
chk = placed.clone(); chk.setOptionsPrefix("cf_")
o = PETSc.Options(); o["cf_dm_plex_check_faces"] = ""
ok = 1
try:
    chk.setFromOptions()
except PETSc.Error:
    ok = 0
del o["cf_dm_plex_check_faces"]
oks = uw.mpi.comm.allgather(ok)
uw.pprint("CheckFaces: " + ("OK" if all(oks) else f"FAIL {oks}"))
# then wrap in Mesh and run plain Stokes to see the grind

Acceptance

1. DMPlexCheckFaces green on the rebuilt mesh at np=2..5 (options route; it is a valid oracle here — the base mesh passes it).
2. Plain Stokes on a placed mesh at np=2 within ~2x of the base box (13 s on the review machine).
3. Same two checks for a place_thin_volume embedding (shared skeleton).

Serial capability is unaffected and fully validated (place -> split -> contact converges in 18 s, leak 8e-18). This issue gates parallel FE on placed/thin-volume meshes only.

Underworld development team with AI support from Claude Code

[lmoresi]

Fixed by #521 (merged as ef69815 on development). All three acceptance criteria met, plus a second, older defect found and fixed en route:

1. DMPlexCheckFaces green at np=2..5 — and promoted to an always-on internal gate in both place_sheet and place_thin_volume.
2. Plain Stokes on the placed mesh at np=2: 12.1 s vs base box 7.6 s (bound ~2x; the defect measured 3594 s).
3. Thin volume: same gates, P2 quadratic reproduced through the zone to 9e-16 at np=1,2,4.

The second defect: the #518 review's cell-handedness fix (8ff3659) had been orphaned on a review side branch by the squash-merge — every placed mesh since #517 had mixed handedness and even SERIAL FE assembled a wrong operator (linear solution off by 27 under exact LU), invisible to timing-only checks. Recovered verbatim in #521; P2 quadratic-exactness oracles added to test_0854/test_0855 so this class of defect can never again pass the suites.

Underworld development team with AI support from Claude Code