llama/ggml: Hadamard weight-fold runtime (prism.hadamard GGUF contract, FWHT to 2048)

[bri-prism]

What

Runtime support for models whose weights were Hadamard-folded offline, plus the kernel coverage to run them at full speed:

• A prism.hadamard v1 GGUF metadata contract (block size, transform, axis, sign mode, folded weight names). The loader validates it, materializes one persistent rotation tensor per (block size, buffer type), and applies the activation-side transform immediately before every folded mul_mat / mul_mat_id through the existing FWHT matmul hint. Identity signs only for now; any unsupported contract value refuses to load rather than running wrong math.
• FWHT hint kernels extended from 512-wide to 1024/2048-wide blocks on CUDA and Metal. Backends without a matching kernel width fall back to a plain matmul against the real rotation matrix, so results stay correct everywhere.
• F16 inputs accepted by the FWHT hint on CPU, CUDA, Metal, and Vulkan.
• convert_hf_to_gguf picks up a hadamard_packing.json manifest from the model directory and writes the contract keys.
• Two fixes found while validating: a Metal crash when an F16 hint input hits a width without a dedicated FWHT kernel (now declined in supports_op so it falls back to CPU), and a loader ordering bug where rotation setup ran before mmap-backed buffers were assigned, which broke every folded model on Metal.

Why

Folding W' = W H^T into the weights offline and applying x' = Hx at runtime preserves the original computation exactly while flattening weight outliers ahead of low-bit quantization. The runtime half needs an explicit, validated metadata contract so a folded GGUF can never be silently loaded by a runtime that skips the activation transform.

Validation

• test-backend-ops -o MUL_MAT_HADAMARD: 14/14 on CUDA (SM90) and on Metal, against the CPU reference, including 2048-wide F32 and F16 cases.
• All four Vulkan fwht shader variants compile under glslc.
• End to end, the activation transform on a folded Q2_0_g128 model costs low single-digit percent decode at batch 1 and is prefill-neutral, measured with llama-bench on both an H200 and an M5.

[Copilot]

Pull request overview

Adds runtime support for Hadamard-folded GGUF weights and expands FWHT backend coverage.

Changes:

• Validates and emits the prism.hadamard GGUF contract.
• Injects activation transforms before folded matrix multiplications.
• Adds F16 and wider FWHT kernels with backend tests.

Reviewed changes

Copilot reviewed 21 out of 21 changed files in this pull request and generated 2 comments.

Show a summary per file

File	Description
tests/test-backend-ops.cpp	Expands FWHT cases through width 2048 and F16.
src/llama-model.h	Stores folded-weight metadata and rotations.
src/llama-model.cpp	Validates metadata and creates persistent rotations.
src/llama-model-loader.cpp	Enables string-array metadata loading.
src/llama-graph.h	Passes rotation mappings into graph construction.
src/llama-graph.cpp	Applies transforms before mapped matmuls.
src/llama-context.cpp	Supplies model rotations to graph parameters.
ggml/src/ggml-vulkan/vulkan-shaders/vulkan-shaders-gen.cpp	Generates F16 FWHT shaders.
ggml/src/ggml-vulkan/vulkan-shaders/fwht.comp	Supports F16 FWHT input.
ggml/src/ggml-vulkan/ggml-vulkan.cpp	Adds Vulkan F16 FWHT pipelines.
ggml/src/ggml-metal/ggml-metal.metal	Adds F16 and 1024/2048 Metal kernels.
ggml/src/ggml-metal/ggml-metal-ops.cpp	Dispatches typed Metal FWHT kernels.
ggml/src/ggml-metal/ggml-metal-device.m	Rejects unsupported F16 FWHT widths.
ggml/src/ggml-metal/ggml-metal-device.h	Defines FWHT pipeline and width interfaces.
ggml/src/ggml-metal/ggml-metal-device.cpp	Selects Metal pipelines by input type.
ggml/src/ggml-cuda/ggml-cuda.cu	Allows F16 Hadamard inputs on CUDA.
ggml/src/ggml-cuda/fwht.cu	Adds typed CUDA kernels through width 2048.
ggml/src/ggml-cpu/ops.cpp	Adds CPU F16 FWHT input conversion.
ggml/src/ggml-cpu/ggml-cpu.cpp	Advertises CPU F16 Hadamard support.
ggml/src/ggml-cpu/ggml-cpu.c	Converts F16 matmul inputs for CPU fallback.
conversion/base.py	Imports Hadamard manifests into GGUF metadata.

Suppressed comments (1)

ggml/src/ggml-vulkan/ggml-vulkan.cpp:5747

• The shared-memory path has the same unconditional F16 pipeline creation. On a Vulkan device where device->fp16 is false, this shader requires a feature/extension that was not enabled and can make backend initialization fail. Gate this variant on device->fp16 as well.

            ggml_vk_create_pipeline(device, device->pipeline_fwht_f16[idx], "fwht_shmem_f16", fwht_shmem_f16_len, fwht_shmem_f16_data, "main", 2, sizeof(vk_op_fwht_push_constants), {1, 1, 1}, { block_size, n }, 1);

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

[bri-prism]

Closing in favour of #121, which is the same work ported onto prism-v7.

This targets prism-v6, and the prism-v7 README now marks that branch as a stale mid-migration snapshot that should not be built from, so this cannot land as-is.

Nothing is lost. #121 carries all 18 commits from this branch and its follow-up (#120), rebased onto prism-v7, with the Metal FWHT kernels re-landed into the new ggml-metal/kernels layout since v7 moved the shaders out of the monolithic metal file. Verified there: 18/18 MUL_MAT_HADAMARD on CUDA (RTX 4090, 2/2 backends) and 17/17 on Metal (3/3 backends), and a folded checkpoint loads with 402 folded weights plus 1 inverse-lookup table and decodes at the same rate as on this branch.