PyC

NVIDIA CUDA GEMM Kernel Workbench

PyC gives kernel engineers one controlled path from a CUDA candidate to a compiled runtime dispatch target:

flowchart LR
    A["Create CUDA candidate"] --> B["Validate numerical output"]
    B --> C["Benchmark and profile"]
    C --> D["Promote with evidence"]
    D --> E["Generate CUDA registry and CMake sources"]
    E --> F["Compiled PyC runtime dispatch"]

Loading

The first-class product is the Python command group:

pyc hpc create | validate | bench | profile | promote

PyC keeps the existing C compiler/runtime and CUTLASS integration underneath that workflow. It is not a scheduler, remote file-staging tool, or runtime JIT system. Run the same commands on the CUDA host that owns the target GPU.

What You Get

• A standard CUDA GEMM candidate workspace with source, harness, and metadata.
• Deterministic correctness evidence against a reference implementation.
• Benchmark and profiler records that capture commands, logs, toolchain, GPU, shapes, and timing metrics.
• A promotion gate: correctness must pass and a candidate must improve the incumbent median latency by at least 2% for the same selector.
• Generated CUDA registration and CMake source lists consumed by the native runtime build.
• The legacy kernel lab preserved for pre-existing Ada and Hopper experiments.

The v1 creation template is a FP32 SIMT GEMM. Candidate metadata supports FP16 and BF16 target contracts; existing FP16/BF16 prototypes can be imported and run through the legacy lab until dedicated templates are added.

Requirements

Local, metadata-only work:

• Python 3.10 or later.
• uv is recommended for running the package without a manual environment.

CUDA validation and benchmarking:

• Linux with an NVIDIA GPU.
• A CUDA toolkit with nvcc that supports the target architecture.
• NVIDIA driver and nvidia-smi available on the GPU host.

Native runtime integration:

• CMake 3.10 or later and a C11 compiler.
• CUDA C++ compiler when building CUDA kernels.
• CUTLASS headers only for the optional full CUTLASS kernel set.

macOS can run the CLI, inspect metadata, perform dry runs, and build the CPU fallback targets. It cannot validate or benchmark CUDA kernels locally.

Quick Start

Start by checking the workspace and GPU toolchain:

uv run pyc hpc doctor

Create a new Ada FP32 candidate:

uv run pyc hpc create gemm \
  --name tiled_fp32_v1 \
  --arch sm89 \
  --shape-family square \
  --tile-m 16 --tile-n 16 --tile-k 16

On the GPU host, validate and benchmark it. Each command prints the evidence path it writes under kernels/results/.

validation_result=$(uv run pyc hpc validate --name tiled_fp32_v1)
benchmark_result=$(uv run pyc hpc bench --name tiled_fp32_v1)

Review the generated JSON, then establish the first baseline for this selector:

uv run pyc hpc promote \
  --name tiled_fp32_v1 \
  --validation-result "$validation_result" \
  --benchmark-result "$benchmark_result" \
  --bootstrap

For later candidates on the same dtype, layout, and shape family, omit --bootstrap. Promotion fails unless the candidate is at least 2% faster than the current promoted record.

Kernel Workflow

1. Create

pyc hpc create gemm --name candidate_name [options]

Creation writes:

kernels/prototypes/candidate_name/
  kernel.cu       CUDA kernel and runtime dispatch function
  harness.cu      correctness and timing executable
  kernel.json     candidate contract and runtime metadata

The candidate contract includes:

Field	Meaning
name	Stable lower-snake-case candidate ID
dtype	fp32, fp16, or bf16
layout	GEMM input layout, such as nn
arch	NVIDIA target, currently sm89 or sm90
shape_family	small, square, large_square, tall_skinny, or wide_skinny
symbol	CUDA dispatch symbol compiled into the runtime
runtime	priority, occupancy estimate, tensor-core eligibility, shared memory, and register pressure

The runtime selector key is:

gemm/<dtype>/<layout>/<shape-family>

2. Validate

pyc hpc validate --name candidate_name --nvcc /path/to/nvcc

Validation compiles the candidate and runs every configured shape with a deterministic input seed. It records per-shape absolute and relative error, compiler output, harness output, host GPU details, and the tolerance used.

Default tolerance contracts:

Dtype	Maximum absolute error	Maximum relative error
fp32	1e-4	1e-4
fp16	5e-2	5e-2
bf16	8e-2	8e-2

3. Benchmark And Profile

pyc hpc bench --name candidate_name --warmup 10 --repeats 50
pyc hpc profile --name candidate_name \
  --command "ncu --set full {binary}"

The profiler command is explicit by design. It can use these placeholders: name, source, candidate_dir, and binary. PyC stores stdout, stderr, return code, and the exact expanded command with the profile record.

4. Promote

pyc hpc promote \
  --name candidate_name \
  --validation-result kernels/results/candidate_name-validation-<timestamp>.json \
  --benchmark-result kernels/results/candidate_name-benchmark-<timestamp>.json

Promotion requires all of the following:

1. Completed validation for the same candidate with zero failed shapes.
2. Completed benchmark evidence from an available target GPU.
3. A positive measured median latency.
4. At least 2% median-latency improvement over the incumbent selector.
5. A standard, runtime-compatible workbench candidate.

The first reviewed candidate for a selector uses --bootstrap. This is deliberate: it makes the baseline choice visible in version control.

Promotion Output And Runtime Dispatch

Promotion updates the tracked registry:

kernels/registry/kernels.json

It also regenerates:

src/compiler/cutlass_kernels/registry/generated_registry.cu
src/compiler/cutlass_kernels/registry/generated_kernels.cmake

The generated C++ unit registers each promoted candidate with the existing kernel registry. The generated CMake list compiles its CUDA source into pyc_cutlass_kernels. At runtime, the CUDA backend selects the matching FP32 selector key before falling back to the existing promoted matmul path, cuBLASLt, cuBLAS, or CPU execution.

Promotion changes tracked source and requires a normal native rebuild. PyC does not JIT-compile or dynamically load promoted candidates in v1.

Workspace And Evidence

Path	Purpose	Version control
kernels/templates/gemm/	Maintained source and harness templates	tracked
kernels/prototypes//	Candidate source and metadata	tracked
kernels/results/	Validation, benchmark, and profile evidence	ignored
kernels/.build/	Local nvcc executables	ignored
kernels/registry/kernels.json	Promoted runtime records	tracked
kernels/lab/	Legacy prototype/task workflow	tracked

To inspect candidates:

pyc hpc list
pyc hpc show --name candidate_name
pyc hpc compare left-benchmark.json right-benchmark.json

Dry-run validation and benchmarking are available without CUDA:

pyc hpc validate --name candidate_name --dry-run
pyc hpc bench --name candidate_name --dry-run

They write planned evidence only and cannot be promoted.

Legacy Kernel Lab

Existing kernel-lab manifests, task records, and Ada/Hopper prototypes remain supported:

python3 kernels/lab/kernel_lab.py doctor
python3 kernels/lab/kernel_lab.py task-create ada-sm89-gemm \
  --task-kind gemm --candidate-tag ada

To copy an existing manifest-backed prototype into the new workspace:

pyc hpc import-legacy --legacy-name ada_gemm_k64_warp32_async \
  --name ada_async_import --dtype fp32 --arch sm89

Imported candidates are intentionally not runtime-promotable until they are converted to the standard source/harness/dispatch contract.

Building The Native Runtime

Build the portable core and compiler-next runtime:

cmake -S . -B build \
  -D PYC_BUILD_COMPILER_NEXT=ON \
  -D PYC_BUILD_COMPILER_NEXT_TESTS=ON
cmake --build build --parallel
ctest --test-dir build --output-on-failure

Build CUDA support on the GPU host:

cmake -S . -B build-cuda \
  -D PYC_BUILD_COMPILER_NEXT=ON \
  -D PYC_ENABLE_CUDA_RUNTIME=ON \
  -D PYC_CUTLASS_PATH=/path/to/cutlass
cmake --build build-cuda --parallel

Without CUTLASS headers, PyC still builds the minimal promoted CUDA registry when the CUDA toolkit is available. CUDA-free hosts build the fallback-only runtime path.

Repository Map

python/pyc/hpc/                  Canonical workbench CLI and compatibility layer
kernels/templates/gemm/          Tool-owned CUDA candidate templates
kernels/prototypes/              Candidate and legacy CUDA sources
kernels/registry/                Promoted-kernel source of truth
src/compiler/cutlass_kernels/    CUDA dispatch and generated registry integration
src/compiler/runtime/            Native selector and CUDA backend
include/pyc/                     Public C compiler/runtime headers
benchmark/                       Broader benchmark adapters and workload suites
tests/                           Python and native regression tests
docs/compiler-next/              Runtime, CUDA, and GPU testing references

Verification

Run the workbench and legacy compatibility tests:

uv run --with pytest python -m pytest \
  tests/test_hpc_workbench.py tests/test_kernel_lab_task_builder.py -q
uv run --with ruff ruff check python/pyc/cli.py python/pyc/hpc tests/test_hpc_workbench.py

The workbench tests cover candidate creation, metadata contracts, dry-run evidence, promotion generation, the 2% speed gate, and legacy-lab imports. CUDA execution must be verified on a Linux GPU host.

Status

• The CUDA GEMM workbench is the active product surface.
• New creation is FP32 SIMT GEMM; FP16/BF16 template authoring is the next kernel-family expansion.
• The native compiler/runtime, CUDA fast path, and legacy lab remain available.
• Remote scheduling, SSH staging, multi-vendor backends, and runtime JIT are intentionally outside the v1 contract.

Documentation And License

Start with docs/README.md. The most relevant references are:

• docs/compiler-next/kernel-lab.md for the legacy task builder.
• docs/compiler-next/gpu-testing-playbook.md for remote GPU procedure.
• docs/compiler-next/cuda-gemm-fast-path.md for the native CUDA backend.
• docs/compiler-next/kernel-optimization-playbook.md for GEMM optimization priorities and promotion context.

PyC is licensed under the Apache License 2.0. See LICENSE.