From 432473cd347bfd047d2ca158154296307d8c8122 Mon Sep 17 00:00:00 2001 From: Anthony Shoumikhin Date: Fri, 28 Aug 2026 22:30:26 -0700 Subject: [PATCH 1/2] Ship the TorchAO kernels on the Linux aarch64 wheel too The wheel builds these kernels for macOS arm64 and for nothing else, so a Linux aarch64 install has no `libexecutorch_kernels_torchao` even though it has exactly the hardware they target. Verified against the published nightly: the aarch64 row ships seven libraries and none of them is this one. The gate was one condition. `pybind.cmake` enables the option inside the Darwin branch under `CMAKE_SYSTEM_PROCESSOR STREQUAL "arm64"`, and Linux aarch64 reports `aarch64`, so it never matched. Nothing about these kernels is Apple specific: they are selected by `TORCHAO_BUILD_CPU_AARCH64` and reach their NEON paths through `TORCHAO_ENABLE_ARM_NEON_DOT`, both of which hold on any aarch64 host. The comment claiming otherwise is corrected here, along with two others that repeated it. x86 is deliberately left out. In torchao's `kernel_selector.h` every `return PackedWeightsFormat` is behind an ARM guard and there is no portable one, so both format selectors fall through to `throw std::runtime_error("No packed_weights_format was selected")`. Enabling it there would build and then throw on every op, which is worse than shipping nothing. Test plan: Built on real Linux aarch64 hardware, gcc 11.4, at the pinned TorchAO commit, with the same options the preset now sets: configure rc=0 libtorchao_kernels_aarch64.a built kernel_selector.h under gcc compiles, and that is the header the ExecuTorch ops include The prerequisites the option demands are already satisfied on this row: `EXECUTORCH_BUILD_XNNPACK` is on for the wheel and `check_required_options_on` forces `EXECUTORCH_BUILD_CPUINFO` and `EXECUTORCH_BUILD_PTHREADPOOL` with it. Checked the new condition matches only what it should: aarch64 ON arm64 ON x86_64 off amd64 off i686 off armv7l off so 32-bit ARM stays out. The macOS path is untouched, and `setup.py` already ships the library whenever the flag is set, so no packaging change is needed. Two GCC portability bugs exist upstream in TorchAO's own aarch64 tree, found while building its test suite here: `embedding_lut.h` uses `std::memcpy` without including ``, and `test_bitpacking.cpp` brace-initialises `int8x16_t`. Neither file is reached by `torchao_ops_executorch`, so neither affects this build, and both need fixing in the submodule rather than here. --- .ci/scripts/wheel/test_shared_libraries.py | 4 ++-- tools/cmake/executorch-wheel-config.cmake | 6 +++--- tools/cmake/preset/pybind.cmake | 14 +++++++++----- 3 files changed, 14 insertions(+), 10 deletions(-) diff --git a/.ci/scripts/wheel/test_shared_libraries.py b/.ci/scripts/wheel/test_shared_libraries.py index 2ed6464105b..87544d30c1e 100644 --- a/.ci/scripts/wheel/test_shared_libraries.py +++ b/.ci/scripts/wheel/test_shared_libraries.py @@ -162,8 +162,8 @@ "executorch::backends::coreml::CoreMLBackendDelegate::get_registered_delegate()", ) -# A representative symbol from the TorchAO kernels. These are Apple Silicon only, so -# most wheels ship no such library and the row below is not required. +# A representative symbol from the TorchAO kernels. These need aarch64, so the x86 and Windows +# wheels ship no such library and the row below is not required. _TORCHAO_KERNEL_SYMBOLS = ("torchao::quantization::get_qvals_range",) # A representative symbol from the profiler. A second definer means two event diff --git a/tools/cmake/executorch-wheel-config.cmake b/tools/cmake/executorch-wheel-config.cmake index 1b7ae3eb5a8..7e9229fe816 100644 --- a/tools/cmake/executorch-wheel-config.cmake +++ b/tools/cmake/executorch-wheel-config.cmake @@ -75,7 +75,7 @@ # executorch::backend_mlx The MLX delegate. macOS on Apple Silicon only. # Its Metal kernel archive is published as # MLX_METALLIB_PATH, see below. -# executorch::kernels_torchao The TorchAO kernels. macOS on Apple Silicon only. +# executorch::kernels_torchao The TorchAO kernels. aarch64 rows only. # executorch::backend_cuda The CUDA delegate. Linux only. # executorch::extension_cuda The CUDA stream extension. Linux only. # executorch::backend_openvino The OpenVINO delegate. Linux only. Opens the @@ -594,8 +594,8 @@ _executorch_define_component(threadpool executorch_threadpool) # checks, so it has to be defined here or a consumer following the documentation # gets a bare name that CMake hands to the linker as a literal flag. _executorch_define_component(kernels_optimized executorch_kernels_optimized) -# The TorchAO kernels, present only in a wheel built for Apple Silicon, which is -# the only architecture they build for. +# The TorchAO kernels, present only in a wheel built for aarch64, which is the only +# architecture they build for. _executorch_define_component(kernels_torchao executorch_kernels_torchao) # The quantized kernels, optional in the same way: a wheel built without them # simply has no such library and the component is not defined. diff --git a/tools/cmake/preset/pybind.cmake b/tools/cmake/preset/pybind.cmake index 2021df8b349..de1503204ca 100644 --- a/tools/cmake/preset/pybind.cmake +++ b/tools/cmake/preset/pybind.cmake @@ -59,11 +59,9 @@ if(CMAKE_SYSTEM_NAME STREQUAL "Darwin") set_overridable_option(EXECUTORCH_BUILD_EXTENSION_TRAINING ON) set_overridable_option(EXECUTORCH_BUILD_EXTENSION_LLM_RUNNER ON) set_overridable_option(EXECUTORCH_BUILD_EXTENSION_LLM ON) - # Both of these are Apple Silicon only. The TorchAO kernels build only for - # aarch64, which is what TORCHAO_BUILD_CPU_AARCH64 selects; the Apple - # framework build already ships them and this brings the wheel in line. MLX - # additionally needs the Metal compiler (xcrun -sdk macosx metal), which comes - # with Xcode and not with the Command Line Tools. + # MLX needs the Metal compiler (xcrun -sdk macosx metal), which comes with Xcode and not with + # the Command Line Tools, so it is probed rather than assumed. The TorchAO kernels are enabled + # on every aarch64 row, here and under Linux below, since aarch64 is what they require. if(CMAKE_SYSTEM_PROCESSOR STREQUAL "arm64") set_overridable_option(EXECUTORCH_BUILD_KERNELS_TORCHAO ON) execute_process( @@ -89,6 +87,12 @@ elseif(CMAKE_SYSTEM_NAME STREQUAL "Linux") set_overridable_option(EXECUTORCH_BUILD_EXTENSION_TRAINING ON) set_overridable_option(EXECUTORCH_BUILD_EXTENSION_LLM_RUNNER ON) set_overridable_option(EXECUTORCH_BUILD_EXTENSION_LLM ON) + # The same aarch64 kernels the macOS arm64 wheel gets above. The hardware is what these + # need, not Apple: they are selected by TORCHAO_BUILD_CPU_AARCH64 and reach their NEON + # paths through TORCHAO_ENABLE_ARM_NEON_DOT, both of which hold here too. + if(CMAKE_SYSTEM_PROCESSOR MATCHES "^(aarch64|arm64)$") + set_overridable_option(EXECUTORCH_BUILD_KERNELS_TORCHAO ON) + endif() if(CMAKE_SYSTEM_PROCESSOR MATCHES "^(x86_64|amd64|i.86)$") # Auto-enable QNN on Linux x86 when the SDK is available. - QNN_SDK_ROOT set # explicitly → always enable - GitHub Actions CI → skip (avoids flaky 1.3GB From c4a82b97340f09e23ac4de2691fe6f129205ca1e Mon Sep 17 00:00:00 2001 From: Anthony Shoumikhin Date: Fri, 28 Aug 2026 22:55:40 -0700 Subject: [PATCH 2/2] Wrap the new comments the way cmakelang wants lintrunner failed on formatting: the repo wraps cmake comments at the width cmakelang 0.6.13 enforces, and I wrote them wider. Reflowed the three comment blocks to match, with no change to any condition. Test plan: Ran the same formatter CI uses, cmakelang 0.6.13: tools/cmake/preset/pybind.cmake format OK tools/cmake/executorch-wheel-config.cmake format OK and re-parsed the preset as Linux/aarch64 to confirm the reflow left the logic alone. --- tools/cmake/executorch-wheel-config.cmake | 4 ++-- tools/cmake/preset/pybind.cmake | 14 ++++++++------ 2 files changed, 10 insertions(+), 8 deletions(-) diff --git a/tools/cmake/executorch-wheel-config.cmake b/tools/cmake/executorch-wheel-config.cmake index 7e9229fe816..c135bf897fd 100644 --- a/tools/cmake/executorch-wheel-config.cmake +++ b/tools/cmake/executorch-wheel-config.cmake @@ -594,8 +594,8 @@ _executorch_define_component(threadpool executorch_threadpool) # checks, so it has to be defined here or a consumer following the documentation # gets a bare name that CMake hands to the linker as a literal flag. _executorch_define_component(kernels_optimized executorch_kernels_optimized) -# The TorchAO kernels, present only in a wheel built for aarch64, which is the only -# architecture they build for. +# The TorchAO kernels, present only in a wheel built for aarch64, which is the +# only architecture they build for. _executorch_define_component(kernels_torchao executorch_kernels_torchao) # The quantized kernels, optional in the same way: a wheel built without them # simply has no such library and the component is not defined. diff --git a/tools/cmake/preset/pybind.cmake b/tools/cmake/preset/pybind.cmake index de1503204ca..6e9cb4714ed 100644 --- a/tools/cmake/preset/pybind.cmake +++ b/tools/cmake/preset/pybind.cmake @@ -59,9 +59,10 @@ if(CMAKE_SYSTEM_NAME STREQUAL "Darwin") set_overridable_option(EXECUTORCH_BUILD_EXTENSION_TRAINING ON) set_overridable_option(EXECUTORCH_BUILD_EXTENSION_LLM_RUNNER ON) set_overridable_option(EXECUTORCH_BUILD_EXTENSION_LLM ON) - # MLX needs the Metal compiler (xcrun -sdk macosx metal), which comes with Xcode and not with - # the Command Line Tools, so it is probed rather than assumed. The TorchAO kernels are enabled - # on every aarch64 row, here and under Linux below, since aarch64 is what they require. + # MLX needs the Metal compiler (xcrun -sdk macosx metal), which comes with + # Xcode and not with the Command Line Tools, so it is probed rather than + # assumed. The TorchAO kernels are enabled on every aarch64 row, here and + # under Linux below, since aarch64 is what they require. if(CMAKE_SYSTEM_PROCESSOR STREQUAL "arm64") set_overridable_option(EXECUTORCH_BUILD_KERNELS_TORCHAO ON) execute_process( @@ -87,9 +88,10 @@ elseif(CMAKE_SYSTEM_NAME STREQUAL "Linux") set_overridable_option(EXECUTORCH_BUILD_EXTENSION_TRAINING ON) set_overridable_option(EXECUTORCH_BUILD_EXTENSION_LLM_RUNNER ON) set_overridable_option(EXECUTORCH_BUILD_EXTENSION_LLM ON) - # The same aarch64 kernels the macOS arm64 wheel gets above. The hardware is what these - # need, not Apple: they are selected by TORCHAO_BUILD_CPU_AARCH64 and reach their NEON - # paths through TORCHAO_ENABLE_ARM_NEON_DOT, both of which hold here too. + # The same aarch64 kernels the macOS arm64 wheel gets above. The hardware is + # what these need, not Apple: they are selected by TORCHAO_BUILD_CPU_AARCH64 + # and reach their NEON paths through TORCHAO_ENABLE_ARM_NEON_DOT, both of + # which hold here too. if(CMAKE_SYSTEM_PROCESSOR MATCHES "^(aarch64|arm64)$") set_overridable_option(EXECUTORCH_BUILD_KERNELS_TORCHAO ON) endif()