Open Source

Testing hrx-loom: AMD's New Stack for Custom Inference Kernels

An experimental AMD stack named hrx-loom aims to generate faster LLM inference kernels than llama.cpp, but its real-world utility is limited to new hardware.

Alex Walsh 3 min read

An AMD developer's experimental project, hrx-loom, claims to generate custom LLM kernels that run faster on new AMD hardware than standard llama.cpp implementations. I took a look at the tool stack. While the idea of an agent automatically writing optimized kernels is compelling, the project is still raw and its performance benefits are tied to specific, unreleased hardware.

What is hrx-loom?

The project, called hrx-loom, is an end-to-end stack for building and serving custom models on AMD hardware, including CPUs, GPUs, and NPUs. The core idea is to let you generate optimized kernels without modifying the underlying inference engine. In a post announcing the project, the developer shared results on Reddit showing significant speedups for the top 51 Hugging Face models running on AMD's Strix Halo platform.

How does it work?

Unlike frameworks that provide a fixed set of pre-optimized kernels, hrx-loom uses an agent-based approach to generate them on the fly for your specific model architecture. You describe the model program, and the tool writes and optimizes the low-level code. This is a powerful concept because it avoids the cat-and-mouse game of waiting for framework maintainers to add support for new model architectures or hardware features. The catch is that this process lives on top of the existing AMD software ecosystem, so you're still dependent on the ROCm stack, which has its own set of challenges.

What's the catch?

The performance claims are impressive but narrow. The benchmarks were run on AMD's Strix Halo hardware, which isn't widely available. It's unclear how much of a speedup, if any, you would see on older Radeon GPUs. Being an experimental project from a single developer, expect sharp edges. The setup isn't a simple `pip install`. You'll likely be compiling components and troubleshooting dependencies with the AMD driver stack. This isn't a drop-in replacement for a production-ready framework.

Should you use it?

Probably not, unless you're specifically targeting Strix Halo and enjoy tinkering with experimental toolchains. The project is a valuable proof-of-concept for automated kernel generation. It shows a path forward where we don't have to manually port every new LLM architecture to every new piece of hardware. For now, most developers running inference on AMD hardware are better off sticking with the more mature implementations in llama.cpp or PyTorch. I'm bookmarking the GitHub repo, but I'm not swapping it into any of my projects yet.

FAQ

Is hrx-loom an official AMD product? No, hrx-loom is an experimental project released by an individual developer who works in AMD's AI Group. It is not an officially supported AMD product.

Will hrx-loom work on older AMD GPUs? The developer's benchmarks are specific to the new Strix Halo hardware. While the tool might run on older hardware, the impressive performance gains are not guaranteed and will likely be much lower.

Does hrx-loom replace the AMD ROCm stack? No, hrx-loom builds on top of the existing AMD software stack. You will still need the appropriate ROCm drivers and libraries installed to use it for GPU acceleration.

How does hrx-loom's approach differ from llama.cpp? llama.cpp relies on a set of hand-optimized kernels for various hardware backends. In contrast, hrx-loom uses an agent to automatically generate new, specialized kernels for a given model and target hardware, which can yield better performance on that specific configuration.