Running Laguna on AMD GPUs with Lemonade.

Laguna-S-2.1 support was merged into upstream llama.cpp at build b10087. Because Lemonade v11.5.0 ships with an older llama.cpp-ROCm binary (build b9752), loading the model fails out of the box. This post shows how to pin a newer backend build to run Laguna today, along with temporary workarounds until upstream package updates land.

What Lemonade Is

Lemonade is an open-source, cross-platform local LLM server providing an OpenAI-compatible HTTP API. While it supports multiple hardware vendors (NVIDIA, AMD, Apple Silicon, and CPU), this guide focuses on its llama.cpp-ROCm backend for AMD devices.

Architecturally, Lemonade uses a daemon (lemond) to handle model registry management and process execution (llama-server), while the lemonade CLI acts as a lightweight client. When a model load fails, detailed error logs reside in lemond, while the CLI simply returns HTTP 500.

Model Overview & Memory Requirements

Laguna-S-2.1 is a sparse Mixture-of-Experts (MoE) model. Key GGUF architectural parameters include:

There are currently no Q2 or Q3 quantizations available. System planning must account for 89.4 GiB of weights plus ~3 GiB of KV cache at a 32k context size.

Why a Newer Build Is Required

Stock Lemonade v11.5.0 bundles llama.cpp-ROCm build b9752. Attempting to load Laguna on a stock installation triggers an immediate failure:

llama_model_load: error loading model: unknown model architecture: 'laguna'

This error occurs before VRAM allocation because build b9752 lacks the architecture loader code introduced in upstream build b10087. To fix this, we must point Lemonade to build b10155 or newer.

Manual Enablement Steps

Step 1: Download & Extract Build b10155

Download a post-b10087 build directly from the lemonade-sdk/llama.cpp releases:

RELEASE=b10155
RELEASE_DIR=~/.local/opt/llamacpp/rocm
RELEASE_URL=https://github.com/lemonade-sdk/llama.cpp/releases/download/${RELEASE}/llama-${RELEASE}-bin-ubuntu-rocm-7.14-x64.tar.gz

mkdir -p ${RELEASE_DIR}
curl -sSL "${RELEASE_URL}" | tar -xz -C "${RELEASE_DIR}"

Step 2: Verify GPU Detection

Verify the binary can run on your system:

LS=${RELEASE_DIR}/llama-${RELEASE}/llama-server
"$LS" --version
"$LS" --list-devices

If --list-devices outputs Available devices: (none), it means system ROCm libraries (libamdhip64.so.7, etc.) are missing from the standalone path.

This is not an issue under Lemonade. When lemond spawns the binary, it automatically injects its bundled runtime (TheRock) into LD_LIBRARY_PATH. To test device detection using Lemonade's environment, run:

LD_LIBRARY_PATH="$(echo ~/.cache/lemonade/bin/therock/*/lib)" "$LS" --list-devices

The above assumes that you are using Lemonade as a user service and not a system one. The cache directory used will vary depending on your installation method and platform.

sudo dnf install rocblas hipblas hip-runtime-amd

Step 3: Pin the Absolute Path

Configure Lemonade to use the extracted binary:

lemonade config set llamacpp.rocm_bin="$LS"
lemonade config | grep llamacpp.rocm_bin

flatpak override --user \
    --filesystem=~/.local/opt/llamacpp \
    ai.lemonade_server.Lemonade

To revert to stock settings in the future:

lemonade config set llamacpp.rocm_bin=builtin

Deployment Configurations

Multi-GPU Tensor-Split (4x R9700)

For discrete multi-GPU pools connected via PCIe without XGMI interconnects, use pipeline (layer) parallelism (--split-mode layer). This avoids the high PCIe overhead associated with row-based tensor splitting.

lemonade load Laguna-S-2.1-GGUF-Q4_K_M \
  --llamacpp rocm \
  --llamacpp-device ROCm0,ROCm1,ROCm2,ROCm3 \
  --ctx-size 32768 \
  --llamacpp-args "--flash-attn on -ts 1,1,1,1 --split-mode layer"

Key Parameters:

• -ts 1,1,1,1: Distributes layers evenly across all 4 GPUs (~22.4 GiB VRAM per card).
• --split-mode layer: Minimizes inter-GPU communication over PCIe.
• --flash-attn on: Reduces KV cache memory overhead.

Strix Halo Unified Memory System

On a 128 GB Strix Halo system (e.g., Ryzen AI MAX+ PRO 395), system reserves and a 32 GiB BIOS UMA carveout can leave only ~94.05 GiB of kernel-visible memory. Because the 92.4 GiB working set (weights + KV cache) leaves negligible OS memory headroom, execution requires specific flags:

lemonade load Laguna-S-2.1-GGUF-Q4_K_M \
  --llamacpp rocm \
  --llamacpp-device ROCm0 \
  --ctx-size 32768 \
  --llamacpp-args "--load-mode mmap --cpu-moe --flash-attn on --cache-type-k q8_0 --cache-type-v q8_0"

Key Parameters:

• --load-mode mmap: Overrides Lemonade's injected --no-mmap flag. File-backed memory mapping allows the OS kernel to manage page eviction cleanly without swapping to disk.
• --cpu-moe: Keeps MoE expert weights in CPU page cache. On unified memory, offloading experts to GTT pins memory and starves the OS page cache, dropping performance from 2.6 tok/s down to 0.9 tok/s.
• --cache-type-k q8_0 --cache-type-v q8_0: Quantizes KV cache to save ~1.5 GiB of memory.

Additional Considerations

• Speculative Decoding (DFlash Draft): Draft decoding with laguna-s-2.1-DFlash-BF16.gguf is currently broken on build b10155 due to internal tensor layout mismatches (expected 76, got 69).
• vLLM-ROCm: Quantized 4-bit ROCm execution is currently unsupported in vLLM. Stick with the llama.cpp-ROCm backend for AMD hardware.
• Reasoning Tokens: Laguna outputs thinking chains to choices[0].message.reasoning_content. Ensure your clients set sufficient max_tokens values to avoid empty content fields during generation.