Achieving 110 tok/s with RTX 4070 and the Right Setup on Qwen LM

I've been on a bit of a journey optimizing performance with large language models, particularly with Qwen3.6 35B. My setup includes an NVIDIA RTX 4070 Super 12GB, AMD Ryzen 7 9700X, and 48GB of DDR5-6000 RAM running CachyOS. Like many, I initially relied on llama.cpp for MTP support, but recent updates caused a noticeable drop in performance.

That's when I stumbled upon ik_llama.cpp. It seems tailored for better CPU offloading, which piqued my interest. After setting it up, the performance was drastically improved. I hit around 110 tokens per second with my MTP benchmarks, which is a significant boost over my results with llama.cpp after their update.

Here's a little overview of how it performed:

• Python Code Inference: 79.8 tok/s
• C++ Code Inference: 89.1 tok/s
• Summarization Task: 95.0 tok/s
• Factual QA: 97.0 tok/s

For context, I'm using byteshape's recent quantization of Qwen3.6-35B-A3B-IQ4_XS-4.19bpw. It's remarkably compact and compares well in accuracy against other options I've tried.

Here’s a snippet of my launch command for reproducibility:

llama-server \
    -m Qwen3.6-35B-A3B-IQ4_XS-4.19bpw.gguf \
    --fit on \
    --fit-target 512 \
    --ctx-size 131072 \
    --cache-type-k q8_0 \
    --cache-type-v q8_0 \
    --threads 8 \
    --temp 0.0

I'm curious if others have tried similar setups or have any recommendations on further optimizations. Also, if you're on the lookout for OS options, I can't recommend CachyOS enough for this kind of performance tuning.