Releasing geniex-rs: Safe Rust Bindings for Qualcomm GenieX

⚠️ Community Notice: geniex-rs is an open-source, community-maintained Rust binding for Qualcomm’s GenieX C API (geniex.h). It is not an officially maintained product of Qualcomm Technologies, Inc. For official C/C++ runtime releases, visit qualcomm/GenieX on GitHub.

The “AI PC” landscape is shifting rapidly from cloud-dependent web wrappers to high-efficiency, on-device silicon execution. With the introduction of Windows on ARM and Qualcomm’s Snapdragon X architecture (Snapdragon X Plus and X Elite), modern laptops possess dedicated Hexagon Neural Processing Units (NPUs) and Adreno GPUs capable of running frontier Large Language Models (LLMs) and Vision-Language Models (VLMs) locally with minimal power consumption.

Today, I am excited to announce the release of geniex-rs, available on crates.io.

What started as an early R&D exploration (formerly Aura SDK) has matured into a standalone, community-maintained Rust binding for Qualcomm’s GenieX C API. geniex-rs provides safe, idiomatic, high-level Rust wrappers around geniex.h, allowing developers to load almost any GGUF model from Hugging Face - or pre-compiled binaries from the Qualcomm AI Hub and execute them locally in Rust across the Hexagon NPU, Adreno GPU, or CPU in just a few lines of code.

The Science of NPU Acceleration: Memory Bandwidth & Quantization

To understand why NPU acceleration is game-changing, we have to look at the mathematics of Large Language Models. LLM inference is fundamentally auto-regressive: to generate a single new token, the model must read its entire weight matrix from RAM, process it, and output the token. This makes LLM inference a memory-bandwidth bound task, rather than a compute-bound one.

Qualcomm’s Hexagon NPU solves this using two main engineering principles:

1. Low-Precision Quantization (INT4/INT8): By converting standard 32-bit floating-point weights (FP32) into 4-bit or 8-bit integers (INT4/INT8), the model size is reduced by up to 8x. A 7B parameter model shrinks from 28GB to just 3.5GB. This massive reduction allows the model to fit inside high-speed cache and drastically decreases the data transferred from LPDDR5x RAM, immediately accelerating tokens per second.
2. Dedicated Tensor Acceleration (HTA): Unlike CPUs (sequential execution) or GPUs (optimized for massive parallel graphics shading), the Hexagon NPU contains specialized hardware pipelines designed specifically for high-speed dot-product and matrix multiplication operations (the mathematical foundation of the Self-Attention mechanism). It handles low-precision integer arithmetic natively at the hardware layer with extreme power efficiency (often consuming less than 5 Watts).

geniex-rs unlocks this low-level hardware power directly for Rust developers, bridging the raw C engine to safe, compile-time checked code.

---

1. Architecture: Bringing GenieX to the Rust Ecosystem

GenieX is an on-device Gen AI inference runtime engineered by Qualcomm. Beneath the hood, GenieX dispatches execution either to the qairt runtime (Qualcomm AI Engine Direct on the Hexagon NPU) or to the llama_cpp runtime (GGML kernels over CPU/GPU/Hexagon HTP).

geniex-rs bridges this C/C++ runtime directly into idiomatic Rust, providing a layered architecture:

1. Native Bindings (geniex-sys): Raw bindgen bindings to geniex.h and target libraries (geniex.lib / geniex.dll on Windows, libgeniex.so on Linux).
2. Safe RAII Wrappers (geniex): High-level Llm and Vlm abstractions that automatically manage native C handles using Rust’s Drop semantics, avoiding memory leaks and dangling pointers.
3. Automated Platform Support: Support for aarch64-pc-windows-msvc (Snapdragon X Plus / X Elite), aarch64-unknown-linux-gnu (IoT & Edge devices like Dragonwing QCS9075 / RB5), and x86_64 workstation fallbacks for CI/CD.

---

2. Zero-Setup Windows ARM64 DX: Automated DLL Resolution

A major pain point when developing native C/C++ hardware applications on Windows ARM64 is dynamic link resolution. Native binaries depend on geniex.dll and dynamic plugin libraries (such as geniex_plugin_llama_cpp.dll or qairt.dll). In standard Cargo workflows, missing runtime DLLs cause immediate execution crashes with code 0xc0000135 (STATUS_DLL_NOT_FOUND).

geniex-rs eliminates this friction through build automation in build.rs:

// Inside geniex-rs/build.rs
fn main() {
    // 1. Locate native SDK binaries via CARGO_GENIEX_LIB_DIR
    if let Ok(lib_dir) = std::env::var("CARGO_GENIEX_LIB_DIR") {
        println!("cargo:rustc-link-search=native={}", lib_dir);
        println!("cargo:rustc-link-lib=dylib=geniex");
    }

    // 2. Automated Target Copying for Windows ARM64
    let target_os = std::env::var("CARGO_CFG_TARGET_OS").unwrap_or_default();
    if target_os == "windows" {
        // Automatically stage geniex.dll and plugin DLLs to Cargo's output folder
        // (target/debug or target/release), enabling friction-free `cargo run`
        stage_windows_arm64_dlls();
    }
}

By configuring CARGO_GENIEX_LIB_DIR once (or placing libraries in vendor/lib/), developers can execute cargo run and cargo test out of the box without manually modifying system %PATH% variables or copying files.

# Set the native C SDK library path once
$env:CARGO_GENIEX_LIB_DIR="C:\path\to\GenieX\sdk\pkg-geniex\lib"

# Run tests or example binaries out of the box
cargo test --workspace
cargo run -p geniex-rust-example

---

3. High-Level Usage Example: Safe Streaming Inference

The geniex-rs API simplifies model initialization, native chat template formatting, and streaming response tokens into a safe, clean interface:

use geniex::*;

fn main() -> Result<()> {
    // 1. Initialize the GenieX C runtime
    init()?;
    println!("GenieX SDK version: {}", version());

    // 2. Discover available execution plugins (e.g. qairt, llama_cpp)
    let plugins = get_plugin_list()?;
    println!("Available plugins: {:?}", plugins);

    // 3. Instantiate an LLM with default configurations
    let config = ModelConfig::default();
    let mut llm = Llm::create(
        "models/qwen2.5-0.5b-instruct.gguf",
        "llama_cpp", // or "qairt" for NPU acceleration
        &config,
        None, // SamplerConfig
        None, // GenerationConfig
        None, // DialogConfig
    )?;

    // 4. Format prompt using native model chat templates
    let messages = vec![ChatMessage {
        role: "user".to_string(),
        content: "Explain on-device NPU acceleration in one sentence.".to_string(),
    }];
    let prompt = llm.apply_chat_template(&messages, None, false, true)?;

    // 5. Stream generated tokens in real time
    println!("--- Prompt Response ---");
    let (response, profile) = llm.generate::<fn(&str) -> bool>(
        Some(&prompt),
        None,
        None,
        Some(|token| {
            print!("{}", token);
            true // Continue generation loop
        }),
    )?;

    println!("\n\nTokens generated: {}", profile.token_count);
    println!("Speed: {:.2} tokens/sec", profile.decoding_speed);

    // 6. Clean up native runtime resources
    deinit()?;
    Ok(())
}

---

4. Multimodal & Enterprise Features

Beyond standard text generation, geniex-rs includes built-in support for advanced GenAI workflows:

• Multimodal (VLM) Support: Safe APIs via Vlm::create() for vision-language models capable of processing image inputs alongside text prompts.
• KV Cache Management: State persistence APIs to save and reload Key-Value (KV) cache states across session boundaries for low-latency conversations.
• Dynamic Device Alias Resolution: Switch between cpu, gpu, npu, or custom plugin backends dynamically at runtime.
• Comprehensive Integration Test Suite: Built-in test suite verifying version checks, default configs, error code mapping (GeniexError), dynamic plugin scanning, and device resolution (geniex_resolve_device).

---

5. Benchmark Performance: Surface Pro & Snapdragon X Plus

Tested on a Microsoft Surface Pro equipped with a Snapdragon X Plus (8-core) and 16GB LPDDR5x RAM, geniex-rs delivers low-latency on-device inference:

Metric	Local NPU / GPU Execution (Qwen 0.5B / Gemma 2B)
TTFT (Time to First Token)	~130ms - 160ms
Decoding Speed (TPS)	~30+ tokens/sec
Target Architecture	Windows ARM64 (aarch64-pc-windows-msvc)
Hardware Accelerator	Hexagon NPU / Adreno GPU via GenieX Runtime
Memory Consumption	~1.8GB LPDDR5x RAM

Near-zero TTFT provides instant responsiveness for local agentic workflows, code completion tools, and offline assistants-all without sending data to external cloud APIs.

---

6. Getting Started & Resources

geniex-rs is dual-licensed under the BSD 3-Clause License to strictly align with Qualcomm’s upstream GenieX project.

• 📦 Crates.io: crates.io/crates/geniex-rs
• 🐙 GitHub Repository: github.com/Maki-Grz/geniex-rs
• 🔗 Upstream Engine: qualcomm/GenieX on GitHub

Contributions, issues, and PRs are welcome on GitHub!