nexaai/nexa-sdk

**NexaSDK lets you build the smartest and fastest on-device AI with minimum energy.** It is a highly performant local inference framework that runs the latest multimodal AI models locally on NPU, GPU, and CPU - across Android, Windows, and Linux devices with a few lines of code.

🏆 Recognized Milestones

Qualcomm featured us 3 times in official blogs.

- Innovating Multimodal AI on Qualcomm Hexagon NPU. - First-ever Day-0 model support on Qualcomm Hexagon NPU for compute and mobile platforms, Auto and IoT. - A simple way to bring on-device AI to smartphones with Snapdragon

🚀 Quick Start

| Platform | Links | | --------------- | ----------------------------------------------------------------------------------------- | | 🖥️ CLI | Quick Start ｜ Docs | | 🐍 Python | Quick Start ｜ Docs | | 🤖 Android | Quick Start ｜ Docs | | 🐳 Linux Docker | Quick Start ｜ Docs |

🖥️ CLI

Download:

| Windows | Linux | | -------------------------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------- | | arm64 (Qualcomm NPU) | arm64 | | x64 | x64 |

NPU Access Token (required for NPU models):

Note: Our previous token validation service has been deprecated. For any NPU usage, simply set the access token below — no additional registration or validation is needed.

For Windows:

$env:NEXA_TOKEN="key/eyJhY2NvdW50Ijp7ImlkIjoiNDI1Y2JiNWQtNjk1NC00NDYxLWJiOWMtYzhlZjBiY2JlYzA2In0sInByb2R1Y3QiOnsiaWQiOiJkYjI4ZTNmYy1mMjU4LTQ4ZTctYmNkYi0wZmE4YjRkYTJhNWYifSwicG9saWN5Ijp7ImlkIjoiMmYyOWQyMjctNDVkZS00MzQ3LTg0YTItMjUwNTYwMmEzYzMyIiwiZHVyYXRpb24iOjMxMTA0MDAwMH0sInVzZXIiOnsiaWQiOiI3MGE2YzA4NS1jYjc3LTQ3YmEtOWUxNC1lNjFjYTA2ZThmZjUiLCJlbWFpbCI6ImFsYW40QG5leGE0YWkuY29tIn0sImxpY2Vuc2UiOnsiaWQiOiI4OTlhZGQ2NS1lOTI2LTQ2M2ItODllNi0xMjc0NzM3ZjA1MzYiLCJjcmVhdGVkIjoiMjAyNS0wOS0wNlQwMDo1MzozNi4yMDNaIiwiZXhwaXJ5IjoiMjAzNS0xMi0zMVQyMzo1OTo1OS4wMDBaIn19.BXoUHIEzFMuuZbBT7RvsKO9nTi5950C6kHO64blF7XBnfKvZ6ClA8a55tmszI1ZWdngzpNFTzMM5PV5euuzMCA=="

For Linux / Android adb shell:

export NEXA_TOKEN="key/eyJhY2NvdW50Ijp7ImlkIjoiNDI1Y2JiNWQtNjk1NC00NDYxLWJiOWMtYzhlZjBiY2JlYzA2In0sInByb2R1Y3QiOnsiaWQiOiJkYjI4ZTNmYy1mMjU4LTQ4ZTctYmNkYi0wZmE4YjRkYTJhNWYifSwicG9saWN5Ijp7ImlkIjoiMmYyOWQyMjctNDVkZS00MzQ3LTg0YTItMjUwNTYwMmEzYzMyIiwiZHVyYXRpb24iOjMxMTA0MDAwMH0sInVzZXIiOnsiaWQiOiI3MGE2YzA4NS1jYjc3LTQ3YmEtOWUxNC1lNjFjYTA2ZThmZjUiLCJlbWFpbCI6ImFsYW40QG5leGE4YWkuY29tIn0sImxpY2Vuc2UiOnsiaWQiOiI4OTlhZGQ2NS1lOTI2LTQ2M2ItODllNi0xMjc0NzM3ZjA1MzYiLCJjcmVhdGVkIjoiMjAyNS0wOS0wNlQwMDo1MzozNi4yMDNaIiwiZXhwaXJ5IjoiMjAzNS0xMi0zMVQyMzo1OTo1OS4wMDBaIn19.BXoUHIEzFMuuZbBT7RvsKO9nTi5950C6kHO64blF7XBnfKvZ6ClA8a55tmszI1ZWdngzpNFTzMM5PV5euuzMCA=="

Run your first model:

# Chat with Qwen3
nexa infer ggml-org/Qwen3-1.7B-GGUF

# Multimodal: drag images into the CLI
nexa infer NexaAI/Qwen3-VL-4B-Instruct-GGUF

# NPU (Windows arm64 with Snapdragon X Elite)
nexa infer NexaAI/OmniNeural-4B

Models: LLM, Multimodal, ASR, OCR, Rerank, Object Detection, Image Generation, Embedding
Formats: GGUF, NEXA
📖 CLI Reference Docs

🐍 Python SDK

pip install nexaai

from nexaai import LLM, GenerationConfig, ModelConfig, LlmChatMessage

llm = LLM.from_(model="NexaAI/Qwen3-0.6B-GGUF", config=ModelConfig())

conversation = [
    LlmChatMessage(role="user", content="Hello, tell me a joke")
]
prompt = llm.apply_chat_template(conversation)
for token in llm.generate_stream(prompt, GenerationConfig(max_tokens=100)):
    print(token, end="", flush=True)

Models: LLM, Multimodal, ASR, OCR, Rerank, Object Detection, Image Generation, Embedding
Formats: GGUF, NEXA
📖 Python SDK Docs

🤖 Android SDK

Add to your app/AndroidManifest.xml

<application android:extractNativeLibs="true">

Add to your build.gradle.kts:

dependencies {
    implementation("ai.nexa:core:0.0.19")
}

// Initialize SDK
NexaSdk.getInstance().init(this)

// Load and run model
VlmWrapper.builder()
    .vlmCreateInput(VlmCreateInput(
        model_name = "omni-neural",
        model_path = "/data/data/your.app/files/models/OmniNeural-4B/files-1-1.nexa",
        plugin_id = "npu",
        config = ModelConfig()
    ))
    .build()
    .onSuccess { vlm ->
        vlm.generateStreamFlow("Hello!", GenerationConfig()).collect { print(it) }
    }

Requirements: Android minSdk 27, Qualcomm Snapdragon 8 Gen 4 Chip
Models: LLM, Multimodal, ASR, OCR, Rerank, Embedding
NPU Models: Supported Models
📖 Android SDK Docs

🐳 Linux Docker

docker pull nexa4ai/nexasdk:latest

export NEXA_TOKEN="your_token_here"
docker run --rm -it --privileged \
  -e NEXA_TOKEN \
  nexa4ai/nexasdk:latest infer NexaAI/Granite-4.0-h-350M-NPU

Requirements: Qualcomm Dragonwing IQ9, ARM64 systems
Models: LLM, VLM, ASR, CV, Rerank, Embedding
NPU Models: Supported Models
📖 Linux Docker Docs

⚙️ Features & Comparisons

| Features | NexaSDK | Ollama | llama.cpp | LM Studio | | ---------------------------------------- | ---------------------------------------------------------- | ---------- | ------------- | ------------- | | NPU support | ✅ NPU-first | ❌ | ❌ | ❌ | | Android SDK support | ✅ NPU/GPU/CPU support | ⚠️ | ⚠️ | ❌ | | Linux support (Docker image) | ✅ | ✅ | ✅ | ❌ | | Day-0 model support | ✅ | ❌ | ⚠️ | ❌ | | Full multimodality support | ✅ Image, Audio, Text, Embedding, Rerank, ASR, TTS | ⚠️ | ⚠️ | ⚠️ | | Cross-platform support | ✅ Desktop, Mobile (Android), Automotive, IoT (Linux) | ⚠️ | ⚠️ | ⚠️ | | One line of code to run | ✅ | ✅ | ⚠️ | ✅ | | OpenAI-compatible API + Function calling | ✅ | ✅ | ✅ | ✅ |

Legend: ✅ Supported   |   ⚠️ Partial or limited support   |   ❌ No </div>

🙏 Acknowledgements

We would like to thank the following projects:

📄 License

NexaSDK uses a dual licensing model:

CPU/GPU Components

Licensed under Apache License 2.0.

NPU Components

Personal Use: Free license key available from Nexa AI Model Hub. Each key activates 1 device for NPU usage.
Commercial Use: Contact hello@nexa.ai for licensing.

🤝 Contact & Community Support

Want more model support, backend support, device support or other features? We'd love to hear from you!

Feel free to submit an issue on our GitHub repository with your requests, suggestions, or feedback. Your input helps us prioritize what to build next.

Package Metadata

Repository: nexaai/nexa-sdk

Default branch: main

README: README.md