Instructions to use OsaurusAI/Nemotron-3-Nano-Omni-30B-A3B-JANG_6M with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use OsaurusAI/Nemotron-3-Nano-Omni-30B-A3B-JANG_6M with MLX:

# Make sure mlx-vlm is installed
# pip install --upgrade mlx-vlm

from mlx_vlm import load, generate
from mlx_vlm.prompt_utils import apply_chat_template
from mlx_vlm.utils import load_config

# Load the model
model, processor = load("OsaurusAI/Nemotron-3-Nano-Omni-30B-A3B-JANG_6M")
config = load_config("OsaurusAI/Nemotron-3-Nano-Omni-30B-A3B-JANG_6M")

# Prepare input
image = ["http://images.cocodataset.org/val2017/000000039769.jpg"]
prompt = "Describe this image."

# Apply chat template
formatted_prompt = apply_chat_template(
    processor, config, prompt, num_images=1
)

# Generate output
output = generate(model, processor, formatted_prompt, image)
print(output)

Notebooks
Google Colab
Kaggle
Local Apps Settings
LM Studio

How to use OsaurusAI/Nemotron-3-Nano-Omni-30B-A3B-JANG_6M with Pi:

Start the MLX server

# Install MLX LM:
uv tool install mlx-lm
# Start a local OpenAI-compatible server:
mlx_lm.server --model "OsaurusAI/Nemotron-3-Nano-Omni-30B-A3B-JANG_6M"

Configure the model in Pi

# Install Pi:
npm install -g @earendil-works/pi-coding-agent
# Add to ~/.pi/agent/models.json:
{
  "providers": {
    "mlx-lm": {
      "baseUrl": "http://localhost:8080/v1",
      "api": "openai-completions",
      "apiKey": "none",
      "models": [
        {
          "id": "OsaurusAI/Nemotron-3-Nano-Omni-30B-A3B-JANG_6M"
        }
      ]
    }
  }
}

Run Pi

# Start Pi in your project directory:
pi

Hermes Agent

How to use OsaurusAI/Nemotron-3-Nano-Omni-30B-A3B-JANG_6M with Hermes Agent:

Start the MLX server

# Install MLX LM:
uv tool install mlx-lm
# Start a local OpenAI-compatible server:
mlx_lm.server --model "OsaurusAI/Nemotron-3-Nano-Omni-30B-A3B-JANG_6M"

Configure Hermes

# Install Hermes:
curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash
hermes setup
# Point Hermes at the local server:
hermes config set model.provider custom
hermes config set model.base_url http://127.0.0.1:8080/v1
hermes config set model.default OsaurusAI/Nemotron-3-Nano-Omni-30B-A3B-JANG_6M

Run Hermes

hermes

Atomic Chat

OpenClaw

How to use OsaurusAI/Nemotron-3-Nano-Omni-30B-A3B-JANG_6M with OpenClaw:

Start the MLX server

# Install MLX LM:
uv tool install mlx-lm
# Start a local OpenAI-compatible server:
mlx_lm.server --model "OsaurusAI/Nemotron-3-Nano-Omni-30B-A3B-JANG_6M"

Configure OpenClaw

# Install OpenClaw:
npm install -g openclaw@latest
# Register the local server and set it as the default model:
openclaw onboard --non-interactive --mode local \
  --auth-choice custom-api-key \
  --custom-base-url http://127.0.0.1:8080/v1 \
  --custom-model-id "OsaurusAI/Nemotron-3-Nano-Omni-30B-A3B-JANG_6M" \
  --custom-provider-id mlx-lm \
  --custom-compatibility openai \
  --custom-text-input \
  --accept-risk \
  --skip-health

Run OpenClaw

openclaw agent --local --agent main --message "Hello from Hugging Face"

Nemotron-3-Nano-Omni-30B-A3B-Reasoning · JANG_6M

29.1 GB · ~110.0 tok/s decode on M5 Max · 31B total / ~3B active · hybrid Mamba-2 + Attention + MoE · native MLX

Full multimodal (text + image + audio + video) port of NVIDIA's Nemotron-3-Nano-Omni-30B-A3B-Reasoning to Apple MLX — calibrated with a measured Hessian, an imatrix fit and AWQ, with the RADIO vision tower and the Parakeet speech encoder kept at fp16 so every modality actually works.

Modality	Status
Text	✅ reasoning on by default
Image (RADIO ViT, 512px tiles + thumbnail)	✅
Video (frame sampling + EVS pruning)	✅
Audio / ASR (Parakeet 24-layer Conformer)	✅
Tool calling (agentic)	✅ XML `<tool_call><function=…>`

Quantization recipe

JANG_6M is a mixed-precision allocation (6-bit, 8-bit), not a single width. The bit map is chosen per tensor role, then calibrated:

Component	Precision
Routed experts `switch_mlp.fc1` / `fc2` (93% of params)	6-bit / 6-bit
Shared expert, attention q/k/v/o, `lm_head`	8-bit
Mamba-2 `in_proj` / `out_proj`, embeddings	8-bit
`conv1d`, `A_log`, `D`, `dt_bias`, router gate, all norms	fp16
Vision tower, Parakeet encoder, both projectors	fp16 (never quantized)

Group size 64. 164 quantized modules; 1106 tower tensors passed through untouched.

Why there is no 2-bit variant

This model's expert MLP is unusually sensitive to low-bit quantization, so the routed experts carry a hard minimum width regardless of profile. A nominal "2-bit" build lands within ~7% of the 4-bit one and is measurably worse — not a 2-bit bundle in any useful sense, so we do not ship one.

Calibration

Quantization is calibrated against measured activations from a domain-weighted corpus (coding, agentic/tool, reasoning, general, multilingual, long-context), not fitted to the weights alone:

imatrix — the affine fit is weighted toward the channels that actually carry signal.
AWQ — per-channel scales, folded so the result is numerically equivalent and costs nothing at inference.
Hessian — captured per MoE layer and used to steer the bit allocation.

QAT is not available for this model: NVIDIA's own quantized releases are post-training (modelopt, MIXED_PRECISION), and NVFP4 codes carry no meaning in an affine (scale, bias) layout.

Sampling — use the right preset

Reasoning is ON by default (enable_thinking); turning it off prefills an empty <think></think> block rather than omitting it.

Mode	temperature	top_p	top_k	max_tokens	reasoning_budget	grace
Thinking (default)	0.6	0.95	—	20480	16384	1024
Instruct (non-thinking)	0.2	—	1	1024	—	—
ASR / transcription	1.0	—	1	—	—	—

Stamped into both generation_config.json and jang_config.chat.sampling_defaults. EOS is [2, 11] — dropping 11 means the model never stops.

All bundles in this family

Variant	Size	Notes
JANG_4M	—
JANG_6M	29.1 GB	this bundle
MXFP4	—

Install

hf download OsaurusAI/Nemotron-3-Nano-Omni-30B-A3B-JANG_6M \
  --local-dir Nemotron-3-Nano-Omni-30B-A3B-JANG_6M

Runs in Osaurus / vMLX, which carries the native MLX NemotronHOmni stack (RADIO vision, Parakeet audio, EVS video). The LLM half also loads directly with mlx_lm:

from mlx_lm import load, generate
model, tokenizer = load("Nemotron-3-Nano-Omni-30B-A3B-JANG_6M")

Credits

Downloads last month: 102

Safetensors

Model size

8B params

Tensor type

F32

U32

BF16

F16

MLX

Hardware compatibility

Quantized

Model tree for OsaurusAI/Nemotron-3-Nano-Omni-30B-A3B-JANG_6M

Base model

nvidia/Nemotron-3-Nano-Omni-30B-A3B-Reasoning-BF16

Finetuned

(16)

this model