Authors: Kshitij Thakkar Date: June 2026 Project: The Mind of Tashi Space: Thousand Token Wood Build Small Hackathon Thumbnail: https://cdn-uploads.huggingface.co/production/uploads/67af2623dad5fcb9007967e4/CDs2QxZubDP7_l2bHGJB3.png

The Mind of Tashi: making a 200M-active model's reasoning the game

A Build Small Hackathon entry (Track 2 — An Adventure in Thousand Token Wood). Play it: build-small-hackathon/mind-of-tashi · the whole bundle: collection · 🎬 watch the demo (on LinkedIn).

Most LLM "games" on the Hub are chess, sudoku, or a chatty text adventure. We wanted to build something where the model's reasoning itself is the toy — not a debug panel, not flavour text, but the thing you actually play against. So we built a fighting game where you duel a small reasoning model, and its private chain-of-thought is rendered to you, live, as a mind-scroll. The whole game is out-thinking something that narrates how it's out-thinking you.

The blind-commit contract

Each round, you and the opponent secretly commit one move at the same time — six moves (Strike, Guard, Grapple, Focus, Prāṇa Art, Mist-Step) over a rock-paper-scissors core plus a prāṇa (energy) economy. The catch that makes it a reasoning game and not RPS-with-extras:

The model commits blind. It sees only the match history — never your pending move.

It has to predict you, not react to you. The signature move, Mist-Step, only pays off if your opponent attacks this turn, so it's a pure bet on a read. That recursion — "I think you'll strike, so I Mist-Step / I think you think that, so I draw breath" — is exactly what a reasoning model is good at and a scripted bot is bad at. After both reveal, the model's <think> block scrolls across the footer: you get to see whether it read you correctly.

The blind-commit contract is enforced at every layer — the game, the self-play data harvest, and the OpenEnv gym all run on the same zero-dependency engine.

Bilingual reasoning: English + Hindi/Sanskrit

The game's surface is Himalayan — prāṇa, monks, mist-step, monastery, summit. So we made the cognitive surface match the cultural one: the model reasons in a code-switched English + Hindi/Sanskrit (IAST) register. A real mind-scroll reads like:

They played STRIKE twice, then I drew breath first. They are trying to bait a pattern, not a choice. Their abhyāsa is clear — prahār without prāṇāyāma. I must not fall into their cycle.

A Hub search turned up no prior art for a deliberately Hindi/Sanskrit code-switched reasoning model. It's a clean moat — and it makes every mind-scroll feel like it belongs in the world.

David vs Goliath: a ~200M-active custom MoE

The opponent isn't a frontier model. It's our own custom Mixture-of-Experts student — ~0.4B total parameters, ~200M active per token — which is 10–100× smaller (active) than the API models it learned from. That's the whole thesis: can a tiny, local model be a genuinely good opponent at a reasoning game?

We got there in two stages, both shipped openly:

1. SFT on self-play traces harvested from a frontier-API teacher pool (Gemini, Mistral, Sarvam, OpenRouter free tier) under the persona scaffold. → mind-of-tashi-micro-sft
2. GRPO from the SFT checkpoint, with the game's own scoring as the reward (HP-delta + win/loss + a bilingual-lexicon bonus + format/legality terms). → mind-of-tashi-micro-grpo

Training ran on Modal L4; the SFT recipe and the GRPO reward shape are self-contained and reproducible. The format gate (does it emit <think>…</think> + a parseable {move, taunt} with a legal move?) scores 20/20 via transformers and 20/20 on the deployed Q4_K_M GGUF.

The originality bet: fight the model's uncertainty

Because the opponent is a small local model, we can read signals a cloud API can never expose — and we turned them into mechanics:

• 🎯 Conviction Meter. We read the model's per-token logprobs and compute 1 − normalised entropy over the top-k candidates — how concentrated its belief was. The mind-scroll is tinted by confidence and the lowest-confidence "wavering" word flashes red. You can literally see when it's unsure.
• 🔥 Crack Her Composure. Landing reads and counters raises the opponent's sampling temperature between turns — its reasoning genuinely frays, and (because hotter sampling = more entropy) the Conviction Meter drops as you rattle it.
• 🔒 Grammar-Locked Oath. Spend prāṇa to seal one of the opponent's moves: a per-turn GBNF grammar drops that token from its decode grammar, so it cannot pick it — and its reasoning visibly reroutes around the hole.

All three preserve the contracts: blind-commit (signals derive from the model reasoning about history, never your pending move) and off-the-grid (every signal comes out of the local inference loop, no new network surface).

A field report worth keeping: llama.cpp silently renormalises qwen3moe routing

A deployable-blocker we root-caused and want on the record. Our SFT'd MoE scored 20/20 through transformers but produced total garbage through every llama.cpp runtime (ollama, llama-cpp-python, llama-cli) — mangled tokens, repetition loops, broken JSON. It wasn't quantization (BF16 GGUF failed identically), not the tokenizer (IDs matched 17/17), not the chat template (byte-identical renders).

The cause: our model trains with norm_topk_prob: false (raw top-k expert routing), but llama.cpp's qwen3moe graph hardcodes norm_w = true and ignores the GGUF expert_weights_norm key — and convert_hf_to_gguf.py reads route_norm while Qwen3 configs name the field norm_topk_prob, so the flag is never even written. We proved it by flipping norm_topk_prob false→true in PyTorch in-process: the clean output collapsed into the exact llama.cpp garbage.

Our fix keeps the pipeline reproducible with stock pip install (no custom llama.cpp build): we train to match the runtime — SFT with norm_topk_prob = true so the model adapts to normalised routing, and the resulting GGUF runs coherently on stock llama.cpp / ollama. (The underlying bug is real and worth fixing upstream regardless.)

Two runtimes, one model

• Deployed Space → Hugging Face ZeroGPU. The opponent runs through transformers with a GPU allocated on demand per move (@spaces.GPU), so it's fast and scales to many concurrent players. It's still off the grid: the GPU is local to the Space, no cloud API in the loop.
• Local clone → llama.cpp. Clone the repo and docker build (or pip install llama-cpp-python and set BACKEND=llamacpp) and the same GGUF student runs entirely on CPU — no GPU, no cloud. That's the Llama Champion path.

Same model, two runtimes, one BACKEND switch.

A locally-generated soundtrack 🎶

Even the music is small-and-local. Each of the ten arenas has its own upbeat battle loop, generated with MusicGen on a laptop RTX 3060 — no audio API, no licensed tracks, just one prompt per arena (the village opens on driving bamboo flute and hand-drums; the summit boss is a rock guitar riff; the veil is otherworldly electronic). They ship as static assets, so the soundtrack is as off-the-grid as the opponent, and a 🔇/🔊 toggle keeps it opt-in. Generating the whole set was a two-minute GPU job once we swapped sparse "ambient drone" prompts (which MusicGen turns to noise) for rhythmic, melody-forward ones.

The six-artifact bundle

We shipped the whole vertical stack, openly:

The OpenEnv gym wraps the duel as a trl.GRPOTrainer-compatible environment (action = a raw completion, observation = {history, hp, prāṇa, legal_moves, persona}, reward = the game's own scoring). The api opponent backend is hard-refused when SPACE_ID is set, so a deployed gym can only spar against a local/mock opponent — the off-the-grid boundary holds.

What we're claiming

Six badges: Off the Grid (local inference, no cloud API at runtime), Llama Champion (GGUF via llama.cpp on the local path), Off-Brand (a fully custom frontend on gradio.Server), Well-Tuned (a fine-tuned custom-architecture MoE, SFT + GRPO), Sharing is Caring (dataset, models, GGUFs, gym, and leaderboard all public), and Field Notes (this writeup).

Thank you 🙏

Made possible by our sponsors: Hugging Face (the Hub, ZeroGPU, Gradio, OAuth), Modal (L4 training compute), llama.cpp (the local runtime), and NVIDIA (the silicon under all of it — ZeroGPU's RTX Pro 6000 Blackwell, L4 for training, and a local RTX 3060 for the dev loop).

Watch & follow

• 🎬 Demo video (posted on LinkedIn): watch the duel
• 💼 LinkedIn announcement: the post
• 🐦 X / Twitter announcement: the thread

Come read a mind that's reading you: build-small-hackathon/mind-of-tashi.

---