🤝 Open to Collab

R PRO

juiceb0xc0de

44 29 163

https://github.com/JuiceB0xC0de

AI & ML interests

Mechanistic interpretation SAE weight training Novel architecture exploration Brain atlas builds

Recent Activity

reacted to Bc-AI's post with 😎 about 2 hours ago

Building a 6.58B sparse MoE model from scratch on a single GPU. Hey everyone! Today is day 1 of training Smilyai-Lab's new model I call G1-MINI. It's basically the smaller version of our planned model G1 which will be 20B and activate about 2B per token. MINI activates about 1.16B params per token and is currently training right now. If no errors spring up now, I'd say i can launch sometime around September 15th~ish. Thanks to our beta testers: @guardamarcos @ProCreations @juiceb0xc0de @Timmy6767 @Sbui503 @atom77777 @Fishtiks @smartdigitalnetworks @EmetTheGolum @smilyai-large-team @MUK-IS-GOAT @Bc-AI @Banaxi-Tech @vovaRL @Datdanboi25

updated a dataset about 2 hours ago

juiceb0xc0de/qwen38-27b-cu128-torch280-py311

updated a dataset about 2 hours ago

juiceb0xc0de/gemma-4-e2b-it-cu128-torch291-py311

View all activity

Organizations

reacted to Bc-AI's post with 😎 about 2 hours ago

Post

Building a 6.58B sparse MoE model from scratch on a single GPU.
Hey everyone! Today is day 1 of training Smilyai-Lab's new model I call G1-MINI. It's basically the smaller version of our planned model G1 which will be 20B and activate about 2B per token. MINI activates about 1.16B params per token and is currently training right now. If no errors spring up now, I'd say i can launch sometime around September 15th~ish. Thanks to our beta testers:
@guardamarcos
@ProCreations
@juiceb0xc0de
@Timmy6767
@Sbui503
@atom77777
@Fishtiks
@smartdigitalnetworks
@EmetTheGolum
@smilyai-large-team
@MUK-IS-GOAT
@Bc-AI
@Banaxi-Tech
@vovaRL
@Datdanboi25

3 replies

reacted to Bc-AI's post with 😎 4 days ago

Post

2548

Smilyai News
Hello everyone! August has been a crazy month for us at Smilyai-Labs. We've been doing lots behind the scenes, so here's the latest 👇

1. MiniCoder
We are very close to releasing MiniCoder-1, our first-generation coding model designed for reasoning and coding. Our planned context window is 128K, but the earlier versions probably will not support that long! It's currently in the final stages of DPO so expect a release in early september.
Release: VERY SOON™🤣

2. Smilyai G1
So, the current plan is 20B parameter model total, with a MoE architecture, activating around 2B parameters per token. Its desgigned for maximum performance but keeping it runnable on consumer hardware. It's only a plan and i have no idea when me and the team can finish it. Expect a launch around the end of september to early october-ish. I have no guarantees so don't quote me on the launch date.

3. T1
Smilyai-T1 is another major model we are working on.
The goal for T1 is to take what we learnt from the countless architectural experiments and creating a powerful model designed for thinking. Think MiniCoder but reasons more and G1 but more capable. Its main goals are coding, math, reasoning and general capability.
4. Omni
We are also planning Omni, our first from scratch multimodal model. It will not launch this year as it will take a while. We are actively researching the best architecture for it and we will update progress as we go!

Thanks to our beta testers:
@guardamarcos
@ProCreations
@juiceb0xc0de
@Timmy6767
@Sbui503
@atom77777
@Fishtiks
@smartdigitalnetworks
@EmetTheGolum
@smilyai-large-team
@MUK-IS-GOAT
@Bc-AI

Thanks to my friends who work with me at lunchtimes (Smilyai-Labs team):
@MUK-IS-GOAT
@smilyai-large-team

August was wild. Let’s see what September brings. 🚀

— Bc-AI, on behalf of SmilyAI Labs

reacted to Banaxi-Tech's post with 😎 5 days ago

Post

2861

We’re excited to release BananaMind OS 2.0, a major update to our portable operating system for running AI models locally.

BananaMind OS runs directly from an ISO without Linux, installation, a cloud connection, or modifying your disks.

The new Version 2.0 adds a graphical interface with mouse support, a model library, multi-turn chat, configurable KV cache, maximum tokens and temperature
controls, automatic x87/SSE/SSE2 CPU detection, BIOS support, native UEFI support and a dedicated 486 compatibility mode.

Model weights are not loaded during GRUB or startup. Only the model catalog is read, and the selected model is loaded after BananaMind OS starts.

We now have a new .litemodel format supports multiple architectures, including BananaMind models, SmolLM, SmolLM2, GPT-X2.5, min-spark 1.1 and Rose-Mini.
Check it out at:
https://github.com/BananaMind/BananaMindOS

You can build an ISO yourself using the .sh or .bat script. Select the models and quantizations you want, and the builder will download, quantize
and package them locally.
Prebuilt 10MB, 25MB, 100MB and 250MB model presets are available here:
https://github.com/BananaMind/BananaMindOS/releases/tag/v2.0.0
Use the regular preset ISOs for BIOS and GRUB, including the 486 compatibility mode.

Warning: Im currently uploading the ISOs, all up to 100MB is present, 250MB is getting uploaded
Use the files ending in -uefi.iso for the native x86-64 UEFI graphical frontend and improved firmware mouse support.

The preset name describes the RAM class of the individual included models. Models remain on the ISO until selected, so including multiple models does not load all of them into RAM.
486DX with an x87 FPU, Pentium and newer x86 processors are supported. Modern x86-64 computers are supported through UEFI. It currently doesent support processors without an FPU.

Comment if you want me to run it on 0.04MHz (pls dont)

Video Credit:
Song: Matzan - Redesigned
Music provided by NoCopyrightSounds

3 replies

posted an update 5 days ago

Post

100

Just dropped a new SAE juiceb0xc0de/gemma-4-e2b-it-SAE

I built this using the Event-Aware SAE Trainer I have been developing. It trained all 35 layers in 1 run and handled the transition from KV owning layers to the KV cache at layers 15 and 16 where MLP width doubles.

I developed the trainer with two goals in mind, keep it accessible and affordable. I wanted to make SAE training simple enough that you don't require a team of frontier developers or a corporate budget. Anyone can choose a model -> hit play -> train a set of SAE's. Zero hyperparameter tweaking. Train every layer.

Optimized features:

- Single block rolling capture approx. 2.9x faster vs full forwards capture zero measured relative error
- Hoist system the model lives in CPU the layers required for SAE training stay in VRAM
- Augmented-Lagrangian λ controller with two sided nudge controller for faster, bidirectional L=0 control
- Dead feature revival preventing dead pileup saving layers from requiring another pass
- Layer-aware early stop triggers prevent training from executing maximum training steps dropping training as low as recorded 1000 steps

juiceb0xc0de/gemma-4-e2b-it-SAE
https://github.com/JuiceB0xC0de/event-aware-SAE-trainer

Drop a new discussion in the community and let me know what you think!

reacted to GoktugD's post with 😎 6 days ago

Post

1984

🇹🇷 Introducing T3 Gemstone — Edge AI & Cybersecurity Models from Türkiye

We've been building an open AI ecosystem focused on practical models that can run closer to the edge — not only in large datacenters.

Today, I'm introducing T3 Gemstone, a growing family of compact AI models built around edge inference, cybersecurity and computer vision.

💎 T3 Gemstone currently includes:

* NanoSOC Gemstone 2B — GGUF
* NanoSOC Gemstone 4B — GGUF
* Gemstone Person/Object Detector Nano
* Edge-focused AI experiments and deployments

The goal is simple:

Build smaller, practical and open AI systems that can actually run on constrained hardware.

This is part of a much larger open-source effort we're building from Türkiye across LLMs, cybersecurity, computer vision, retrieval, speech and edge AI.

There is much more coming.

🤗 Explore my models, datasets and demos:
@GoktugD

🛡️ NanoSOC:
Werea-co/Werea-NanoSOC-8B

Feedback, benchmarks, collaborations and contributions are very welcome.

If you're interested inopen-source AI, Turkish AI research, edge AI or cybersecurity models, follow the journey.

We're just getting started. 🇹🇷

#AI #OpenSource #HuggingFace #LLM #EdgeAI #Cybersecurity #ComputerVision #TurkishAI #MachineLearning

reacted to Bc-AI's post with 😎 6 days ago

Post

2003

Hello everyone! Happy to say that MiniCoder-1 is now in the instruction tuning phase. It is a 216M~ parameter model trained on 16B tokens. It ran on an RTX 6000 Pro Blackwell gpu for around 24~ hours. Now we will do SFT and launch as beta while we work on the final important DPO and RLHF phases. Our goal is a small extreemly fast on device coding assistant with CoT reasoning* baked in! - Bc-AI on behalf of the Smilyai-Labs team

*it is a small model so the reasoning quality wont be as good obviously!

2 replies

replied to their post 7 days ago

juiceb0xc0de/qwen3.8-27b-atlas

My apologies for the slow turn around on this one. I got distracted and tried to re-version the atlas building suite I use. Next thing I knew 2 days had gone by, I was knee deep in debugging and I was back to bartending an event weekend with no free time. I made this one queryable in the repo. Hope you like it.

@Banaxi-Tech I haven't forgotten about you. Im just catching up a bunch of projects right now. I'll get your model finished up right away here.

reacted to Bc-AI's post with 🤗 9 days ago

Post

3774

New update! We are currently training a few new models now! Our 3rd generation main LLM standard edition is in training right now. We are also training a new LLM line called Tiny Coder around 350~ish M params. Thanks to @Banaxi-Tech for inspiring the architecture with his Bananamind-2.1-unified test model. Thanks to our beta testers: @juiceb0xc0de @ProCreations @Sbui503 @Fishtiks @MUK-IS-GOAT

4 replies

replied to Bc-AI's post 10 days ago

What are they feeding you in that school? Obscure LLM data and ancient MS DOS tomes?

replied to Bc-AI's post 11 days ago

As always I cant wait to see what kind of hidden conflict your unpredictable and chaotic architecture has in store for me while configuring my mech-interp suite.

reacted to Bc-AI's post with 😎 11 days ago

Post

4140

@juiceb0xc0de @ProCreations @Keeby-smilyai @MUK-IS-GOAT @Sbui503 @Fishtiks Hello Beta testers! I will release our new model in our beta testers organisation as soon as I can! Thanks for being part of our beta testers community!

7 replies

replied to their post 12 days ago

The model is downloading right now. It should take a couple of hours and it will be done

replied to their post 12 days ago

Ya I got you. Give me a bit to set up up here and I'll get it started right away.

replied to danielhanchen's post 20 days ago

Hey just a heads up you split your unsl/oth link accidentally. 🦥

reacted to danielhanchen's post with 😎 20 days ago

Post

5482

Introducing Unsloth Desktop 🦥
The first desktop app to run and train models locally.

• Open-source. Runs on Mac, Windows and Linux
• Supports MLX, diffusion image/video, audio, GGUF
• Connect Claude Code and Codex to local LLMs
• 50% more accurate, self-healing tool calls + sandboxed code exec
• Works for CPU + multiGPU setups - NVIDIA, AMD, Intel, Mac
• Train models 2× faster with 70% less VRAM
• Private web search, deep research, RAG, MCP and exports (NVFP4, GGUF)
• Use Unsloth’s OpenAI-compatible API and cloud models
• Securely deploy LLMs remotely and access anywhere

Unsloth Desktop is now available on http://unsloth.ai
and GitHub.

GitHub: https://github.com/unslothai/unsloth
Blog and Guide: https://unsloth.ai/docs/desktop

6 replies

reacted to sergiopaniego's post with 😎 26 days ago

Post

2625

LFM2.5-2.6B just dropped!

and the @liquidai blog comes with some nice details about the training procedure, so let's analyze it.

basically, a full agent training pipeline but compressed into 2.6B

base model → SFT → specialized teachers per domain (SFT + RLVR) → on-policy distillation back into one student → agentic RL

the two most interesting stages

→ MOPD: the student generates, each prompt routes to its domain teacher for token-level feedback. teachers branch from the same SFT checkpoint, so their signal stays close to the student's distribution

→ agentic RL: multi-turn GRPO inside real harnesses (OpenClaw, Hermes Agent), one sandbox per rollout, a proxy captures token-level trajectories while the harness stays a black box

this makes a 2.6B that beats much larger models on instruction following and tool use

SFT, distillation, RL, RL envs: exactly what we're covering in our Training Agents livestream series (next one coming soon!)

→ model: LiquidAI/LFM2.5-2.6B
→ blog: https://www.liquid.ai/blog/lfm2-5-2-6b
→ live series: https://www.youtube.com/playlist?list=PLo2EIpI_JMQvQZm-kVlz4wY1vWF0LBcf5

3 replies

replied to sergiopaniego's post 26 days ago

Yo convolution layers... You caused me to pay attention hahaha get it. An AI model would laugh at this joke and say "That joke is doing more work than the 22 conv layers, this is a pun on the causal convolution layers." Ahhh dad jokes aside I'm actually kinda stoked to run a little atlas on your model. It's like 22 people sitting at a bar eavesdropping on a few people adjacent to the left of them and predicting based on that narrow view point with 8 patrons listening to the entire thing with clarity and scratching their heads. Ima see what saving KV cache is doing here. To hell with the benchmarks and arXiv papers! Let's weigh it measure it and properly fit it in a suit. I wanna see if it looks properly proportioned or if it's a saw toothed oddball with wide shoulders a narrow midriff big ol' booty tiny calves huge shins and stiletto sized feet.

reacted to Bc-AI's post with 👍 29 days ago

Post

192

Hello Everyone! Bc-AI here from Smilyai-labs! Today we have done our latest update for CodVa-1-Small. It is very powerful for coding, and benchmark results will come soon. However, it is NOT good for other tasks, with high hallucination rates. We will perform RLHF and DPO very soon!

reacted to badaoui's post with 😎 about 1 month ago

Post

2392

432 GB of ultra-fast HBM4 and up to 23.3 TB/s of memory bandwidth on a single GPU 🤯.

Two weeks ago, we got early access to AMD's new Instinct MI455X, and our first goal was simple: make sure 🤗 Transformers works on day one.

Over the past few weeks, we worked closely with the AMD team to validate the platform, enable Flash Attention, add torchcodec support for multimodal models, and resolve issues uncovered during testing.

The result:
✅ 99.5% success rate across our 24 core Transformers model architectures - already on par with our daily CI on previous AMD and NVIDIA platforms.

The hardware is just as exciting. With 432 GB of HBM per GPU, our early capacity experiments showed more than 3× the concurrent long-context requests compared to MI300, thanks to the much larger KV cache capacity.

A huge thanks to the AMD team for the early access and the great collaboration!

Read the full blog 👇
https://huggingface.co/blog/badaoui/transformers-on-amd-mi455

1 reply

replied to kingabzpro's post about 1 month ago

Hey! If you're looking for a more in depth break down regarding pricing or any of the other features on Hugging Face the Docs section is a great resource as well. The Hugging Chat model is free to use for queries on a lot more than just the platform. You can find everything from optimizing inference speed to working with newer architectures like Mamba, or getting started with a Reachy Mini. Hope that helps!

R PRO

AI & ML interests

Recent Activity

Organizations

juiceb0xc0de's activity