Caro5 Day 6 part 1: Reviewing 🍄

After the disappointment from yesterday, it was time to review. What I have now is supervised learning from self-play data:

ONNX + ONNX Runtime Web

Pipeline: Python (PyTorch training) ↓ export ONNX model (.onnx) ↓ ONNX Runtime Web (in TypeScript) ↓ CNN inference inside your engine

1. Train baseline model from current JSONL.
2. Plug model into MCTS as the evaluator/prior provider.
3. Generate new self-play games using model-guided MCTS.
4. Train on those games.
5. Evaluate new model vs previous model/bots.
6. Promote only if it wins enough.

That loop is the real RL system, so the supervised foundation is needed before that.

1. VCF solver Search only forcing moves that create immediate winning threats:

   • open four
   • broken four
   • direct five threat Defender replies are only the required blocks. This catches lines like k5 -> l4 forced -> i3 -> i4 forced -> h2.

2. VCT solver Broader search that also allows forcing open-threes / square-threes:

   • live three
   • square-three pivot
   • double-three
   • four-three Defender replies are the moves that reduce/kill the threat. This catches earlier moves like j6 and h4.

3. Use it only at the root Before minimax:

   • if I have a VCF/VCT win, play it
   • if opponent has one, block its first gain move
   • otherwise use existing minimax

4. Depth/time bounded Something like:

   • VCF depth 7-9 plies
   • VCT depth 5-7 plies
   • strict candidate caps
   • transposition cache

Gomocup expert pretraining is done and it helped.

50-game full Swap2 arena with 8 workers:

• Model: Gomocup-trained ONNX
• Opponent: mcts_balanced
• Wins: 19
• Draws: 5
• Losses: 26
• Avg move count: 72.46
• Duration: 267,856 ms
• Report: /tmp/caro5-arena-swap2-gomocup-8w.json

Previous full Swap2 model result was 9W / 1D / 40L, so Gomocup expert pretraining is a clear improvement.

Next best step: train a mixed model using Gomocup expert examples plus model-guided/self-play MCTS examples, with early stopping around the validation peak instead of pushing to overfit.

First Win 25W / 1D / 24L

Finally, our first win! Mixed model arena result is basically even:

• As opener: 11W / 1D / 13L
• As chooser: 14W / 0D / 11L
• As color 0: 16W / 0D / 13L
• As color 1: 9W / 1D / 11L

This is a real improvement over the Gomocup-only model result: 19W / 5D / 26L. The mixed dataset looks better than expert-only for this bot/search setup, crossing from clearly negative to roughly parity against mcts_balanced.

This is the first result that looks directionally useful.

More important signal is not just 25/1/24 is that adding Gomocup expert data plus our self-play/model-guided data beat Gomocup-only under the same full Swap2 arena setup. Next best step is probably to make this loop more systematic:

1. Train mixed model with a longer run, maybe 20-40 epochs instead of 6.
2. Generate fresh mcts_model_guided self-play using this stronger mixed model.
3. Retrain on Gomocup + old self-play + new model-guided self-play.
4. Re-run the same 50-game arena, then maybe 200 games once it looks stable.