Palace-9
Local model inspection
A playable tic-tac-toe lab that pairs a deterministic controller with a compact MoE model trained from scratch, and a browser inspector that reads the checkpoint locally.
Live demo Source Ollama Hugging Face Dataset Video
Executive summary
Why it exists
I wanted to look inside a small model, not only run it. That meant keeping two things separate: a perfect-play controller for the game, and a trained model that reads raw move histories.
Outcome
A live demo, a published Ollama model, and a Hugging Face release in F16, Q6_K and Q4_K_M GGUF. Each GGUF file had zero errors in llama.cpp on 294,777 valid and 683,226 generated invalid move histories. A routing analysis across every history showed which experts favored which kinds of positions, as a description rather than a causal test.
My role
Designer, director, maintainer and publisher. AI coding assistant: GPT-5.6-terra (medium reasoning) through Hermes, with Honcho for project memory.
Technical overview
How it works
A 3x3 simulation with 0-, 1-, and 2-player modes. A deterministic minimax policy in browser JavaScript is the perfect-play oracle. A separate compact causal model uses the Qwen2-MoE architecture, so it works with Ollama and llama.cpp without a custom fork. The browser decodes the packed FP32, F16, Q6_K, or Q4_K_M release artifact locally and computes a forward trace from its weights.
Hard parts
Keeping real and fictional parts apart. The site never calls Ollama; its board moves come from the local oracle, and the visualizer is a local trace. The map, DEFCON strip, trajectories, and cipher are labeled visual fiction. My September post on the project describes the visualization, training, and tokenizer problems I hit, and how they led to the smaller Crystal-9.
How it was made
Palace-9 started as a model trained in the web browser, ten years after my first neural network experiment taught 200 small networks tic-tac-toe; then I moved training to PyTorch on my GPU. Along the way I learned to publish models and make them run on Ollama. With no control over the token set, the first model was very large, so I worked it down to the smallest vocabulary possible. The architecture follows Qwen2-MoE only so that Ollama knows how to run its mixture of experts; it is not a Qwen model.
The website has a Cold War theme, after Crystal Palace, the NORAD call sign in WarGames where an AI plays against itself. The computer plays itself over maps of the USA and USSR, missiles launch with each player's mark and speed up as the game goes on, and a launch code is worked out digit by digit, but the last digit never locks in.
The browser model went through three designs: a dense network under an illustrative expert diagram, then a real trainable soft-routed mixture of experts, then the separately trained Qwen2-MoE model that became the release. Training ran in JavaScript workers, then custom CUDA compiled with nvcc, then PyTorch with AdamW on an RTX 3080 Ti. The release model has 648,528 parameters; one last history error was fixed with 24 focused training updates. Conversion used llama.cpp's GGUF tools, and Q8_0 wasn't possible because the layer widths don't fit its 32-value blocks, so Q6_K took its place.