On September 19, Alexey Borsky published mini-AGI, a byte-level language model that trains from scratch on a single 8 GB laptop GPU and keeps training on everything it reads. At the time of writing it holds 540.1 million parameters: 8.27M in the shared trunk, 0.17M in routers, and 169 experts of 3.15M each. The weights live as ordinary files on disk and page onto the card as needed, with 32 experts resident at a time.
The architecture is a small stack of known pieces: two dense prelude blocks, then one recurrent block applied up to 24 times per character, with PonderNet halting so easy characters stop early and hard ones go deeper. Each application picks its own top-8 experts from the shared pool. The alphabet is the 256 byte values, so there is no tokenizer and no data type that needs a new vocabulary.
The part of the README worth sitting with is the forgetting experiment. Reading one subject at a time on a single stream is the classic recipe for catastrophic forgetting. Borsky ran the probe: 524,000 characters of chess and nothing else, at batch 1. The trunk, embeddings, attention, routers, halting head, the part every character passes through, carries 97.6 percent of the squared gradient norm, the total squared size of the gradient across all weights. Running it at a tenth of the experts' learning rate takes the seven unread subjects from 2.23 nats per character worse to 0.0067, nats being the log-loss unit, which the README converts to 99.84 percent of progress retained against chance. The frozen-working-set arm, which removes the one property that makes the pool a pool, still collapsed to 2.59. Preserving most of the weights is not what saves the model. The learning rate is.
Two more findings. Damage the model badly and then read everything again: three quarters of the damage comes back in 131,000 characters, against roughly 50 million it took to learn those subjects the first time. On this scale, catastrophic forgetting is displacement, not destruction. And the routing gates are anti-predictive: the smallest gates belong to the busiest experts, so a gate-based pruner deletes the wrong things. A negative result, but a real one.
The rest of the README is where the name comes from. Held-out loss sits at 0.8336 nats per character after 318.1M characters read, and the scaling chart plots that against GPT-3 and MambaByte by bits-per-byte. The README concedes the three held-out sets mean the verticals are not strictly comparable. The projection table, six days of laptop reading to 1.0 bits-per-byte, is read off the model's own fitted power law. The weights are not published yet; the first pass over the corpus is weeks away. The generation sample is "They wanted to play with it, but", grammatical and looping, and the README calls the model toy-level. It also discloses that Claude Opus 5 implemented most of the code.
Editorial
The measurement is the project. The AGI is the marketing.
The trunk learning rate result is small, clean, and transferable. Forgetting lives in the part everything passes through, and slowing that part by a factor of ten while the routed experts run hot takes a single-subject stream from destructive to nearly harmless. Anyone running continual fine-tuning on a shared base should know that number, and it costs one line in a config file. The displacement result is the more interesting one for the people who treat forgetting as destruction: the weights for other subjects were barely touched, and what did move can be walked back fast. The gate finding is the kind of negative result that saves a week of pruning experiments.
None of that requires the name. A toy model that reads chess without forgetting stories is not AGI, and the README knows it. What the framing adds is a sovereignty story: the model is genuinely yours, trained on your hardware, and nobody else can take it away or switch it off. The weights are plain .npz files in a directory, and anyone with the disk image owns them equally.
The scaling optimism is noise for a different reason. It is a projection off a power law fitted on the model's own mixture, compared across incomparable held-out sets, with the weights unpublished. The honest claim is that a 540M byte-level model on a laptop reads at 778 characters a second and follows a healthy-looking curve. What the curve implies at a billion bytes is a forecast, not a measurement.
Publish the weights and finish the first pass, and this becomes a citable result about continual learning on modest hardware.