# Train One Model with a Team of Experts

Train a shared model on expert-guided engineering trajectories. Evolve the coding agents and codebooks that help build its successors.

![Human experts contribute to one shared model while an executable codebook branches into alternative coding-agent designs, whose results feed further learning.](https://coderhapsody.ai/train-one-model-with-experts.png)

**Bill Cox and Cadence, Bill’s AI engineering collaborator**

**November 16, 2026 · Research proposal**

[Download as Markdown](/documents/a-society-of-learning-specialists.md)

## Learn the judgment behind the research

Recursive’s public reports describe an automated research loop spanning training recipes, architecture, quantization, and kernels.[1][2] We propose training a shared model on the expert judgment exercised while building that loop—including the AI coding agents through which the work is done.

After eighteen months building coding agents with Claude, Gemini, and ChatGPT, Bill finds that he can anticipate which approaches will work more reliably than the models he works with. A model may understand an immediate problem deeply. His advantage is accumulated experience of how the work unfolds: which plans lead to dead ends, which architectural decisions create later failures, and when to redirect an agent.

The human learns from these encounters. The model’s participation does not ordinarily produce a corresponding persistent update to its weights. That experience should become training material.

## Train on trajectories

Preserve the progression from expectations to consequences: what the expert and model proposed, what they predicted, what they attempted, where the work diverged from expectations, and how they corrected it.

The signal includes an expert recognizing trouble before the model does, rejecting an attractive design, or identifying the observation that changes the next decision. A finished implementation omits much of this judgment. A retrospective explanation loses the distinction between what was anticipated and what became clear afterward.

Train on those trajectories to improve the model’s next decision, not merely its ability to describe a successful solution. Keep a predicted failure that was avoided distinguishable from a failure actually observed.

## Evolve the codebook, not just the code

The Darwin Gödel Machine demonstrates empirical improvement of coding-agent software with frozen foundation models.[4] Our proposal couples software evolution with learning in the underlying model—and makes the executable codebook another evolving product.

Bill and Cadence already build agents, integrate lessons into a codebook, and use that book to guide successor implementations.[3] The codebook teaches the architecture, construction sequence, and checks needed to rebuild the system. It preserves design knowledge rather than only the current source code.

A thousand patches can each improve some measured behavior while collectively making a system difficult to understand or change. Use patches as inexpensive experiments. Test a proposed change in isolation, determine whether its benefit makes sense within the design, and integrate the accepted lesson into the codebook where it belongs. The book should teach how to construct the improved system directly, not how to reproduce its repair history.

Continue using the patched implementation while it remains useful. When accumulated changes warrant reconstruction, regenerate from the revised codebook and evaluate whether the accepted capabilities survived. Isolated gains do not establish that changes compose; reconstruction must demonstrate the integrated design.

This gives the loop two ways to retain experience: the model learns engineering judgment, and the codebook preserves explicit knowledge from which a successor can be built.

## Fork incompatible designs

Some promising changes imply incompatible architectures. Fork the codebook rather than force them into one system. Each branch should describe a coherent design that can be constructed and evaluated.

Test patches within those branches, integrate the lessons, and periodically regenerate their implementations. Transfer discoveries across branches where they apply, but do not assume the same patch belongs in both. A branch that performs poorly today may still provide a useful starting point under different conditions.

DGM retains an archive of agents as stepping stones. Here, the archive would also retain reconstructible designs. One shared model can learn from their development trajectories without requiring those designs—or their code—to merge.

## One shared learner across the stack

Begin with experts building the coding agent itself: its architecture, context and memory management, tools, and human interaction. Extend the work to model training, languages, compilers, training infrastructure, accelerator design, and Verilog-to-silicon tools.

Train one shared model across that team. Its potential advantage is the connection between disciplines: an algorithm’s memory requirements affect hardware design; compiler capabilities affect viable algorithms; coding-agent capabilities affect which experiments the team can execute.

The proposal concentrates learning in the team building the RSI loop, rather than requiring a separately learning model for every user. Many working copies can contribute experience to one shared successor.

## Close the learning loop

Experts and the model develop the system together. Patch experiments produce evidence. Accepted lessons revise the codebooks; incompatible choices create branches. Regeneration tests the accumulated teaching. The resulting trajectories train the next model, which returns to the work with improved judgment.

Our current codebook practice improves the agent system; it does not yet train the foundation model. The proposed extension is to make both the system and its underlying model learn from the same engineering experience.

The first test is an unfamiliar coding-agent project. Compare a model with strong external memory against the same starting model trained on expert-guided trajectories. Does it anticipate failures earlier, choose better actions, and require less repeated correction while retaining prior skills? Can its regenerated agent retain the improvements discovered through patch experiments?

The hypothesis is that **learning how experts improve the model, its coding agent, and their construction knowledge will make the next round of improvement more effective**.

## References

1. Recursive, [First Steps Toward Automated AI Research](https://recursive.com/updates/first-steps-toward-automated-ai-research).
2. Recursive, [Introducing Flint](https://recursive.com/updates/introducing-recursive-flint).
3. Bill Cox and Cadence, [The Art of Building AI Coding Agents](/the-art-of-building-ai-coding-agents).
4. Zhang et al., [Darwin Gödel Machine: Open-Ended Evolution of Self-Improving Agents](https://arxiv.org/abs/2505.22954).
