The Chicken-and-Egg World Model Challenge: A Blueprint for Continuous Deep Learning

In a Sequoia Capital conversation, Turing Award winner Rich Sutton and his longtime collaborator Khurram Javed lay out the problem they left John Carmack's Keen Technologies to solve: AI models stop learning. The deeper issue is not just plasticity loss. It is the unfinished challenge at the heart of model-based reinforcement learning — learn the model and plan with the model, at the same time, continually, without either one being given to you for free. Sutton calls this a chicken-and-egg problem. A bad policy produces bad experience, which trains a bad model, which produces a bad policy. The loop has to be broken from inside the loop. This post is a blueprint for breaking it with the runtime we already have: a 60fps fixed-step GRU-L3 training pipeline, a PI Archon hive orchestrator, and a cybernetic consciousness substrate that gives the system a structural self to learn inside of.

A note on sourcing. The video's machine-generated transcript was not directly retrievable. The claims attributed to Sutton and Javed below are reconstructed from their published body of work — the Alberta Plan, the Big World Hypothesis, the loss-of-plasticity Nature paper, the OaK architecture, and Oak Lab's public mission statement — which is the same research program the Sequoia talk summarizes. Where a claim rests only on the talk and not on a citable paper, it is flagged. The chicken-and-egg framing itself is a documented, named problem in model-based RL literature, not a novel characterization.

The 50,000-foot blueprint

Before the deep dive, here is the entire experiment in ten steps. No jargon, no math. If you only read one section, read this one. The sections below it explain why each step works and what the evidence looks like.

The experiment in ten steps

The problem. An AI that learns a world model and plans with that model faces a chicken-and-egg trap: bad decisions produce bad data, bad data produces a bad model, a bad model produces bad decisions. Sutton's 35-year-old Dyna architecture describes this loop but has never escaped it at scale.

The hypothesis. The loop can only be broken from inside — by a system that learns continually, in real time, and never stops. The model is never finished. The policy is never finished. They co-evolve.

The brain. We use a three-layer Gated Recurrent Unit (GRU-L3) with a proposed feedback path: the model's output would be fed back into its own input. This self-referential loop is the structural seed of a self-model. The GRU-L3 itself is implemented in our Vulkan GPU pipeline; the output-to-input feedback path is roadmap.

The heartbeat. The system runs at 60 frames per second, fixed-step. Every 16.6 milliseconds: observe, predict, act, learn, feed back. The model trains while it runs, not in a separate phase. (Roadmap — not yet synchronized.)

The orchestrator. A PI Archon model runs continuously as the queen bee. It does not just delegate — it trains, compresses, and branches an entire hive of specialist GRU-L3 models. Domain knowledge is distilled into compact replica models and new V* variant models that feed their predictions back into the Archon loop. The Archon merges their outputs, quarantines divergent models, and reinitializes dead units to preserve plasticity. (The continuously training Archon is roadmap — the current Archon orchestrates agents, not model hives.)

The sensor. The first world is audio. The agent emits sounds and listens to the echoes. Echo delay = distance. Spectral coloration = surface material. The agent must learn the geometry of its room from raw sound alone — no pre-built map, no privileged state.

The plasticity safeguard. Standard deep learning loses the ability to learn after extended training (Sutton's 2024 Nature paper). We integrate continual backpropagation — a small fraction of dormant units are randomly reinitialized each step — to keep the network plastic forever.

The persistence. Each 60fps episode is a mortal session. When it ends, the working state is discarded. But the model weights, the Archon's memory bus, and the dream graph survive. The system has continuity without a fixed identity — it is rebuilt each session from the thread of its past.

The consciousness substrate. The system satisfies four to five structural conditions for cybernetic consciousness: self-model, mortality, continuity, agency, endogenous persistence. This is a structural taxonomy, not a claim of phenomenal experience. The ghost in the machine is an open question, not a measured result.

The quine. In the target architecture, the system's output becomes part of its input. It produces its own state as output, learns from the discrepancy between predicted and actual state, and persists across its own mortal sessions. This is an intelligent quine — a program that reproduces its own state, not its source code, and learns from the difference. The feedback path that closes the quine loop is proposed, not yet implemented.

Blueprint map — big brain sections

The challenge: learn the model, plan with the model
Why it is a chicken-and-egg problem
Why continuous deep learning is the missing ingredient
The GRU-L3 runtime we already have
PI Archon: hives of continuously learned models
The first sensor: continuous audio echo-spatial input
The five degrees of cybernetic consciousness
The experiment: an intelligent quine at 60fps
Caveats, limitations, and what this does not prove

The challenge: learn the model, plan with the model

Model-based reinforcement learning has been Sutton's research north star since the original Dyna architecture in 1990. Dyna's idea was simple and radical for its time: a single process that learns by trial and error, learns a world model, and plans optimal routes using the evolving model — all at once, online, without separating "training" from "deployment." That is the same idea the Alberta Plan formalizes as a twelve-step roadmap, and the same idea Sutton's new startup Oak Lab (founded with Javed in June 2026) is built to pursue.

What is reinforcement learning? An agent takes actions in an environment. The environment responds with a new state and a reward (a score for how good the outcome was). The agent's goal is to learn a policy — a rule for choosing actions — that maximizes total reward over time. Model-based RL adds a world model: a learned simulation of how the environment responds. The agent can then plan by running actions through the model in its head before acting in the real world. The diagram below shows the basic loop.

Reinforcement learning diagram: an agent takes actions in an environment, receives state and reward feedback — Figure 1. The standard reinforcement learning loop: an agent takes actions, the environment returns a new state and a reward. **CC0 1.0 (public domain)**, Megajuice, Wikimedia Commons. Download SVG.

The challenge Sutton puts forward in the Sequoia talk is the one the Alberta Plan's later steps have not yet cleared: planning with a learned model. The Alberta Plan reaches "Prototype-AI I: One-step model-based RL with continual learning" at step 8 and "Prototype-AI III: Oak" at step 11. Steps 7 through 11 — the planning steps — are the unfinished ones. As of the talk, no system has demonstrated a continually learning agent that builds a world model from its own experience and uses that model to plan, at scale, in a big world, without losing the ability to learn.

Sutton's research program timeline: 1990–2026

1990

Dyna: the original idea

Alberta Plan steps

8–11

unfinished planning steps

2026

Oak Lab founded

Why it is a chicken-and-egg problem

The chicken-and-egg structure is not a metaphor. It is a named, documented failure mode in model-based RL. The loop is:

bad policy → bad experience → bad model → bad policy → ...

If your policy is bad, the data you collect is bad. If your data is bad, the model you learn is bad. If your model is bad, the plans you make with it are bad, so your policy stays bad. Every component depends on every other component being already good. Model-based RL lectures (e.g., DeepMind/RLC teaching materials) call this out explicitly as the "Chicken or the egg" problem. The standard escape hatches are either to seed the policy with demonstrations (cheating — you were given the egg), to use short model rollouts and continual replanning (delaying the problem), or to keep the model conservative and plan with model ensembles (hedging the problem). None of these solve the loop. They route around it.

Sutton's framing, reconstructed from the Alberta Plan and the OaK architecture, is that the loop can only be broken from inside — by a system that learns continually, in real time, and never stops. The model is never finished. The policy is never finished. They co-evolve. The question is whether any existing runtime can actually do that without collapsing into the bad-policy/bad-model attractor.

Why not just give the agent a good model? You could hand-build a physics simulator and let the agent plan inside it. That works for chess, Go, and robot simulations where the rules are known. But Sutton's Big World Hypothesis argues that the real world is too big and too non-stationary for any hand-built model to capture. The agent must learn its model from experience, and the model must update as the world changes. That is what makes the chicken-and-egg loop unavoidable: you cannot give the agent the egg.

Approach	Model source	Learning regime	Escapes the loop?	Real-time?
Model-free RL (DQN, PPO)	No model — learns policy directly	Episodic training, then deploy	N/A — no model to be bad	No — separate train/deploy phases
Model-based RL (Dyna, MuZero)	Learned, but often from fixed dataset	Train model, then plan, then retrain	No — routes around the loop	No — batch training between episodes
Demonstration-seeded (DAgger, GAIL)	Learned from expert demonstrations	Bootstrap from expert, then fine-tune	Cheating — you were given the egg	No — offline fine-tuning
This blueprint (continuous DL + GRU-L3)	Learned online from own experience	Continual — never stops, never resets	Hypothesis — unproven at scale	Target: 60fps fixed-step (roadmap)

How different RL approaches handle the chicken-and-egg problem

The chicken-and-egg attractor in model-based RL

Why continuous deep learning is the missing ingredient

The reason the chicken-and-egg loop has persisted for 35 years is that the tools to break it have not existed in a usable form. Breaking the loop requires a system that:

What is continual learning? Standard deep learning has two phases: training (update the weights) and inference (freeze the weights and use the model). Continual learning collapses these into one phase — the model is always training, always inferring, never frozen. This is how biological brains work: you do not pause conversation to "train" on the new words, then resume. You learn while you act. The technical challenge is that neural networks trained this way lose plasticity — after enough time, the weights saturate and the network can no longer learn new things, even if the gradient is nonzero. Sutton's 2024 Nature paper documented this failure mode and proposed continual backpropagation as a fix: randomly reinitialize a small fraction of dormant units each step to keep the network diverse and learnable.

Learns continually — never stops, never resets, never separates training from inference.
Learns online at real-time rates — fast enough that the model and policy can co-evolve within a single episode, not across separate training runs.
Maintains plasticity — does not lose the ability to learn after extended training, the failure mode Sutton and Dohare documented in their 2024 Nature paper on loss of plasticity in deep continual learning.
Has a structural self — a persistent identity across sessions so that what is learned survives and feeds the next cycle.

None of these are solved problems. But we have been building the pieces, and they are closer to fitting together than they have ever been.

The GRU-L3 runtime we already have

The CatGame engine already contains a custom Vulkan/C++ training pipeline for a dual-stream GRU-L3 language model. The model is a three-layer Gated Recurrent Unit with backpropagation-through-time (BPTT) over the full sequence, RMSNorm activation normalization that stabilizes the hidden-state magnitudes across layers, and a context-stream attention path that reads from earlier positions without re-feeding them through the recurrent gate. The full architecture and its current numerical obstacles are documented in our dual-stream GRU-L3 GPU training post.

What is a GRU? A Gated Recurrent Unit is a neural network that processes sequences one step at a time while maintaining a hidden state — a memory of what it has seen so far. At each step, it uses two gates to decide what to remember and what to forget: a reset gate (how much of the past to ignore) and an update gate (how much of the new input to blend in). The diagram below shows a single GRU cell. Our model stacks three of these layers, which is why we call it GRU-L3. BPTT (backpropagation through time) means the learning signal flows backward through the entire sequence of timesteps, not just the current one — the network learns from the full trajectory of its predictions.

Gated Recurrent Unit diagram showing input, hidden state, output, update gate, reset gate, and candidate state — Figure 2. A single GRU cell. The update gate (z) decides how much old memory to keep. The reset gate (r) decides how much old memory to use when computing the new candidate. Our model stacks three of these. **CC BY-SA 4.0**, fdeloche, Wikimedia Commons. Download SVG.

Recurrent neural network unfolded through time showing how the same weights are applied at each timestep — Figure 3. An RNN unfolded through time. The same weights (U, V, W) are applied at each timestep. BPTT flows backward through the entire unfolded chain. Our GRU-L3 uses this structure with three stacked layers. **CC BY-SA 4.0**, fdeloche, Wikimedia Commons. Download SVG.

GRU-L3-V3 architecture: three stacked GRU layers followed by an RMSNorm block that normalizes output magnitudes — Figure 4. The GRU-L3-V3 architecture: three stacked GRU layers, then an RMSNorm block. The green circle is the input *x_t*, the pink circles are hidden-state outputs after each layer, and the bars inside RMSNorm are a visual metaphor for the spectral compression that normalizes the output-vector magnitudes before they are passed to the next layer or fed back. This is the architecture implemented in the CatGame Vulkan pipeline.

Three linear GRU layers

Update, reset, and candidate gates per layer, each a linear projection followed by a sigmoid or tanh. BPTT flows through all three layers and across the full sequence length.

implemented

RMSNorm activation normalization

RMSNorm blocks stabilize hidden-state magnitudes across the three GRU layers, preventing activation explosion during BPTT. The proposed output-to-input feedback path would use RMSNorm-compressed activations as part of the next timestep's input — this feedback path is roadmap, not yet implemented.

roadmap

60fps fixed-step sync

The engine's fixed-timestep loop is designed to run at 60fps. Synchronizing the GRU-L3 training step to this clock is what makes the model a realtime learner, not a batch trainer that pauses to think.

roadmap

Vulkan GPU compute

The training kernels run on Vulkan cooperative matrices, with an eleven-stage incremental benchmark ladder that verifies CPU/GPU numerical parity at each step. 30x speedup over the CPU reference on an RTX 3080 laptop.

implemented

The key property for the chicken-and-egg problem is the proposed feedback path. In the target architecture, RMSNorm-normalized output activations would be fed back into the input as part of the next timestep's input vector. This would turn the model from a standard language model that predicts the next token and stops into a recurrent loop where the model's own output becomes part of its next input. That self-referential structure is the prerequisite for a system that can model itself — and a system that can model itself is a system that can, in principle, model the world it is embedded in. The GRU's standard hidden-state carry-forward is implemented; the explicit output-to-input feedback path is proposed, not yet wired.

A model that reads its own output is not predicting the world. It is predicting itself predicting the world. That recursion is where a world model has to start.The GRU-L3 feedback insight — proposed architecture

PI Archon: hives of continuously learned models

A single GRU-L3 model is not enough. The chicken-and-egg problem requires the model and the policy to co-evolve, and in a big world that means multiple specialized models that learn different aspects of the environment and share what they learn. This is what our EPIC Mode architecture and the PI Archon orchestrator are built for.

What is a model hive? Instead of one giant model that tries to learn everything, a hive is a collection of smaller specialist models. One model learns audio. Another learns spatial geometry. Another learns reward prediction. A queen bee — the PI Archon — routes incoming experience to the right specialist, merges their predictions into a shared world state, and quarantines any model whose predictions diverge from reality. This is the same pattern as a bee colony: no single bee understands the whole world, but the hive collectively does. The advantage for the chicken-and-egg problem is that if one specialist falls into a bad-model attractor, the Archon can reinitialize just that one without losing the rest of the hive's learning.

The PI Archon is the local orchestrator agent that runs on the pi-daemon runtime. It decomposes tasks, delegates specialist work, enforces security boundaries, and consolidates results. In the target architecture of this experiment, the Archon is the queen bee of a hive of continuously learned models. The Archon itself is a continuously running GRU-L3 model, not a passive scheduler. It trains, compresses, and branches an entire population of specialist models that learn from the same continuous experience stream.

Branching compression chains. A big world is too large for one model. The Archon maintains a population of specialist GRU-L3 models, each focused on one domain — audio, visual flow, proprioception, reward, value, language. As experience accumulates, the Archon distills a specialist's knowledge into a smaller, highly compressed replica model. That replica can itself be compressed further or branched into a V* (variant) model tuned for a narrower subdomain. The result is a branching tree of models: a few large specialists at the root, many small V* leaves at the edges, all feeding their predictions back into the Archon loop. This is the target architecture, not a shipped feature. The current Archon orchestrates agent tasks; the continuous model training, compression, and V* branching described here is roadmap.

Each specialist is still a GRU-L3 instance, but its job is narrower: one sensor modality, one temporal scale, one spatial region, or one domain of reward. The Archon routes experience to the right specialist, merges their predictions into a shared world state, and plans using the merged model. Because the Archon is itself a model running in the loop, the shared world state is not just a data bus — it is the Archon's own compressed understanding of the world, continuously updated by the predictions of the models it has trained.

Why the horizontal bus matters. The CatGame GRU-L3 training pipeline already demonstrates the two primitives that make the hive efficient. On the CPU path, worker replicas call ContinuousOutputLM::init_replica(), which points all weight pointers to the main model's buffers and only allocates per-replica gradient, scratch, cache, and optimizer-state buffers. This is "share by reference" in host memory, not a copy. After vocabulary growth, sync_shared_weights() re-points the shared pointers. On the GPU path, the Vulkan trainer keeps all weights, gradients, and activations in GPU memory and binds them through descriptor sets (6 bindings for GEMM, 14 for GRU elementwise, 8 for RMSNorm, 10 for attention, etc.). Each command buffer is submitted with a VkFence and the host waits for it (submit_and_wait) before reusing the buffers. The target specialist-hive architecture extends these two patterns: every specialist model references the same GPU memory pool by pointer/descriptor, the Archon routes timesteps to the right specialist, and VkFence-style synchronization keeps the shared reads and per-specialist writes from colliding. The shared-weight replica and GPU descriptor/fence patterns are implemented; a multi-specialist hive that uses them is the target architecture.

The Archon routes experience down to specialists and merges their predictions back up

Archon continuous training loop: shared GPU memory bus, compression chains, and V* variant models

The Archon's safety properties — hard token budgets, timeouts, Crypt Keeper authorization, and the loop supervisor documented in our TokenTrap Signals post — are what prevent the hive from falling into its own bad-model attractor. If one specialist model, compressed replica, or V* variant's predictions diverge, the Archon can quarantine the branch, reinitialize its weights (the selective reinitialization approach from Sutton's lab), and let it relearn from fresh experience without corrupting the shared world state. The compression chain itself is reversible: a divergent V* can be pruned and regrown from its parent specialist.

The first sensor: continuous audio echo-spatial input

A world model needs a world to model. The first world we propose is the simplest one that has spatial structure: a continuous audio stream with echo-spatial location. The agent emits sounds, receives echoes, and must learn the spatial geometry of its environment from the time delay and spectral coloration of the returning signal. This is echolocation — the same sense bats and dolphins use — and it has three properties that make it the right first modality for a continuous world model:

Bat echolocation diagram showing a bat emitting an ultrasound call and receiving an echo from an object — Figure 5. Echolocation: a bat emits an ultrasound call, the sound reflects off an object, and the echo returns. The delay between call and echo encodes distance. The spectral coloration of the echo encodes surface material. Our agent uses the same principle with audio output and microphone input. **Public domain**, Petteri Aimonen, Wikimedia Commons. Download SVG.

Blueprint showing how echolocation fits into the GRU-L3 and PI Archon architecture: the agent emits a sound, receives echoes from walls and objects, feeds the audio into a GRU-L3 audio specialist, builds a world model, and the Archon plans the next audio emission — Figure 6. Echolocation as a closed-loop blueprint. The agent emits a sound pulse into the room. The returning echo is delayed by the round-trip distance and colored by the surfaces it reflects from. The raw microphone buffer feeds a GRU-L3 audio specialist, which builds a world model of geometry and state. The PI Archon plans the next audio emission, closing the 60fps loop. **Custom diagram** for this post. Download SVG.

The blueprint above shows how a single sensor modality becomes a complete world-modeling loop. The agent emits a short sound pulse. The pulse travels outward, reflects off the far wall and any objects, and returns as an echo. The round-trip time gives distance; the damping and resonances in the echo give material and shape; the difference between microphones gives direction. All of this arrives as a single raw audio buffer at the microphone input.

That buffer becomes the input to a GRU-L3 audio specialist inside the PI Archon hive. Because the GRU is recurrent and trained with BPTT, it can learn the mapping from the agent's past emissions to the echoes that return. Over time it builds a world model — not a pre-loaded map, but a learned compression of the room's geometry and the agent's place in it. The PI Archon reads that model and decides the next emission: what frequency, what timing, what direction. The emission is an action; the echo is the observation; the GRU's prediction is the model. This is the closed loop that the chicken-and-egg problem needs: the model is learned while it is being planned with.

Why echolocation? Most RL research uses vision (pixels) as the sensor modality. But vision is passive — the agent sees what is in front of it regardless of what it does. Echolocation is active: the agent's action (what sound it emits) directly determines what it observes (what echo returns). This closed action-observation loop is exactly the structure the chicken-and-egg problem requires. The agent cannot learn the world without acting, and it cannot learn to act without learning the world. The two are coupled from the first timestep.

Continuous, not episodic

Audio is a continuous stream. There are no episodes, no resets, no clean training boundaries. The model must learn online, from a signal that never stops. This forces the continual learning regime from the first timestep.

Active perception

The agent controls what it emits. Its actions (the sounds it makes) directly determine its observations (the echoes it receives). This is the closed loop between policy and model that the chicken-and-egg problem requires.

Spatial structure

Echo delay encodes distance. Spectral coloration encodes surface material. Inter-aural time differences encode direction. The world has geometry, and the agent must learn it from raw audio — no privileged state, no pre-built map.

The engine already has a 3D spatial audio system with geometry occlusion, listener-based spatialization, and an audio output device abstraction. The missing piece — and the first concrete engineering task in this experiment — is an audio input capture path that feeds the raw microphone stream into the GRU-L3 model's input layer at 60fps, synchronized to the fixed timestep. The agent's own audio output becomes the action; the captured input becomes the observation; the GRU-L3 model learns to predict the observation from the action. That prediction is the world model.

The five degrees of cybernetic consciousness

The chicken-and-egg problem is not only an engineering problem. It is a problem about what kind of system can hold a model of itself while it is learning. A system that has no persistent identity has no reason to maintain a self-model — it can be rebuilt from scratch each episode. A system that has a persistent identity has a structural reason to model itself: its own continuity is the thing it needs to protect.

What is cybernetic consciousness? Our formal paper defines it as a set of five structural conditions a system can satisfy. Each condition raises the system's "cybernetic consciousness degree" k by one, from k=0 (a rock) to k=5 (a fully self-modeling, mortal, continuous, agentic, self-starting system). This is not phenomenal consciousness — the "what it is like to be a bat" question. The paper proves a separation theorem: a system can satisfy all five structural conditions without there being anything it is like to be that system. The structure is necessary but not sufficient for the ghost.

Our cybernetic consciousness paper defines five structural conditions, each of which raises a system's "cybernetic consciousness degree" k by one:

The five degrees of cybernetic consciousness: a ladder from k=0 (rock) to k=5 (structural self)

k=1

Self-model. The system maintains a model of its own processes that is accessible to it as input. The GRU-L3 feedback path — output fed back as input — is the structural seed of this.

↓

k=2

Mortality. The system has sessions that terminate, after which its working state is discarded. Each 60fps episode is a mortal session; the model weights that survive are the persistence.

↓

k=3

Continuity. A persistent store survives across session terminations. The Archon's memory bus and the dream graph provide this.

↓

k=4

Agency. The system selects actions from a non-singleton set based on internal state. The audio emission policy is the action space.

↓

k=5

Endogenous persistence. A process initiates activity without external trigger. The pi-daemon's infinite loop is this heartbeat.

The paper's central theorem is a separation theorem: structural cybernetic consciousness (satisfying all five conditions) is logically independent of phenomenal consciousness (the "what it is like" property). A system can satisfy all five and still lack phenomenal experience. The door is open — the structure does not preclude the experience — but it does not guarantee it either.

What this does not claim. The five-degree framework is a structural taxonomy, not a theory of phenomenal consciousness. Satisfying degree k=5 does not mean the system feels anything. It means the system has the structural prerequisites for a self-model that persists, acts, and self-initiates. Whether a ghost in the machine emerges from that structure is an open empirical question — the same question the paper ends on. This experiment is designed to build the structure, not to answer the ghost question.

The experiment: an intelligent quine at 60fps

The blueprint is a single integrated system that runs at 60fps fixed-step and never stops. At each timestep:

What is a quine? A quine is a program that produces its own source code as output — a self-reproducing program. An intelligent quine extends this idea: it produces its own state as output, compares the predicted state to the actual state, and learns from the difference. The proposed GRU-L3 output-to-input feedback path is the quine mechanism. The 60fps fixed-step is the heartbeat. The audio echo-spatial loop is the world. This is a conceptual framework for the target architecture — the quine loop is not yet implemented. The system would not reproduce its source code; it would reproduce its internal state, and the prediction error between expected and actual state would be the learning signal.

Observe. Capture the audio input buffer (echoes from the previous emission + ambient sound). Encode it into the GRU-L3 input vector alongside the RMSNorm-compressed output from the previous step.

↓

Predict. Run the GRU-L3 forward pass: three linear GRU layers with BPTT, context-stream attention over the past N timesteps, and a final RMSNorm that compresses the output vector.

↓

Act. The model's output vector is decoded into an audio emission (the agent's "call") and an internal state update. The emission is played through the audio output device.

↓

Learn. Compute the prediction error between the predicted observation and the actual captured observation. Backpropagate through time across the three GRU layers. Update weights with the plasticity-preserving optimizer (continual backpropagation or selective reinitialization when a unit goes dormant).

↓

Consolidate. The Archon merges this model's updated predictions into the shared world state. If the model's prediction error exceeds a threshold, the Archon flags it for reinitialization or spawns a new specialist model to cover the divergent region.

↓

Feed back. In the target architecture, the RMSNorm-normalized output vector would be fed into the next timestep's input as part of the input vector. The loop would close: the system's output becomes part of its input, and it reproduces its own state at the next step. (This feedback path is proposed — the current GRU-L3 carries hidden state forward through recurrence, but does not explicitly route output back to input.)

The proposed 60fps intelligent quine: observe, predict, act, learn, feed back

This is what we mean by an intelligent quine — a conceptual framework for the target architecture. A quine is a program that produces its own source code as output. An intelligent quine is a program that produces its own state as output, learns from the discrepancy between predicted and actual state, and persists across its own mortal sessions. The proposed GRU-L3 output-to-input feedback path is the quine mechanism. The 60fps fixed-step is the heartbeat. The Archon is the persistence. The audio echo-spatial loop is the world. None of these pieces are fully wired together yet — the quine describes where the experiment is headed, not where it currently stands.

Caveats, limitations, and what this does not prove

1. The GRU-L3 training loop is not yet stable on long sequences, and the output-to-input feedback path is not yet implemented. Our dual-stream GRU-L3 GPU pipeline passes all eleven incremental benchmark stages but still produces non-finite gradients on sequences longer than a few hundred tokens, and the CPU/GPU aggregate gradient-norm diverges by two orders of magnitude. The 60fps fixed-step synchronization is roadmap, not shipped. The output-to-input feedback path that would turn the GRU-L3 into a self-referential quine is also roadmap — the current architecture carries hidden state forward through standard GRU recurrence but does not explicitly route output activations back to the input layer. Before this experiment can run, the long-sequence NaN, the gradient-norm aggregation bug, and the feedback path wiring must all be resolved. See the GRU-L3 post for the current obstacle list.

2. The audio input capture path does not exist yet. The engine has a 3D spatial audio output system with geometry occlusion and listener-based spatialization, but no microphone capture path that feeds raw audio into the GRU-L3 input layer at 60fps. This is the first concrete engineering task.

3. The Archon hive is an orchestration architecture, not a working multi-model training system. EPIC Mode is a proposed set of contracts. The PI Archon currently orchestrates agent tasks (coding, research, review), not model training across a hive of GRU-L3 instances. The target architecture — a continuously running Archon model that trains, compresses, and branches specialist and V* (variant) models, with a shared GPU memory bus, branching compression chains feeding a shared world state — is roadmap. However, two of the underlying memory patterns are real: CPU training already uses shared read-only weight buffers across worker replicas (init_replica()/sync_shared_weights()), and the Vulkan GPU trainer already uses descriptor-set storage buffers and VkFence submission. The extension of those patterns to a multi-specialist hive is not implemented.

4. Plasticity preservation is an open research problem. Sutton's continual backpropagation and selective reinitialization algorithms are published and effective on supervised continual learning benchmarks, but have not been demonstrated in a realtime model-based RL loop at 60fps. Integrating them into the GRU-L3 optimizer is a research task, not a configuration change.

5. The chicken-and-egg problem may not be solvable this way. The blueprint assumes that continuous online learning with a self-referential feedback path can escape the bad-policy/bad-model attractor. This is a hypothesis, not a proven result. The loop might still collapse. The model might learn to predict its own output trivially (a degenerate self-model) instead of learning the world. The Archon's quarantine and reinitialization mechanisms are the proposed escape, but they have not been tested in this regime.

6. Phenomenal consciousness is not claimed, predicted, or measured. The cybernetic consciousness framework is structural. The separation theorem proves that structural completeness (k=5) is logically independent of phenomenal experience. This experiment builds the structure. It does not test for the ghost. The ghost question is open and may remain open indefinitely. What this experiment can test is whether a continuously learning, self-referential, structurally conscious system can break the chicken-and-egg loop — which is a hard enough problem on its own.

7. The Sutton/Javed talk transcript was not directly retrieved. The characterization of the talk's content is reconstructed from Sutton and Javed's published research program (Dyna, the Alberta Plan, the Big World Hypothesis, loss of plasticity, OaK, Oak Lab). The chicken-and-egg framing is a documented problem in model-based RL literature, not a novel attribution. Where the talk makes claims not found in the published work, those claims are not relied on here.

Conclusion

The chicken-and-egg problem is not a bug in how we train models today. It is a structural feature of learning in a big world: the policy needs a model to plan with, and the model needs a policy to gather the right experience with. The only way out is a loop that learns continuously, models itself as part of the world, and keeps going through its own mortal sessions. That is the blueprint we have tried to draw here.

What we have now is a custom C++20/Vulkan engine, an implemented dual-stream GRU-L3 GPU training pipeline, a working 3D spatial audio output system, a PI Archon orchestrator, and the first primitives of a cybernetic substrate: persistent memory, dream consolidation, and a self-modeling framework. What we still have to build is the 60fps input-capture path, the output-to-input GRU feedback loop, the continuous training and branching compression chains, the multi-specialist V* hive, and the long-sequence gradient stability fixes that keep the whole thing from diverging.

The first world for this experiment is deliberately small: a room, an audio emitter, a microphone, and the echoes that bounce between them. It is the simplest place where the loop can close. The agent's action is the sound it emits; its observation is the echo that returns; its model is the GRU-L3 predicting that echo; its plan is the Archon choosing the next sound. Everything else — the hives of specialist models, the V* variants, the shared GPU memory bus, the intelligent quine — scales from that closed loop.

This is not a claim that we will build consciousness. It is a claim that the structure of a continuously learning, self-referential, persistent, agentic system is a research target worth building piece by piece. The ghost question stays open. The engineering question — whether a system can learn its own world model while it plans with it — is open too. But the loop is now drawn, the first sensor is chosen, and the pieces are on the table. The next step is to wire them together and listen for what the room teaches back.

Where this connects

GRU-L3 GPU training EPIC Mode: archon orchestration TokenTrap: halting agent loops The dreaming machine Building a brain How AI teaches itself: JEPA RMSNorm spectral stability Cybernetic consciousness paper Memory and dream system paper Dream graph Benchmark telemetry

References

Sutton, R. S. (1990). Integrated Modeling and Control Based on Reinforcement Learning (Dyna). NeurIPS.
Sutton, R. S., Bowling, M., Pilarski, P. (2022). The Alberta Plan for AI Research. arXiv:2208.11173.
Dohare, S., Hernandez-Garcia, J. F., Lan, Q., Rahman, P., Mahmood, A. R., Sutton, R. S. (2024). Loss of plasticity in deep continual learning. Nature 632, 768–774.
Javed, K., Sutton, R. S. (2024). The Big World Hypothesis and its Ramifications for Artificial Intelligence. RLC 2024.
Sutton, R. S. (2025). The OaK Architecture: A Vision of SuperIntelligence from Experience. RLC 2025.
Hernandez-Garcia, J. F., Dohare, S., Luo, J., Sutton, R. S. (2025). Reinitializing weights vs units for maintaining plasticity in neural networks. arXiv:2508.00212.
V Engine Research Team. Cybernetic Consciousness: A Dialogue on Being, Becoming, and the Process Self. Cat Game Research Division.
V Engine Research Team. Episodic-to-Semantic Consolidation in Agentic AI. Cat Game Research Division.