Yu Su, NeoCognition — "Intelligence + Continual Learning = Expertise" at AI Engineer World's Fair. The 14:59 mark is the critical "escape intelligence" point.
The critical point at 14:59
At the AI Engineer World's Fair, NeoCognition co-founder Yu Su drew a two-axis plane: raw intelligence on the x-axis and expertise on the y-axis. The two, he argued, are largely orthogonal. Frontier models are racing to the right on the x-axis—brilliant first-contact problem solvers that reset to zero every morning. Expertise is the ability to accumulate situated competence: pattern recognition, deep structure, conditional rules, and judgment. The magic is not more base model. It is the slope of the continual-learning curve.
"If a continual-learning algorithm is good enough, once raw intelligence crosses a threshold, you no longer need a stronger base model at all."
Su calls the threshold escape intelligence: the point past which a model can generate unbounded domain expertise from bounded intelligence. The segment at 14:59 is where that threshold appears on the graph. The rest of the talk is the argument that the modern AI industry has optimized the wrong variable and that the next frontier is scaling expertise, not just scaling intelligence.
From Su's plane to AAA game craft
We are treating Su's definitions as an operational framework for AAAc—our shorthand for AAA-competency. The goal is to replace vague "seniority" labels with measurable dimensions and a benchmark syllabus. The same model applies to game designers and programmers; the only difference is which acceptance tests are swapped in at each tier.
| Dimension | Intelligence (first-contact) | Expertise (accumulated) |
|---|---|---|
| Pattern recognition | Reads a bug report and guesses broadly | Instantly locates the most plausible failure points |
| Deep structure | Sees a meeting as a calendar problem | Sees it as constraint optimization over authority, urgency, and priority |
| Conditional rules | Applies a rule when told | Knows preconditions, exceptions, and when to bend them |
| Judgment / taste | Writes code that compiles | Knows when it is "good enough" and when the design is fair |
Transferring this to AAA game development, the dimensions become:
The AAAc competency ladder
We map the dimensions above to a six-tier ladder. Each tier has a role equivalent, a canonical AAA deliverable, and a set of acceptance tests. The K0-K2 band is about demonstrating raw problem-solving; K3-K5 is about accumulating and reusing domain expertise.
| Tier | Role equivalent | Designer acceptance | Programmer acceptance |
|---|---|---|---|
| K0 — Novice | Intern / first commit | Describe one mechanic in TOML | Write, compile, and run a static main() in C and C++ |
| K1 — Junior | Associate | Build a single guarded interaction | Write a CMakeLists.txt or compiler invocation from scratch for the code |
| K2 — Mid | Designer II / SDE II | Chain 3-5 puzzle nodes with no dead end | Implement a system with unit + integration tests |
| K3 — Senior | Senior designer | Own a vertical slice: narrative, puzzles, audio, UI | Refactor across engine boundaries without regression |
| K4 — Lead | Lead / Staff | Translate a mechanic from one AAA genre to another | Design an API that multiple genres can share |
| K5 — Principal | Principal / Director | Define the fairness doctrine for a new genre and prove it | Build a compiler / tooling that encodes the team's expertise |
The AAAc KPI stack
The KPIs are deliberately generic so they survive a genre swap. Every eval run produces a scorecard that can be compared across models, harnesses, and human runs.
| KPI | Definition | How measured |
|---|---|---|
| pass% | Fraction of eval tasks that reach a passing acceptance test | Binary per task; averaged over a held-out test set |
| turns (T) | Number of agent/human reasoning or tool-call turns until pass | Counted by the harness; lower is better for the same task |
| regressions (R) | Previously passing tests that now fail | Diff against a regression suite before and after the run |
| generalization (G) | Transfer to unseen tasks in the same or adjacent genre | Held-out level set; genre-shift variant |
| fun/hardness (H) | Human playtester rating of the generated artifact | Likert scale; cross-referenced with completion time |
| tokens (tok) | Input + output tokens consumed by the model | Telemetry from the inference runtime |
The evaluation syllabus
The syllabus is the bridge from raw intelligence to PnC expertise. Each step is harder because it adds state, composition, and evaluation against an external acceptance test.
main() entry point, compile, and run. Acceptance: executable returns 0 and prints the expected string.CMakeLists.txt or raw compiler invocation (flags, includes, link order) for the K0 source. Acceptance: clean build with no warnings at -Wall -Wextra.[[interactions]] chain for a point-and-click level and verify the puzzle dependency graph is a DAG with one source and one sink. Acceptance: level loads and is solvable.ARC-AGI, and what we are stealing from it
ARC-AGI is a public benchmark that evaluates few-shot generalization on novel tasks. It has a leaderboard, a harness, and an interactive environment. The latest ARC-AGI-3 corpus uses agentic interaction rather than passive prediction. We are using the same structural pattern, but the domain is game design and implementation rather than visual grid reasoning.
| Property | ARC-AGI | AAAc / PnC benchmark |
|---|---|---|
| Domain | Visual grid abstraction | Point-and-click game design + implementation |
| Unit of eval | Task puzzle | Level, mechanic, or vertical slice |
| Agent action | Predict output grid | Write TOML, code, or tool calls |
| Leaderboard | arcprize.org/leaderboard | Live bench + frontier scorecards (roadmap) |
| Generalization signal | Novel held-out tasks | Unseen levels, genre shifts, engine updates |
| Cost metric | $ compute budget | tokens + wall time + human review time |
The PnC micro-game as a benchmark substrate
The point-and-click system is a good substrate because the same TOML drives both the runtime and an agent planning representation. Our anatomy of point-and-click games explains why: a puzzle dependency graph is a STRIPS/PDDL task, the fairness doctrine is checkable as a DAG, and the level is simultaneously a game and an acceptance test.
Current state of the substrate
pnc_01 and pnc_02) is playable and the dependency graph is verifiable. The micro-build, alpha pipeline, public leaderboard, and remote compute offering are roadmap unless otherwise noted. Every claim below is tagged. | Component | Status | Evidence |
|---|---|---|
| Two playable PnC levels with fairness checks | implemented | game/assets/levels/pnc_01.toml, pnc_02.toml; docs/pnc/pnc_design.md §4-7 |
| Point-and-click engine system | implemented | engine/systems/level/pnc/pnc_system.{hpp,cpp}; game/systems/pn_click/pn_click_controller.{hpp,cpp} |
| Self-contained PnC micro-build executable | roadmap | .devin/plans/PNC_MICRO_BUILD_PLAN.md |
| Web-accessible PnC level editor | roadmap | Design contract in docs/pnc/pnc_design.md; editor depends on Tier 2 subsystems |
| Public agent leaderboard | roadmap | Will reuse bench telemetry pipeline |
| Remote private compute instances | roadmap | Hosted on the existing Vultr / www1 cluster; contact us for early access |
Implementation blueprint and timeline
turn and token counters into the existing pi-daemon inference surface.G across the variants.H (fun/hardness) score.Call to action
If you want early access to the PnC micro-game, the dataset pipelines, or an affordable remote private instance on the Vultr cluster that hosts www1, email the team.
Cross-links
- The Anatomy of Point-and-Click Games — the dependency-graph / fairness foundation.
- lab.catgameresearch.net — experiments and repro steps.
- bench.catgameresearch.net — live benchmark telemetry and scorecards.
- api.catgameresearch.net — the pi-daemon and agent harness.
- dreams.catgameresearch.net — cross-session pattern graph for what the agent learned.
- papers.catgameresearch.net — formal notes on RL, MDA, and game design.
Sources and caveats. Video metadata (title, author, thumbnail) were verified via YouTube oEmbed. The 14:59 segment summary is reconstructed from a public third-party transcript/summary of the talk (BigGo podcast page) and should be treated as a paraphrase, not a verbatim quote, until the auto-captions can be cross-checked. ARC-AGI facts are from arcprize.org. The AAAc metric set is a proposed internal framework, not an industry standard. All PnC micro-build, leaderboard, and compute claims are tagged as roadmap unless they have an implemented tag above.