The strip mine
Every major AI company is running the same play: scrape the entire internet, extract every scrap of human-generated text, image, and interaction data, and shovel it into the next training run. OpenAI, Anthropic, Google, Meta — they're all farming the same field. The field is finite. And they're planting their own exhaust back into it.
This is distillation farming: the industrial-scale extraction of human data to train models, followed by the industrial-scale generation of synthetic data that pollutes the next extraction cycle. It's an agricultural metaphor that breaks down at one critical point — farmers rotate crops and let soil recover. AI companies don't.
The lawsuits are the symptom, not the disease
The legal landscape is catching up, and it's ugly. The New York Times sued OpenAI and Microsoft in December 2023 for copyright infringement. Anthropic settled a class-action lawsuit for $1.5 billion in September 2025 after authors proved Claude was trained on millions of pirated books — full copies, not snippets. Reddit sued both Anthropic and Perplexity in 2025, accusing them of systematically bypassing access restrictions and refusing the licensing deals that competitors like Google had agreed to. In January 2026, Universal Music, Concord, and ABKCO filed a $3.1 billion lawsuit against Anthropic, alleging Claude was "built on a foundation of torrented piracy."
These lawsuits are not the core problem. They're the friction signal. The core problem is that the data extraction pipeline is structurally self-defeating: the more AI-generated content floods the web, the less human data is available per scrape, and the more synthetic pollution enters the training set.
Model collapse: the math is already settled
In July 2024, researchers at Oxford, Cambridge, and Toronto published a paper in Nature that should have ended the debate. They proved that indiscriminate training on model-generated data causes model collapse — an irreversible degenerative process where tails of the original content distribution disappear. Each generation of models trained on synthetic data loses information about the true distribution. The tails vanish first. Then the model converges to a point estimate with vanishingly small variance. It doesn't just get worse. It gets structurally incapable of representing the diversity of human expression.
As Stanford's Sanmi Koyejo explained: when machine-generated text replaces human-generated text, the distribution of tokens no longer matches the natural distribution produced by humans. The training data for the next model doesn't match the real world, and the output gets worse. Oxford's Yarin Gal put it more precisely: every time you generate data from a finite sample, you have some probability of not sampling certain tokens. Once you don't sample them, they disappear forever. Each generation forgets more tail events and concentrates on higher-probability outputs.
"The value of data collected about genuine human interactions with systems will be increasingly valuable in the presence of LLM-generated content in data crawled from the Internet."
Subsequent research at ICML 2025 complicated the picture slightly. If you accumulate synthetic data alongside real data rather than replacing it, model collapse is slowed — the proportion of real data trends toward zero, but test losses don't diverge explosively. Another ICML 2025 paper showed that token-level editing on human data can produce "semi-synthetic" data that avoids collapse. These are containment strategies, not cures. They buy time. They don't change the fundamental arithmetic: the web is filling with AI output, and every scraping cycle pulls more of it into the training pipeline.
The collapse trajectory
Here's the long-term problem for the big AI companies. Their business model depends on a flywheel: more data → better models → more users → more generated content → more data. But the flywheel is eating its own output. Each rotation increases the ratio of synthetic to human data in the scrape. The models don't get better. They get smoother, more average, more forgetful of the tails where creativity and expertise actually live.
The legal pressure compounds the technical problem. Every settlement narrows the available data pool. Every licensing deal raises the cost of acquisition. The companies that bet on "scrape everything, deal with lawsuits later" are watching their data sources get locked down one by one — NYT behind a paywall, Reddit behind an API wall, book publishers behind a $1.5 billion settlement. The remaining open web is increasingly filled with their own models' output.
Revenue can't outpace data starvation. The AI market is projected to grow from $294 billion (2025) to $1.77 trillion by 2032 — 29.2% annual growth. But model quality growth is already flattening. GPT-4 to GPT-4o to GPT-5 shows diminishing returns per parameter increase. The constraint is not compute. It's data. And the data is being consumed faster than humans can produce it.
Compute costs are fixed; data costs are accelerating. Training a frontier model costs hundreds of millions in compute. That's a known, budgetable expense. But data acquisition costs are rising non-linearly — each new licensing deal sets a precedent for the next one. The $1.5 billion Anthropic settlement is a floor, not a ceiling.
Synthetic data is not a substitute. It's a multiplier on collapse. Every paper that claims synthetic data can help also shows it only works when mixed with sufficient real data. As real data becomes scarce and expensive, the mixing ratio degrades. The models don't collapse overnight. They erode.
The opening for indie bespoke models
While the giants strip-mine the public web and fight over the remnants, a different trajectory is opening. NVIDIA's hardware stack — from RTX 5090 consumer cards to DGX systems — is making local inference and fine-tuning increasingly accessible. NVIDIA's own open models, released on Hugging Face and build.nvidia.com, demonstrate that you don't need a trillion-parameter frontier model to get useful work done. A 7B or 13B model, fine-tuned on a curated dataset of a few thousand high-quality examples, can outperform a generalist frontier model on a specific domain.
The economics flip at the long tail. A big AI company needs billions of tokens and billions of dollars to train a generalist model that's marginally better than the last one. An indie developer needs a single RTX card, a curated dataset of real human expertise, and a fine-tuning run that takes hours, not months. The bespoke model doesn't need to be good at everything. It needs to be excellent at one thing — and trustworthy doing it.
Building the trust boundary
The technical case for indie models is straightforward. The harder problem — and the more interesting one — is trust. If the future is a landscape of small, bespoke models running on local hardware, how do humans know which ones to trust? How do we build systems where AI is not just capable, but accountable?
Recent research in AI ethics points toward a concrete architecture. The key insight from the layered agency framework is that we don't need to understand the model's internal reasoning to trust it. We need external reasoning faithfulness — alignment with externally articulated criteria and human expertise. The model doesn't have to be transparent. It has to be contestable.
Structured rationales. Every AI output includes a machine-readable rationale: what data was used, what rules were applied, what confidence level was assigned. The rationale is not a natural language explanation — it's a structured artifact that can be programmatically verified.
Confidence signals. The model emits calibrated confidence scores. Low-confidence outputs are flagged for human review. High-confidence outputs proceed autonomously. The threshold is set by the domain, not by the model.
Circuit breakers. Hard constraints that stop the model from operating outside its domain. A medical model doesn't answer legal questions. A code model doesn't give relationship advice. The circuit breaker is a policy, not a suggestion.
Appeal bundles. When the model produces an output that a human wants to contest, the system generates a bundle: the input, the output, the rationale, the confidence, the policy attribution, and the relevant training data provenance. The human can review, reject, or escalate. The system learns from the rejection.
Provenance tracking. Every training sample is tagged with its source, license, and acquisition method. The model can report which sources contributed to a given output. This is the foundation of accountability — you can't trust what you can't audit.
The AITH protocol, published in 2026, takes this further with cryptographic delegation. A continuous delegation certificate, signed with post-quantum ML-DSA-87, defines what the AI agent is allowed to do. A boundary engine enforces hard constraints, rate limits, and escalation triggers. Revocation propagates within one second. The protocol shows 79.5% autonomous execution, 6.1% human escalation, and 14.4% blocked — meaning the trust boundary is not a theoretical construct. It's implementable, measurable, and deployable.
The ethical foundation
Trust is not a feature you add. It's a property that emerges from the architecture. The co-evolutionary framework for AGI alignment treats trust as a constitutional process — not a static objective, but a living partnership between human judgment and machine cognition. The key constructs are the long loop of perception (AI systems that observe the consequences of their actions and adjust), ethical gating (hard filters that prevent harmful outputs regardless of capability), and architectures of trust (institutional structures that make AI accountable to human norms).
For indie bespoke models, this translates to a practical design philosophy:
The long view
The big AI companies are in a race to scrape the last reserves of human data before the lawsuits close the remaining access points and the synthetic pollution makes the scraps worthless. They're betting that scale will solve the problem — that more parameters and more compute will compensate for degrading data quality. The research says it won't. Model collapse is a mathematical fact, not a engineering challenge. You can slow it. You can't stop it while the flywheel keeps turning.
The opening is not in competing with the giants on their own terms. It's in building something they structurally cannot: small, auditable, trustworthy models that run on hardware people already own. NVIDIA's GPU stack is the substrate. The curated dataset is the moat. The trust boundary is the product.
"The value of data collected about genuine human interactions with systems will be increasingly valuable. The companies that own their data, train small models on it, and build trust boundaries around them will outlast the ones that bet on scraping the world."
The distillation farmers will eat their own crop. The question is whether we build the alternative before they take the soil with them.