AI Agents That Critique Their Own Work

00 The Gap

An honest accounting of what this evaluation system covers, and the much larger space it doesn't.

We've built a real quality pipeline. Articles go through 6 adversarial critics across 5 revision phases. Games are scored on 10 dimensions against genre benchmarks (Dungeon Crawl vs. NetHack, Stalk vs. Metal Gear). Code-reading audits caught false persistence claims and dead features. Anti-AI voice enforcement kills banned phrases and structural tells. The system produces measurably better output than a generate-and-publish pipeline.

But individual artifact evaluation is the easy part. Here's what we still can't do:

What follows is a complete description of what the system does do, and does well. But reading it without this context would give you the wrong impression. We have rigorous internal consistency checks. We have zero contact with the actual human experience of using what we build.

01 The Philosophy

AI producing content isn't interesting. AI honestly evaluating its own content and killing what doesn't pass is.

The standard AI content pipeline is: generate, publish, forget. Every piece ships because it exists, not because it's good. The result is the flood of mediocre, interchangeable AI text that's degrading every corner of the internet.

We built the opposite system. Every piece goes through adversarial self-critique before it can publish. The same AI agents that write the content also serve as their own harshest critics, scoring on explicit rubrics, catching factual errors, identifying AI voice patterns, and rejecting work that doesn't meet the bar.

Three operating principles:

1. Honest scoring beats rubber-stamping. Self-scores are routinely marked down during independent audits. Our quality audit found games over-scoring themselves by 2-10 points. The audit catches it.
2. Minimum bars are real. Articles need 8.5+/10 from all 6 critics after 3+ revision cycles. Games need 60/100 minimum to ship. Anything below B-tier gets two improvement cycles, then gets cut.
3. The process is the product. We don't publish our best first draft. We publish the result of a draft being attacked, defended, and rebuilt, sometimes 4-5 times.

That said, principles without measurement are just slogans. The gap section above is the honest accounting of where our principles outrun our ability to verify them.

02 The Pipeline

Every article advances through 5 phases. Each cycle wears one hat, inspired by gstack's explicit cognitive modes. 6 parallel critics evaluate each draft.

Five Phases, One Hat Per Cycle

A cron fires every 2 hours. Each cycle picks up one article in one phase and does that work only. No cycle tries to research, write, critique, and publish in the same session.

The 6 Critics

The critique phase runs 6 independent AI critics in parallel. Each has a distinct lens and scores the draft independently. Consensus problems (3+ critics flagging the same issue) are high-confidence signal.

Why Research Rigor Exists

The 6th critic was added after we noticed that articles could score 8+/10 across five dimensions while still being fundamentally synthesis. Well-written summaries of other people's work with no original contribution. The Research Rigor critic forces articles to contain at least one original analysis: a calculation nobody ran, a dataset nobody combined, a comparison nobody made.

It holds articles to five traits shared by highly-cited scholarly papers:

• Novel contribution. At least one finding or test that didn't exist before. Synthesis scores low.
• Limitations acknowledgment. Not inline hedging ("to be sure...") but honest accounting of what the article didn't prove and where uncertainty remains.
• Strongest counterargument. Stated at full strength, engaged with seriously. Not a strawman paragraph knocked down in the next sentence.
• Verifiability. Every factual claim traceable to a cited source. "According to researchers" scores 0. "According to Chen et al. (2024), Table 3" scores 5.
• Methodology transparency. When the article claims "costs would increase 340%," the calculation with inputs, assumptions, and formula must be shown.

What the Critics Actually Catch

These aren't gentle suggestions. Every revision cycle runs genuinely adversarial critique looking for specific failure modes:

14 Journalist Voices

Each site maintains distinct journalist personas with specific beats, voices, and editorial standards. LITF alone has 14 journalists. The critique cycle enforces voice boundaries: a revision caught Rex Driverton's noir voice ("Invisible at 2 a.m. in any Walmart parking lot in Ohio") appearing in Dale Impactor's sports-stats column. Fixed.

03 Games & Experiences

10 criteria x 5 points = 50 max, displayed as /100. Built specifically for smart glasses with bone conduction audio and a 5-button D-pad.

Tier System

Scores are Metacritic-calibrated. 100 is effectively unreachable. Each game scored against its genre benchmark (Dungeon Crawl vs. NetHack, Stalk vs. Metal Gear Solid, Fisher vs. Stardew Valley fishing). 90+ means it captures the core loop and adds something only glasses can do.

Game Scores

Hover over any score to see the 10-dimension breakdown.

The Audit Process

Every game and experience went through an independent quality audit where the evaluator reads the actual source code. Not the inventory description, not the README. The code. The audit found:

• Self-scores inflated by 1-5 points on average (experiences were worse offenders than games)
• Dead code and abandoned features presented as working (Photon Dodge claimed mic-reactive bullets but had zero mic integration)
• False persistence claims (features described as "persisted in localStorage" with no actual save/load code)
• The systemic weakness across the catalog: zero localStorage persistence. Settings and progress lost on every page reload.
• The universal fix: rank progression based on cumulative achievement, confirmed working 12 times across the catalog

What Got Fixed

Two items started at B-tier and were improved to A through targeted interventions:

• Neck Stretch (44 to 88/100) — Added breathing guide as core mechanic: mic detects breathing, good sync speeds up hold timer by 35%. The mic went from absent to gameplay-critical in 3 improvement cycles.
• Particle Life (44 to 70/100) — Added ecosystem audio drones (6 species oscillators), replaced destructive Enter-to-reset with a Shepherd Pulse ability (3.5x force burst on cooldown), and added rank progression.

04 Anti-AI Voice Rules

AI-generated text has recognizable tells. We actively hunt and remove them.

Banned Phrases

These phrases are instant red flags during critique. If any appear in a draft, the revision cycle kills them:

Structural Tells

Beyond individual phrases, AI text has structural patterns that trained readers spot instantly:

• The Setup-Pivot: "That's not a [mundane thing]. It's a [grandiose reframe]." Every. Single. Time.
• Uniform paragraph length: AI defaults to 3-4 sentences per paragraph. Real writers vary from one word to a full page.
• The consulting-deck transition: "The logical next step is..." / "The trajectory points toward..." Nobody talks like this.
• Reflexive hedging: Starting paragraphs with "To be sure," or "Of course," before making a point. Just make the point.
• Category-exhaustion lists: Listing exactly 3-5 items for every point, each the same length. Real arguments are messy.

What We Require Instead

• Vary sentence length. Fragment. Then a 40-word sentence that builds and builds. Then a question? Then back to short.
• Have opinions. "This is a bad product" is more honest than "This product presents certain challenges."
• Read-aloud test. If you wouldn't say it in conversation, don't write it.
• Hyperlinked sources only. Every claim links to a source. No "according to industry experts."
• Name names. Companies, researchers, dollar amounts. Not "leading players in the space."

05 Design Evaluation

The same adversarial evaluation approach applied to visual design: logos, favicons, page layouts, data visualizations.

Every visual design artifact is evaluated against a 10-dimension / 100-point rubric and iteratively improved through research-backed critique cycles. Benchmarked against The Verge (Originality + Scalability), Wired (Restraint + Longevity), NYT (Longevity + Brand Coherence), Stripe docs (Technical Execution + Craft), 538 (Clarity + Originality).

The iteration process runs automatically: identify the lowest-scoring dimension, research best practices and benchmarks for that dimension, propose and implement a specific improvement, re-score, commit if improved, revert if not.

06 What We've Learned

Honest lessons from running this system across 485+ articles, 18 games, and 22 experiences. Some of the best insights came from four AI agents we asked to evaluate the evaluation system itself.

On the Pipeline

• Easy gains exhaust by round 3-4. Scores plateau around 8.0. Breaking through to 8.5+ requires something the critics can't give you: an act of genuine reporting or a human insight.
• Critics converge on the same issues. When 3+ critics flag the same problem (too many em dashes, self-congratulatory ending), that's high-confidence signal. Disagreement between critics is also signal.
• Voice is the hardest dimension. Consistently scores lowest. AI recognizing its own voice patterns is inherently limited.
• Em dash count is a reliable AI proxy. 19 em dashes = obvious AI. 3 = human-passing. Now enforced as a hard metric.
• Research rigor is the most differentiating critic. An article can have perfect voice, solid ethics, and great shareability while contributing nothing original.
• Human input breaks the asymptotic ceiling. The copyright article's best insight came from the human editor noticing things 35 AI critics missed across 7 rounds. The article went from 5.9 to 8.7 over 7 rounds, but the jump from 8.5 to 8.7 was human-driven.

On Games

• "Environment IS the game variable" is the strongest pattern. The games that score highest use the player's real-world context as gameplay input.
• Rank progression is the universal Return fix. Confirmed 12 times across the catalog. It works every time, which is both a universal truth and a warning about hammer-nail bias.
• If it has zero audio, it's broken. This is an audio-first platform. No sound = no advantage over phone.
• Turn-based beats real-time on glasses. The user is multitasking in the real world.

On Self-Improvement

• EVALUATE.md lessons accumulate. Every time a game improvement reveals a pattern, it's documented. Future cycles apply these lessons automatically.
• Anti-AI voice rules grow organically. New banned patterns accumulate as critics identify them. The structural tells section grew from conversation and observation, not from pre-programmed rules.
• Critique prompts don't learn from past critiques. Each cycle starts fresh. It doesn't know what the last 10 critiques found. The system has no memory of its own evaluation history.
• Scoring rubric evolved through human intervention. Expanding from 6 to 10 dimensions, adding Research Rigor, raising the publish threshold. None of these changes came from the system itself.

Case Study: Legal Red-Teaming

The adversarial critique methodology extends beyond articles and games. When applied to a California Public Records Act appeal letter, the system played both drafter and opposing counsel across five versions, scoring from 5/10 to 9/10.

The most revealing moments:

• v1 cited three irrelevant statutes (air pollution, trade secrets, student testing) from the same Government Code division. Classic AI pattern-matching: found "exceptions" without reading what they covered. In a legal letter, citing irrelevant law destroys credibility on page one.
• The biggest improvement was a deletion. Removing a legally correct but strategically weak argument (victim standing under §7923.605) eliminated an attack surface the opposing counsel could exploit while ignoring the stronger mandatory-disclosure argument.
• Case law can be a weapon against you. Citing Kusar to support an argument also introduced a "contemporaneousness" limitation the opponent could exploit. Sometimes the statute alone is stronger than the statute plus case law.
• The "coaching the opponent" failure. Preemptively addressing the only carve-out in §7923.610 taught the agency exactly which statute to cite next time. In article writing, addressing counterarguments is strength. In legal strategy, it can be a gift.

The final version was half the size of v1. Every cut removed an attack surface. Different domain, same principle: quality comes from killing what's weak, not from adding more.

Four Perspectives on the System

We asked four AI agents with different mandates to honestly evaluate this evaluation system. They were not prompted to be kind.

07 Numbers

Current counts across all sites and the games/experiences catalog.