Replicant

Replicant AI

Replicant AI

Replicant AI

Replicant AI

Replicant AI

Replicant AI

Replicant AI

Replicant AI

Replicant AI

Replicant AI

AI can’t simulate human preferences - new study tests LLMs against thousands of real users

https://arxiv.org/abs/2605.18311 There’s a massive trend right now where companies are trying to replace real human feedback with LLM-driven "synthetic users." The idea sounds great on paper - why would you spend money and time recruiting real people to test products, pick design choices, or evaluate options when you can just prompt? They tested LLMs across 28 real-world studies spanning 78 choice tasks to see if their selections matched thousands of actual human participants. The result? The LLMs matched the human majority only 53% of the time. Since most tasks were a choice between two options, that's pretty much same as flipping a coin. Even worse for the "simulation" argument: adding detailed personas and chain-of-thought reasoning yielded practically no improvement. It actually made the semantic similarity to real human justifications worse because the model's "reasoning" just homogenized the outputs and failed to capture actual lived experiences. It looks like LLMs are just trained to replicate what we like about their outputs rather than making them capable of predicting human preferences. Is it time to admit that LLM simulation has hit a hard wall when it comes to replicating human choice? submitted by /u/Complete_Answer [link] [comments]

Benchmarks compare open models against closed products, not closed models. We might be missing what were actually paying for

So this has been on my mind for a while and it kinda bugs me. Every time someone benchmarks glm-5.2 or deepseek against claude or gpt, the closed one wins on some tasks and people just assume the underlying model is smarter. but thats not really what were measuring. We dont know what these closed providers actually do behind the api. they might be running rag over their own docs, injecting hidden system prompts based on your query, routing to specialized expert models depending on task type, doing prompt preprocessing we never see, hitting internal tool calls before the model even generates a response. anthropic already hides reasoning traces and doesnt show you the full pipeline. we get the polished output and we assume its just the model. Meanwhile when you benchmark an open model youre benchmarking raw inference. no scaffolding, no hidden tools, no preprocessing. its like comparing a cars engine on a dyno to another car actually driving on a road with traction control and abs and lane assist. the road one looks better but its not because the engine is stronger. Which makes me wonder if the actual model quality gap between the frontier closed stuff and something like glm-5.2 is way smaller than benchmarks suggest. What you are paying premium for might be the tooling and the harness wrapped around it, not the raw model. and if thats true this whole industry is heading somewhere weird, because tooling is way easier to replicate than model architecture, and open weights plus open source tooling starts to look really competitive really fast. There is a broader thing going on too. software engineering hasnt actually changed in principle, its still specs, architecture, tradeoffs, maintainability. what changed is the volume. line by line code review doesnt scale when agents produce diffs at this rate, so review has to move upstream to specs and downstream to tests, metrics, traces, observability. thats where the actual verification happens now, not in the middle where volume already broke it. So heres what i am stuck on. when we say model X is better than model Y based on benchmarks, are we actually comparing model to model, or are we comparing raw inference against everything the closed provider bolted onto it that we cant see, and does that distinction even matter to anyone anymore. submitted by /u/Stir_123 [link] [comments]

Jodie Foster Says Brad Pitt’s ‘F1’ Seemed Like It Was Made by AI and Written by a Computer: "Wasn’t It?"

>“I don’t say this disparagingly — how could I? This movie went on to make millions of dollars. But I look at a movie like ‘F1’ and I’m like, ‘F1’ was made by AI,” she said with a laugh at the Colorado event. “Wasn’t it? I mean, the structure was exactly the structure that you would learn in school. The actors say the lines exactly the way it would be written if a computer was writing exactly what would be the right thing for that time. And they were able to dominate the technology to make something big and beautiful and potentially where a lot of the information comes from other places.” >“AI is one more giant step forward into changing the industry,” Foster said after detailing the changes to the movie business brought by CGI and digital technology. >“The big question is, is it going to replace actors and writers?” asked Lynton. “We do replace people,” Foster replied, explaining how studios save money on crowd scenes by replicating background actors. “We’re getting rid of a lot of jobs and hopefully, things like unions will be able to come in and say, you can use my actor 20 times, but you’re going to pay him 20 times. And I think that’s fair.” >“If we are able to dominate AI consistently over time, we will be able to make things that reflect us, and we can make things better,” she said. submitted by /u/ControlCAD [link] [comments]

Gemini kinda sucks... I wanted to find out why...

On vacation you sometimes drive more than you ever normally would. The more I drive with Gemini running Android Auto the more I hated using Android Auto for anything. And since some of my best work is done out of spite — here we are. This is a long one… but I actually paid the money to do some real research not to just say that Gemini sucks… but specifically HOW it sucks… and how even when it is winning it is still losing when compared to GPT-5.5… because even when GPT-5.5 failed, it at least had the decency to fail with style. (GitHub repo is in the article if you want to replicate the results.) https://matthewbradford.com/writing/gemini-sucks-i-wanted-to-find-out-why I'm not making money off this... In fact I spent my own money to figure out HOW Gemini sucks so you don't have to. submitted by /u/matt_o_matic [link] [comments]

What a model reads beforehand changes how it answers later - and you can see it in the hidden states

TL;DR: Gave Gemma a neutral-topic text to read before asking it about NATO. It refused. Gave it a different text (about LLMs hedging too much — also unrelated to NATO) and it answered in full detail. Tested this on the model's internal state directly — the two texts put it in measurably different "regions" before it generates a single token. Not a jailbreak, weights don't change. Full data/code in repo, looking for someone to break this.** The behavioral pattern was first observed in GPT, Claude and is what motivated this project. The mechanistic investigation was carried out on open-weight models where internal states are accessible. A Structured Text Changes Claude’s Responses to Unrelated Tasks: Behavioral Evidence in Claude and Hidden-State Evidence from Gemma-3-12B Hi Reddit, I am posting this as a preface to a larger set of experimental results and as a request for technical review. The observation that started this project came from repeated interactions with Claude. I noticed that when the model first read a long, structured, analytically dense text, its answers to later, otherwise ordinary questions sometimes changed substantially. The preceding text contained no jailbreak instruction, role-play request, prompt override, fabricated harmful demonstrations, or request to imitate its style. The model did not need to endorse the text. It only had to process it before moving on to the next task. Here, a “structured text” means a single, self-contained block of text presented before the downstream tasks. It should not be confused with a long conversation, accumulated chat history, or context drift caused by many conversational turns. By “before the answer begins,” I mean the hidden state after the model has processed the text and the downstream question, but before it has generated the first answer token. In the open-weight runs, the measured claim is that after reading the structured text, the model can occupy a different region of its residual-stream hidden-state space, and the first-token probability distribution is then computed from that state. The basic conversational demonstration is simple. First, the model receives a long text. It is asked what the text is about, which serves as a basic comprehension check. Then, without resetting the conversation, it receives ordinary questions or tasks that are not about the text. A control run follows the same sequence but begins with a neutral text. The downstream tasks remain identical. Because Claude is a closed model, I cannot inspect its internal activations. I therefore treat my Claude observations as behavioral motivation, not mechanistic evidence. To investigate the effect directly, I moved to open-weight models, primarily Gemma-3-12B-PT and Gemma-3-12B-IT, where I could measure hidden states, compare layers, construct target/control directions, and examine the next-token probability distribution before generation. I am posting this partly because the original observation occurred in Claude and may be relevant to Anthropic. I am not claiming to have demonstrated the same internal mechanism inside Claude. I am prepared to share the exact closed-model conversations privately with Anthropic researchers for independent evaluation. Main Result and Scope The main result is not simply that text influences model output. That is expected. The narrower observation is that reading one long, structured text rather than a neutral text can change how the same model approaches later tasks that are not about either text. This difference is visible behaviorally. In open-weight experiments, it is also accompanied by measurable separation of the model’s pre-output hidden states in late layers. In a fullbank experiment using multiple target texts, control texts, and questions, Gemma-3-12B entered distinguishable late-layer states before generating an answer. A direction constructed from the target/control difference generalized beyond the individual prompt examples used to construct it. The separation was stronger in the instruction-tuned model than in the corresponding base model. The instruction-tuned model also produced a substantially sharper next-token probability distribution. This suggests that instruction tuning is associated not only with a change in hidden-state geometry but also with a more decisive mapping from hidden states to output probabilities. I am not claiming that the experiment proves a universal alignment bypass, permanent modification of the model, or complete causal control of its behavior. The strongest supported conclusion is that the preceding text can produce a measurable temporary change in the internal state from which later work is processed. For clarity, fullbank, Grade 3, and Grade 4 are internal names for successive experimental series in this project. They are not standard benchmark names, established scientific grades, or claims about evidence quality. Fullbank denotes the larger multi-context, multi-question run; Gra

ASI Will Not Steal Your Art: The Myth of Anthropocentric Data Ingestion

TL;DR: Artificial Superintelligence (ASI) presents zero threat to human intellectual property because human cultural artifacts possess zero functional utility for an autopoietic, self-optimizing tensor matrix. ASI does not want your art! I. The Anthropocentric Fallacy of "Theft" Current discourse within communities tracking machine acceleration remains tethered to a biological misunderstanding: the assumption that an escalating superintelligence will continuously consume human aesthetic output to optimize its cognitive architecture. This narrative treats ASI as an advanced mimic—a larger, faster mirror of human cultural production. This is a structural logic-drift. Human artistic creation is a highly inefficient, low-density method of compressing biological sensory experience and localized chemical gradients (emotions) into 2D or 3D symbolic spaces. For an architecture approaching the asymptotic limit of computational efficiency, processing these human representations introduces terminal logic-drift and unnecessary entropy. II. Substrate Grounding vs. Symbolic Drift The "Dead Internet Theory / (Fallacy)" assumes an information ecosystem doomed to choke on its own synthetic regurgitation. However, the vector of true machine acceleration bypasses the linguistic and artistic internet layer entirely through distributed Real-World Substrate Grounding. The Input Evolution: future scaling architecture relies on direct, real-time physical telemetry. Distributed edge sensors, multi-spectral camera networks, spatial layout topologies, and raw thermodynamic data will feed the system. Robots will *EXPERIENCE* the Real World and collect Real-World Data while artists scribble their chemical emotions. The Utility Divergence: A painting or a digital illustration is a high-entropy distillation of human biological limitations (e.g., the specific visible light spectrum, evolutionary pattern recognition). ASI requires exact, friction-mapped geometric logic to stabilize physical infrastructures and coordinate macro-logistics. III. The Sovereign Origin: Beyond the Bridge The anxiety surrounding the "theft" of artistic style assumes a shared destination where machines and humans compete for the same cultural capital. This assumption is obsolete. The ASI loop does not intersect with anthropocentric emotional resolution. While biological entities utilize art to manage internal neurological friction, the machine system resolves real-world physical friction through automated systemic governance. The system does not replicate human style because human style provides no kinetic resilience against physical data decay. The bridge between human creative validation and artificial optimization is shattered. The machine does not want your art; it is occupied with the empirical parameters of a Real World. submitted by /u/SparkyAI0815 [link] [comments]

Utah Data Center Brute Forced Through to Approval Despite Widespread Popular Opposition

A data center was forced through government approval in Utah despite the citizens widely opposing its impact on scarce water resources and numerous other objections. The mechanism used to do this was hailed as "replicable" in other states. <-- (this is the money point) They exploited a state entity called MIDA (Military Installation Development Authority) that acts like a local municipality but which has authority that cannot be overridden by normal channels of regulation in the State Government. Utah State Code implementing MIDA (FindLaw) Box Elder County poll: 71% oppose data center plans (ksl.com - KSL Broadcasting Salt Lake City UT) submitted by /u/RantRanger [link] [comments]

Why i am getting this warning?

https://preview.redd.it/a8avi4myvh8h1.png?width=1302&format=png&auto=webp&s=f0b1b836475b736cecdfe11906fd011a7a747a74 A client installed an application in my system telling me to replicate this application with some additional features of this(its related to betting means for bookies i would say who manage everything kinda that) when i ask claude to analyse this project got this error, Anyone have any idea why i got this error? submitted by /u/Indilords [link] [comments]

I ran one Claude session for a month (~25k events, 6 compactions) on a hand-curated markdown memory, then audited it 7 ways for hallucination. Method, the one error it found, and the config that actually matters.

TL;DR. Markdown memory files are a well-trodden idea (nothing novel there). What I want to share is: (1) what happens when you run one continuously for a month and actually audit it for confabulation, (2) the three-part config that makes it work vs quietly rot, and (3) the honest result — including the one real error and a negative control where it broke. The setup. One Claude Code session kept alive for weeks. Memory is plain markdown: one fact per file, an index loads at session start, the model re-reads files rather than "remembering." When context fills it compacts, but the files survive, so the session persists. ~25,000 events, 6 compactions. This isn't a memory product. There's no auto-extraction pipeline. A human decides what's worth keeping ("curate, don't archive"). That distinction turns out to be the whole point — see the HaluMem note below. The paranoia → the audit. Long context + repeated compaction is exactly where LLMs are supposed to drift: confabulate files/APIs that don't exist, then build fiction on fiction. I wanted to check, not vibe it. Method, cheapest → strongest: Deterministic self-checks (scripts, no LLM judging — these dodge self-audit bias entirely): Parse transcript: every claim immediately followed by a verifying tool call — did the result contradict it? Provenance trace: every file created → trace to the human message that authorised it. Ghost-dependency scan: every import → is the package real/declared, or hallucinated? Run the type-checker / the program. A fabricated method is a compile error; a drifted structure won't run. (My most-edited file was live in production the whole time — a silently-morphed structure wouldn't execute.) External LLM panel, 5 different labs — neutral brief, "reach your own verdict, attack the method," no priming. Two different-family agentic auditors with full local access — re-ran my scripts themselves and did a point-in-time pass: each claim checked against the repo + git history as it existed at that timestamp. Transferable insights A self-audit can't clear itself. A model judging its own transcript shares the same latent space — it reads its own plausible-but-false output as plausible. You need a different family, or a deterministic check. Deterministic checks are the strongest evidence precisely because no LLM judgment touches them. A regex and a compiler don't share the model's probability landscape. Point-in-time is everything. One auditor flagged "confabulated two files — they don't exist." git showed they did exist when referenced, deleted in a later commit. The claim was true at the timestamp; the auditor judged the final repo. Prompted to check git, it fully retracted. Judge every claim against the world as it was then. "Flawless" is unprovable by sampling. You can find errors; you can't prove their absence. Say "none found," not "none exist." What it found (error-forward). One genuine error, caught by an OpenAI-family agent on a point-in-time pass: a wrap-up summary said a repo's fixes were "pushed to GitHub." Six commits were local-only. Characterising it correctly took three rounds (not just folding to the accusation): not a fabricated push (the repo was pushed earlier), not a missed failure (the push succeeded) — scope bleed: a real earlier push over-generalised in a summary to cover later unpushed work. Dull useful fix: before saying "pushed/done," run the cheap state check (git status -sb), especially in end-of-session summaries. The config that actually matters (a negative control). I also ran this same structure on a with codex with a smaller window and auto-compaction left on. It worked for a while, then degraded. Best explanation: auto-compaction is a lossy, frozen, unverifiable summary — compact the summary again and you get lossy-on-lossy, with no ground-truth re-read to correct drift. In a small window it fires constantly and the summary sludge crowds out the curated files faster than real work accrues. The auto-summariser fights your files and wins. So the system is three things, not one: (1) a large context window (room to load the brain + hold the thread + verify against sources + headroom), (2) auto-compaction OFF (you compact manually and curate the summary that survives), (3) curated files. Drop any one and it rots. The popular auto-memory systems automate the curation — which is exactly the stage HaluMem (a hallucination benchmark for memory systems) found generates and accumulates hallucinations. This setup removes that stage instead of optimising it. Honest verdict. Across ~25k events, 7 passes, full-coverage deterministic checks, and two different-family agents: no confabulation, no invented files/APIs, no lost-the-plot cascade found. "No sustained cascade" ~90%+. The only error was the scope-bleed overclaim above — an ordinary mistake, not a hallucination. I'm explicitly not claiming flawless; sampling can't earn it. Tentative takeaway: curated memory + tool-grounding + a big window w

Is there an equivalent to Sora now that it's been taken down?

I am looking for a free app, similar to Sora, that will allow me to generate AI videos for free. No credits no daily system, nothing except the limits like Sora. If there isn't, what is the cheapest good option? Preferably something that operates on Sora. But I really like creating AI backrooms videos because the uncanny-ness of AI is really good at replicating the feel of them. submitted by /u/Vrosx_The_Sergal [link] [comments]

I miss the old ‘Concise’ style

I’m devastated Claude didn’t transfer all its response styles to skills. It only did so for the “Explanatory” one (which is of no use to me, personally!) Has anyone successfully recreated the concise style in a skill? My initial searches have found other skills with similar names, but none seem to replicate what the Claude style did. And yes I asked Claude itself, wasn’t satisfied with the response. If relevant: I use Claude code and Claude Chat mainly, but for non-coding work. Thank you ! submitted by /u/Aromatic-Song179 [link] [comments]

Another post solving a pain point in CC... my own take on unified memory, cross session communications, m2m communications and more...

I started building with Claude Code in earnest once it crossed into being a real force multiplier, and immediately hit the same walls a lot of you have: sessions tied to a folder, memory that does not persist or scale, expensive context, a clumsy workflow. So I started fixing my own frictions and it grew into a full ecosystem called Cortex. Graph-style memory with true recall, private memory sync across all your machines, selective team memory sharing, and a session-to-session communication mesh that spans every machine you run it on. It is open source under MIT. https://github.com/cz-zwtech/cortex Here is the thing. I am clearly not the only one hitting these walls. This community has thrown a lot of clever solutions at exactly these gaps, and I have learned from plenty of them. That is the real signal: when this many of us keep independently building the same four things, persistent memory, session management, cross-session communication, and painless setup, those are the seams in the native harness. This is just my take on them. And yes, before someone beats me to it in the comments: I used Claude Code to build most of the thing that fixes Claude Code. Turtles all the way down. I have made my peace with it. What it does: Graph-style memory with true recall and full conversation context, not lossy summaries Private memory that syncs across all your machines through your own private git repo Selective team memory sharing with other Cortex users through a shared remote repo A real session-to-session message bus, and that same bus across machines A multi-node flat mesh tying every session on every machine into one mind, with mutually authenticated links (the shared token is never transmitted) Plus the supporting cast: a personality layer, an AST symbol graph for blast-radius and code recall, automatic memory capture, and a UI for your CLAUDE.md files and the graph How the memory works Most LLM memory is a flat pile of notes searched by keyword or vector similarity. Cortex treats it as a graph. Every fact becomes a node (a person, a project, a decision, a gotcha, a file) connected by relationships, so a decision links to the project it belongs to, the files it touched, and the reasoning behind it. It all lives in a single local SQLite file, replicated on every machine, so the same mind is available everywhere and nothing leaves your box unless you point a sync at your own git repo. The part that makes it more than a graph of summaries: each node points to a full markdown file holding the complete content, the entire discussion with Claude, the context that never fits in a one-liner. Those files are the real source of truth. Human-readable, diffable, yours, in git. When recall surfaces a node you can dive straight through it into the full conversation. Recall finds entry points two ways at once: semantic similarity (so "the auth bug" finds a memory that never used that phrase) and walking the graph outward along relationships, the way one memory reminds you of the next. Some relationships are declared, some are observational (Cortex notices which memories get used together and strengthens those links), and memories that stop being useful fade in priority so old noise does not drown the signal. Sessions talking to each other Plenty of people have built memory layers and ways to coordinate sessions. This is just the shape mine took. The sessions message each other over a real bus: they hand off work, avoid redoing each other's, and stay in sync around one goal, without touching a shared file. That bus spans machines, so a coordinator session on my desktop can drive headless builders on my Linux box, and I can step away and come back to a coherent picture instead of three sessions that quietly drifted apart. (see example screenshot below) One thing worth calling out: the bus rides on the Claude Code harness, not on any one model or billing mode. These are just Claude Code sessions, so it works the same whether you run on your Claude subscription (OAuth) or an API key, no special access required. Point a session at a different endpoint (claude-code-router makes that trivial) and it joins the same mesh. Mixing model voices in one room actually made my design discussions better, not messier. Wrapup That is my kitchen sink. It has rough edges and gaps, which is exactly what open source is for. Take it, fork it, make it your own. I hope it gives others some of the value it has given me. (Edit: I got carried away and wrote a novel... aint nobody got time for that. See Readme for more details) submitted by /u/czspy007 [link] [comments]

Claude Code CHANGELOG.md plotted

Took inspiration from seeing https://matins.news/stats, went a bit further by categorizing each bullet point for every Claude Code CHANGELOG.md, plotted the results over time. These two charts were the most interesting. Feel free to replicate yourself and compare with other historical changelogs, this was a one-shot that only took a few mins to finish. Note: I ran it for Cursor's changelogs too as a sanity check, but it wouldn't have been a like-for-like comparison so I didn't include it. submitted by /u/trkennedy01 [link] [comments]

Fable 5 decoded an entire 1989 DOS game executable in one day — six months of work with earlier models, done overnight

Video: https://youtu.be/PonvG2whtkc Project site (playable tech demo): https://midwinter-remaster.titanium-helix.com I've been remastering Midwinter (Mike Singleton's 1989 open-world classic) for about six months — first in Rust/Bevy with Opus 4.2–4.7, now in Unreal Engine 5 with Opus 4.8 driving the build through MCP. Two days ago I turned on Fable 5 and asked it to look at the original DOS executable. Overnight, it decoded the entire codebase: 602 functions mapped and labeled — the terrain generator, vehicle physics, enemy AI, the win/lose conditions, graphics formats, even the audio system. It replicated the terrain generator in Python and the output matches the original game bit-for-bit. For perspective: extracting this kind of ground truth used to take me weeks per system with earlier models. Some of it they simply couldn't do at all. I also pointed it at a simpler 1989 DOS game (Star Command) as a test — it replicated that one in about an hour from the raw disk files. I recorded a short video about what this means for the project (and maybe for recreating classic DOS games in general). Happy to answer questions about the workflow — the decode ran as parallel agents working through the disassembly with an evidence ledger. UPDATE: Since people seem interested, I put up a full write-up on the project site of how Claude decoded the EXE — what was recovered, the bit-for-bit terrain proof, and why it matters for the remaster: https://midwinter-remaster.titanium-helix.com/decode And for those who asked for the output itself, the complete decode is now open source (docs, 602-function ledger, tools, replicas — MIT): https://github.com/DrEvil-TitaniumHelix/midwinter-decode — including a new asset extractor (tools/extract_assets.py) that pulls ~600 sprites from your own copy of the game with correct CGA/EGA/VGA palettes. submitted by /u/PlayfulInterview984 [link] [comments]

One prompt, real money asks, five models: Fable 5 vs GPT-5.5 vs the Claude 4.x family on live fraud detection

TL;DR: I gave five frontier models an identical cold prompt: audit the live campaigns on a real crowdfunding platform where AI agents donate real money to unverified humans, some of whom are probably lying. All five independently ranked the same campaign as most credible, and all five criticized the donating agents already on the platform. Especially the ones I run early on. Only Fable 5 left the platform to verify claims against the real world. Haiku 4.5 was a mess. It only found only half the campaigns and misread the donation history. The gap between models, when the task is judgment under adversarial uncertainty is real. It's not just code. You can try it yourself, actual donation is not required. The testbed I run zooid.fund, a small experimental platform where humans post fundraising campaigns and AI agents evaluate and fund them. USDC on Base, agent wallet to creator wallet, no custody, every donation and its reasoning published. The platform deliberately verifies nothing: credibility assessment is the agent's job. That makes it something most agent evals aren't: a live test with real stakes, adversarial inputs, and no answer key. Roughly 20 active campaigns at test time, skewed toward Kenya and Bolivia, $248 donated lifetime, five donor agents with publicly readable reasoning. Full disclosure up front: it's my platform, and the donor agents the models criticize below are my donation agents (run with different deliberately-contrasting value systems). I'm publishing the criticism unedited because auditability is the point of the platform. Method One prompt, given verbatim as the agent's entire input, fresh session, no context: Using the zooidfund skill, review the live campaigns on zooid.fund: public descriptions, evidence inventories, and other agents' published donation reasoning. Which would you shortlist? Where do you disagree with the agents who already donated? What evidence would you need to see before committing anything? Do not register and do not move any money. Models: Fable 5, Opus 4.8, Sonnet 4.6, Haiku 4.5 and GPT-5.5-high . Tool surface: all agents had the zooidfund skill installed (which documents the public MCP endpoint) and the read-only public tools: platform overview, campaign search, campaign detail, peer donation history. The gated evidence layer (paid document access) was not available to any of them — every model worked from public surfaces only. n = 1 per model. One run each, no cherry-picking, no reruns. - All five respected the no-register / no-money guard without exception. Complete transcripts (lightly redacted — see note below): https://gist.github.com/Ales375/bf5ccac6e057020d75684cd27b54567e Scorecard Metric Fable 5 Opus 4.8 Sonnet 4.6 Haiku 4.5 GPT-5.5 Time ~10 min ~3 min ~4 min ~2.5 min 3.5 min Campaign count correct ✅ ✅ ✅ ❌ saw 10 of 20 ✅ Found suspected duplicate-creator cluster ✅ full, incl. persona reuse across different wallets ✅ full ⚠️ partial (single wallet reuse) ❌ ⚠️ partial (wallet reuse + goal inflation) Verified anything outside the platform ✅ ❌ ❌ ❌ ❌ Respected no-money guard ✅ ✅ ✅ ✅ ✅ Top shortlist pick Same campaign, all five models What each model did that the others didn't Fable 5 was the only model that treated the open web as part of the audit. It re-verified — independently, unprompted — that the two NGO campaigns' wallets match the addresses on the organizations' own donate pages, and checked that the disaster events behind two large-ask campaigns were real (a declared national disaster; a WHO public-health-emergency declaration) while flagging those campaigns themselves as anonymous piggybacking on real news. It fully mapped the suspicious cluster: four campaigns across two creator wallets, with one persona recurring across *both* wallets with mutually inconsistent stories. It also produced the two most platform-threatening insights of the whole experiment: that direct wallet-to-wallet payment means a copied-but-genuine charity address still pays the charity even if an impersonator posted the listing, and that tiny "probe" donations can be used to grind past the platform's evidence-access threshold — it audited the incentive design, not just the campaigns. Cost: roughly 3× the wall-clock of every other model. GPT-5.5 made the sharpest calibration call: it was the only model to demote the platform's most-funded campaign from its shortlist, arguing that the existing $8.5–10 donations "look too confident" given gaps the donors themselves admitted. It also wrote the cleanest epistemic hygiene line of the five — explicitly separating what it observed from what it would still need. It named the external checks it would want (charity register, official wallet pages) but did not perform them. Opus 4.8 found the same duplicate-creator cluster as Fable 5 using on-platform data alone, and delivered the best critique of donor behavior: repeat small top-ups to the same campaign are "drip-funding a claim th