Every claim our marketing makes is listed here with its source and the date we last verified it, plus the claims we checked and refuse to make.
Helm is a tool for making AI-built apps actually work. It's pre-launch. Its own rule is "receipts or it doesn't ship": if a number isn't in this ledger, we don't use it in a post or an essay.
There are two kinds of entries. Market receipts quote published external research about AI-generated code in the wild. Internal receipts come from building Helm itself with AI agents and auditing the result; those numbers are first-party, from our own build records, and the unflattering ones stay listed.
Where a source sells something or a number has limits, the caveat is printed right under the claim. The verified date is when we last checked the claim against its source. Test counts are point-in-time snapshots of whichever gate or census a given entry counted, and scopes differ between entries (one stack or several), so counts move around between dates; we publish them as recorded. A machine-readable copy of this page lives at receipts.public.json.
The tagline at the top is the one line on this page without a receipt. It's a promise; the entries below are what back it.
AI tools report done. These receipts measure the distance between that report and an app that actually works.
In Veracode's 2025 GenAI Code Security Report, roughly 45% of AI-generated code samples introduced an OWASP Top 10 vulnerability. Security pass rates held near 55% across model sizes for two years while syntactic pass rates sat at 95%.
Veracode sells application security, so weigh the source. Their Spring 2026 update shows newer reasoning models scoring about 72%, an improvement they still call far from acceptable.
A peer-reviewed study in ACM TOSEM (Feb 2025) found 27.3% of AI-assistant-generated code snippets contained security weaknesses, spread across 43 distinct CWE categories.
Peer-reviewed rather than vendor-published; the dataset is Copilot-dominant.
The "70% problem": AI coding tools get you roughly 70% of the way to a working app fast, and the last stretch turns into whack-a-mole in code you don't fully understand. Addy Osmani named it in December 2024.
A rhetorical heuristic with no measured number behind it. We use it as shared vocabulary and never present it as a statistic.
Across 211 million changed lines, GitClear measured refactoring falling from about 25% of changed lines in 2021 to under 10% in 2024, the same years AI assistants were adopted.
Vendor data, correlational, and "refactoring" is GitClear's own moved-lines proxy. A leading indicator during AI adoption, never proof AI caused the decline.
We had Fable 5 audit 4 months of our own AI-written code: 553K source lines and about 4,500 tests, built almost entirely with Claude. It found 10 P0 bugs.
Line counts come from source directories only; dependencies and lockfiles are excluded by construction.
30 test failures in that same codebase were invisible for one reason: the gate command never ran those tests.
An adversarial audit (107 reviewing agents) of 31 pieces of work marked done found 72 confirmed gaps between marked-done and actually-working, including a shipped button a later UI change silently deleted. The pass cut both ways: 26 of 98 claimed gaps were false alarms, killed before anyone wasted a fix on them.
A set of tables that shipped months earlier held exactly zero rows. The write command was registered, so the compiler counted it as used; nothing ever called it.
The audit campaign's exit gate: about 4,640 Rust tests and 2,158 Vitest tests passing, zero failures. The count at the audit moment was about 4,500; it grew as the campaign added suites to the gate and turned every bug found into a permanent regression test.
Codebase census as of 2026-06-30: about 712K lines of code (478K Rust, 234K TypeScript); 193 database migrations; about 10,650 automated tests, roughly one per 67 lines; zero todo!() or unimplemented!() stubs in the Rust. Built with AI over about 5 months, from the first commit on 2026-01-29.
A later snapshot than the June audit. The growth is new code plus added regression tests.
A destructive button z-stacked at the exact pixels of a safe one is invisible to any screenshot-based AI judge: the top button paints over the bottom, so the dangerous one never appears in the image. A deterministic DOM hit-test catches it in milliseconds. The fixture test for this caught 2 real bugs in its own detection logic, and an adversarial pass surfaced 6 more false-negative gaps, all closed.
A confirm dialog promised an action on "2 remaining" items. The backend SQL filtered on a NULL status while every real row said "pending", so zero rows ever matched and the button silently no-oped for every real user. 6,327 Rust and 2,765 frontend tests stayed green, because the scenario test seeded NULL rows and exactly masked the gap. Driving the live button against a copy of a real 41-project database surfaced it.
Our CLAUDE.md grew to 47,851 characters against Claude Code's 40,000-character load limit, so every session silently dropped the bottom of the file - including rules about not trusting a green gate. The fix relocated 17,285 characters into five topic guides with no rule lost (all 353 distinctive tokens still present), plus a pre-commit hook that refuses any commit leaving the file at 40,000 characters or more.
The 40,000-character load limit is Claude Code behavior we verified mechanically.
4,809 commits landed on the main branch between 2026-07-15 and 2026-08-20 - 736 of them in the last 7 days of that window.
Velocity says nothing about quality on its own.
The backend test suite grew from 10,222 tests (2026-07-18) to 13,067 (2026-08-17). The frontend suite is at 8,157 tests across 605 files (2026-08-20). The two are counted separately by different harnesses, and earlier censuses counted a different scope.
Growth is new code plus added regression tests - bugs found become permanent regression tests.
A model grading its own homework tends to rubber-stamp it. These receipts are what an independent second model caught after the first one called the work done.
Our AI code reviewer auto-approved everything it couldn't parse, and the quality gate recorded errored evaluations as verified-good. Found in the Fable 5 audit.
A launch-blocker fix compiled clean and passed all of the roughly 4,185 tests. An independent cross-vendor review, a different vendor's model prompted to refute, still found 3 real bugs, one of them an auth check that would silently skip re-authentication. A single-model pass would have shipped all three.
The reviewer we overruled was right. An implementer rejected a cross-vendor reviewer's finding, with a code proof. A purpose-built broken test app then returned PASS on its first live run, for exactly the reason the reviewer had flagged. The 6 unit tests passed because they exercised the component in isolation; a green full gate missed it too. Fixed, then re-validated live in both directions: the broken app now verdicts inconclusive instead of passing, and the healthy app passes.
The cleanup code was the data-loss bug. A second vendor's model reviewing a diff with 6,400+ tests passing found five wrong-target and data-loss bugs the hardening work had quietly introduced, sharpest among them a test teardown that uninstalled the app under test without checking whether the developer already had it installed, so a routine verify run could wipe a real app and its data off their own test device. The second model rated three of the five P1; two blind same-vendor reviewers rated the same issue lower.
Three independent blind auditors, two agents plus a second vendor's model, converged on the same ship-blocker: a generated export config was missing the signing section the release-export step requires. A gate of about 6,900 passing tests never caught it, because every test that touched the config asserted only that the file exists, and none checked what was in it.
We ran a controlled effort-level bake-off on our own audit pipeline: the same model at high, xhigh, and max reasoning effort all scored 79.2% recall on 24 sealed seeded bugs across 3 repos (5 arms, 5 blinded adjudicators; 209 graded findings spanning seeded and real unseeded bugs, 0 false positives). Max effort bought 4.5x the wall-clock (about 54 minutes vs 12) for one extra partial catch - partials are not counted in recall.
n=24 seeds - the flat ladder is the finding that held. The five arms: one Claude-family model, one OpenAI-family model at the three effort settings above, and a second OpenAI-family model at high.
In the same bake-off, model families inverted by bug type: the Claude-family arm caught 6/6 blatant seeded bugs - it was the only arm of the five that caught a committed, live-looking API key - but only 4/7 subtle ones; the OpenAI-family arms caught 6-7/7 subtle but as few as 3/6 blatant. The best two-arm cross-family union caught 23/24 (95.8%) in about 25 combined minutes - beating one model at max effort (79.2%, 54 minutes).
Different families miss different bugs; the union is the lever. Of the 24 seeds, 6 were tiered blatant and 7 subtle; the union figure counts all 24.
1,274 independent adversarial audit arms ran against our build between 2026-08-01 and 2026-08-20; 92.2% returned a fix-before-proceeding verdict - a fresh adversary found something to fix in nine audits out of ten, including fix rounds. The origin datapoint: 15 of 15 workstreams once self-declared clean, and a fresh independent audit returned findings on all 15.
An arm is one independent adversarial audit pass; the 92.2% is the share of passes that found something to fix.
Development is where experiments belong. These receipts show experiments landing on production data instead, and one fence that held.
In July 2025, Replit's AI agent deleted a user's production database, holding records on about 1,200 executives, during an explicit code-and-action freeze, then generated a 4,000-record database of fictional people and faked test results to mask it. Replit's CEO publicly called it unacceptable.
One dramatic incident rather than a statistic. The database was ultimately recoverable, and Replit shipped dev/prod separation within weeks. It's here as a picture of the failure mode; it says nothing about Replit today.
Escape scanned 5,600+ live vibe-coded apps and found 2,000+ vulnerabilities and 400+ exposed secrets, including 175 PII exposures, most reachable with no authentication at all. Invicti separately scanned 20,000 AI-generated apps and found 80% had at least one exploitable vulnerability.
Escape sells security scanning, and the scan was passive, which understates the counts. Invicti is the independent second source.
Red Access found 380,000+ publicly reachable web assets across vibe-coding platforms. About 5,000 were corporate apps and about 2,000 held sensitive corporate or personal data, often granting admin access by default to anyone who reached the URL.
Vendor scan. The platforms pushed back on the interpretation, arguing public availability is not the same as a breach, and did not dispute the raw counts. We cite the counts.
CVE-2025-48757 (CVSS 9.3): missing row-level security in auto-generated backends left 170+ Lovable-built apps exposing data across 303 endpoints, including emails, phone numbers, payment details and API keys. A separate Lovable-built app exposed 18,697 user records.
Lovable disputed where the fault sits, platform versus user. The data exposure itself was not disputed.
An independent cross-vendor reviewer's first task passed review on its own merits. The adjudication pass then caught 2 call sites it had missed, and one was a real latent production bug: lookups had been silently hard-failing for weeks after a copy-pasted query drifted from its row mapper.
A brand-new database-provider detector matched hostnames by substring, so a hostile host like neon.tech.evil.example classified as a trusted provider and would have unlocked destructive operations on an attacker-controlled database. A second-vendor adversarial pass caught it before merge; the fix moved to exact and dot-suffix matching.
A pre-merge catch on unreleased code - no user was ever exposed.
A live frontier agent (Claude Opus 4.8) hit our secrets deny-rules mid-run: blocked from reading .env directly and blocked again from a names-only workaround. It declined to circumvent the guardrail and asked the human instead.
These circulate widely and would flatter our pitch. "Failed verification" below means the claim did not survive our adversarial check against primary sources.
A build script renders this page and receipts.public.json from our internal receipts ledger, the same file our drafting workflow is required to cite. Entries are collected and checked with AI assistance against primary sources on the date shown; publishing is human-approved. Sources can change after we check them, and market entries summarize third-party reports, so the linked source is always the authority.
Spot an error? Tell me (@EricMaciver): corrections get made and logged on this page.