Catch the AI

"Strawberry" has three r's. AI once said two.

Six sentences an AI wrote. Some hide a trap. Can you beat the checker?

Muse Wonder, Meta's AI, wrote these sentences, and I asked it to set traps in some. How this works
It runs in your browser. Nothing you type leaves your device unless you choose to save your tries.

Skip to make your own round

Getting the checker ready. It loads once, about 8 MB. That takes about 3 seconds on a fast connection, or 15 to 30 on a slow phone. After that, everything runs on your device.

Loading the six cards.

How this works

Muse Wonder wrote the cards and sentences. Your browser checks their numbers.

The cards and sentences are exactly what Muse Wonder wrote: rounds.json. The brief it got is saved here. Only its disclosure line changed, on Nov 3, from "We asked" to "I asked": this is a solo project, and that line was set word for word in the brief.

Muse Wonder, a Meta model, wrote these sentences from each card. I asked it to slip mistakes into some. Six fixed, made-up examples, not an accuracy test.

The checker reads names and numbers. It doesn't read seasons, units or words like rose and fell.

A pass does not make a sentence true. The checker compares numbers with the card, and some checks do not run on this page.

The code is at github.com/macayaven/ikigai-loop.

A paid study uses a separate version of this page, with its own consent. Other visitors send nothing unless they choose to save their tries.

Why I built this

Assistants that read your data and answer in numbers get some of them wrong. They sound just as sure either way, and nobody checks every number by hand.

I built an assistant that offers each free hour of my day to promises I wrote down. Before a card reaches my phone, this check compares the numbers on it with my records.

It still let some through. On Nov 2 a card said "Prs rose by 3 in the last period": the right number, from the wrong window. Five more sentences that day made the same mistake. A test that fails on that sentence came first, then the rule that stops it. Every miss so far is in TRUST.md.

When it works, a wrong number is stopped before anyone acts on it. That's why you can try it here, for free, on any sentence an AI wrote for you.

What the checker can't see

Every word of a row's name must appear in the sentence, in any letter case, so deploy doesn't match deploys and synonyms don't count.

Numbers are read as written: 1,200, 3.5 and twelve work. Dates and clock times aren't read as numbers. In the plain form, a percent sign is ignored.

The number must equal the record. 3.5 and 3.50 are equal.

A number with no record fails.

A sentence that names more than one record isn't checked until it's split, one record per sentence. A sentence like "We shipped 12 deploys and closed 40 tickets" names two records, so split it into two sentences.

In the plain form, words like rose or fell aren't checked.

A sentence with a negation word fails, and a quantity word in place of digits fails: about, over, nearly, half, a couple, twice.

It can't tell whether a sentence is true, only whether its numbers match what you typed.

To see for yourself that nothing is sent unless you say yes: open your browser's network panel, then use the page. After loading, the page makes no more requests until you choose to save your tries, and then one per try.

For engineers: JSON input
Made-up examples:

This is what the full checker covers:

This check covers parseable numbers, cited sources, labels, direction, negation, quantity forms, and units and periods where you declare them. It does not check that a sentence is true. Without labels it cannot tell which metric a number belongs to, and a cited fact is only checked to exist. A check listed as not checked did not run.

What fails now: a claim with a negation word fails, unless it states a zero right before a word of the cited metric's labels and the cited value is 0 (for example "No deploys failed" with labels deploys and failed); quantity words in place of digits fail (about, over, nearly, half, a couple, twice); with a declared unit, a unit word that disagrees fails; with a declared period, a missing or stray period fails, and so does a value beside the wrong period.

What this checker does not check: it never checks that a sentence is true; a word after a number that is not a unit, a magnitude or a percent form isn't checked; disguised spellings of your private terms aren't caught; advice and instructions in the text aren't caught unless you list advice words.

Disclosed limits: the bare word one as a value, period compounds and comparisons with than are not checked; a number-like or quantity-like word inside a name that a cited fact carries is not skipped, which can only make the check stricter; a unit word that agrees with one cited metric can still fail against another, because the sentence does not say which metric the word belongs to; the Python function verify_claims keeps the earlier, smaller set of checks and verify_and_repair still wraps it.

Every field is in the worked example, and the adapter recipe maps another shape to it.

Show the JSON (for scripts)
The exact input the checker ran lands here.

The worked example runs the same input on the command line.

The page and the checker it runs are open source in macayaven/ikigai-loop. The page lives at https://carloscrespomacaya.com/ikigai-loop/verify/. Bad input shows a short message and sends nothing.