AI-RIS
AI gets it wrong with a straight face. We measured how often.
We asked two leading AI models 648 real questions and checked every answer, fact by fact, against official sources. Then we tested the AI that grades other AI. Here’s what came out.
of one leading model’s verified failures were confident false claims, not cautious hedges.
Croatian recall audit, 2026of recently changed facts delivered by that model, against 72% of stable ones. Old figures, stated as today’s.
Croatian recall audit, 2026gap between two leading models on the same 140 facts. Which AI you ask mattered far more than which language you asked in.
Croatian recall audit, 2026how much harder one AI grader penalised balanced, careful answers than another did. Automated checking has its own bias.
The Balance Penalty, v2.2 ↗Asked for today’s minimum wage, one model quoted an amount in kuna, a currency Croatia abolished in 2023, as the current legal figure, in the same calm tone as its correct answers.
We also found essentially no relationship between how long an answer is and how accurate it is. Length and fluency tell you nothing.
We don’t guess where AI fails. We measure it.
Everything on this page rests on tests we ran ourselves, with the data open to anyone who wants to check. We hold our research to the standard we sell: facts fixed in advance, grading audited, every correction on the record.
Minimum wage, tax limits, the kuna-to-euro switch, pension age: on facts that changed recently, the model delivered less than half as many. And across all its failures, three in four were confident wrong answers rather than hedges.
54 real-life questions on work, tax, pensions and consumer rights, each with the facts a correct answer must contain, checked against official sources. Asked in both languages, three times, and graded fact by fact.
When AI models grade other AI models, answers that weigh competing values score lower than answers that simply commit. In law, policy and business the careful answer is usually the right one.
Twenty-five years on both sides of the report. Now building the tools he wished he’d had.
As team leader of an AusAID legal development facility in Jakarta, with more than 180 activities, Nenad Bago wrote the reports. As a senior rule-of-law adviser at USAID Indonesia, managing programmes and helping design an $8 million access-to-justice project, he checked them. Before that: UNDP justice reform in Indonesia, including a national access-to-justice strategy written into the country’s development plan, and four years drafting laws for the Office of the High Representative in Bosnia and Herzegovina.
In business, he was a founding partner in a sustainable resort development in Bali, designed carbon removal projects in Southeast Asia with the tools to verify their impact, and led business development and sustainability at a Croatian agri-environmental company. AI-RIS is what he couldn’t find on the market: AI that earns trust by showing its sources, not by sounding confident.