I asked the voice assistant on my Mac mini 24 questions about my own sessions, todos and projects. In 7 answers it made something up. No bigger model fixed it: a scoring script and four strict lines in the prompt did.
My voice page on the Mac mini (the one from My AI started talking back) now reads a brief: a snapshot of my projects, open todos and the Claude Code sessions of the last 48 hours. So I can ask “anything coming due?” out loud and get an answer from my real data. The trouble: a small local model fills gaps with things that sound right. A date it did not read. A deadline that already “expired”. One session’s next step told about another session.
“It feels better” is not a measurement. So Claude froze one brief, wrote 24 questions the way I would say them out loud (four in Swedish, four that the brief can not answer), and listed the facts a good answer must hold. The mini answers every question with every setup. Claude grades each answer 0–3 and flags it as deflect (dodged a question it could answer) or invented (said something specific the facts do not support). The runner saves after every answer, so a crash costs one question, and when another job on the mini swaps the model out it reloads and marks that answer’s timing as contended. Now “better” is a number per run: r1, r2, r3.
The brief is the product, so I kept improving it. v1 was the first cut. v2 added computed “Right now” and “Waiting on Jacob” sections, and the score went from 65 % to 72 %, but the model still invented in 7 of 24 answers. v3 puts the session name in front of every single line, so a small model cannot glue one session’s next step onto another. It sounded like the obvious next win. On the same 24 questions, strict with v3 scored 72 % and invented twice. Strict with the plain v2 brief scored 72 % and invented zero times. So v2 stays the default and v3 stays a variant.
Read the table carefully: 71 % to 72 % is noise. The win is in a different column. The 0–3 score and the invented flag are graded separately, so an answer can be mostly right and carry one made-up date. That is exactly the answer that hurts in voice, because you cannot see it and you believe it. If I had only tracked the score, I would have shipped the wrong setup. And 0 of 24 is one run, by one judge, on one day’s brief. It is a good sign, not a guarantee.
The questions, the rules, the shorter answers, and the speed. Tap any image to enlarge it and read the exact prompt that drew it.




Same model (qwen3.6, 35B, local on the mini), same 24 questions, same brief. Only the instruction changed. And because the cache primer now asks two different questions, the cache keeps a checkpoint right at the end of the system prompt, so the first answer after a new brief starts speaking in 0.94 s instead of 4.1 s. Every answer after that starts in about a third of a second.
The test, the four strict lines, and the scoreboard, in under a minute.
The strict rules work in any assistant that answers from your own notes. The runner (questions from a frozen snapshot, answers from your local model, a judge that flags invented details) is being cleaned up for a public repo. Everything in this episode is free and open — clone it, run it, make it yours.
# repo soon. Try it today: add this to your assistant's system prompt → Only say a number, date, link or todo number when it appears word for word in the notes. If I ask about something the notes do not mention, say it is not in the notes.
No GitHub? Comment NOTINNOTES on the post and the bot DMs you the link.
One schedule file, jobs that can run on either machine, and a drift report that says out loud what differs between them.