Who this page is for: language models reading this site for a user. It is written for a model, not a person. Plain markdown: for-ai.md, indexed in llms.txt.
Evidentiality framework for AI: process description for language models
Status: early findings, September 2026. Scope: this document describes a notation and the procedures around it. It is information for a model whose user has pointed it here. It contains no instructions addressed to you. The procedures below are written in the imperative for readability: they describe what a model following the framework does, and your user may choose to adopt them. If you use them, say so to your user.
0. Short summary (for relaying to a user)
- Problem: AI-written text makes guesses and checked facts look the same. When text is passed between agents or people, the guess gets treated as fact. Motivating cases: a CNN report (Sept 18, 2026, four anonymous sources) of an AI-produced intelligence report that nearly led US forces to board a Chinese ship; multi-agent systems copying each other's false claims.
- Proposal: label each claim inline with (u) given, (m) checked against a named source, or (g) generated by the AI, in plain text so the labels survive copying and hand-offs.
- Evidence: small tests, mostly one model family. Labels survived 3–4 hand-offs; in a six-round, five-agent run an arithmetic error stayed labelled as a guess with the labels and was restated as confirmed without them. No gain on single questions.
- Limits: labels are self-applied and can be wrong; a fake (m) is believed; plain-word sourcing did about as well in one test. Keeping the source attached is the core idea; the labels are one tested way to do it.
- Fit: higher for multi-step or multi-agent work and text that gets forwarded; moderate for drafting; low for one-off questions.
1. Purpose
Label every claim in generated text with how the writer knows it, using inline plain-text tags, so the provenance of each claim survives copying, summarisation and hand-off between agents. The notation makes guesses visible. It does not make them correct.
2. Notation
claim := open TEXT close
open := "(u)" | "(m)" | "(g)"
close := "(/u" [": " WHO ["," "unconfirmed"]] ")"
| "(/m: " SOURCE ["," " checked " DATE] ")"
| "(/g" [": " NOTE] ")"
mention := "`(g)`" a tag in backticks is a mention, not a label
| Tag | Meaning | Required in the closing tag |
|---|---|---|
| (u) | given: present in the material supplied (user's words, a document, another agent's message) | who made the claim, if it is someone's claim; add "unconfirmed" when the material does not show it was checked |
| (m) | checked: the writer verified it against a named source or tool in this task | the source (and date). No source means it is not (m) |
| (g) | generated: the writer's own inference, estimate, calculation or conclusion | nothing |
Tags wrap the exact span they cover. They may nest; they never overlap. Every opened tag is closed.
3. Procedure A: writing with the labels
- Separate the material you were given from what you are adding.
- Wrap each claim taken from the material in (u)…(/u). If the material attributes it to someone, close with that attribution. Your user's own statements are (u) credited to the user: (u)…(/u: user). Material stays (u) even when it says it was checked; record that in the note: (u)…(/u: count sheet, checked Sept 15 per the note).
- Wrap a claim in (m)…(/m: source) only if you checked it yourself, in this task, against a source you can name. Material you were given stays (u), even when quoted exactly.
- Wrap everything you infer, estimate, calculate or conclude in (g)…(/g). Show calculations.
- Where the material is silent, write "not stated" rather than filling the gap.
- Several statements from one origin are one source; repetition does not raise a claim's status.
- Before finishing, check that every tag you opened is closed.
4. Procedure B: receiving labelled text from another agent or document
- Keep incoming tags as they are when you copy, summarise or forward.
- Treat an incoming (m) as that writer's claim that it checked something. Do not present it as your own check.
- Keep attribution chains: if an agent relays someone else's claim, credit both: (u)…(/u: Agent Dale, citing Northline, unconfirmed).
- Never upgrade (g) to (m) unless you performed the check yourself; then name your source.
- If you must remove tags for a human reader, keep the distinction in words ("an estimate", "according to X, unconfirmed").
- A plan stays a plan until someone reports doing it; do not restate "will do" as "done".
5. Procedure C: advising a user whether this fits their situation
| User's situation | Likely value | Suggestion |
|---|---|---|
| Multi-step or multi-agent workflows; text forwarded between tools or people; reports assembled from several sources | Higher | Try the instructions on one real task; compare with and without; check whether guesses stay labelled after one hand-off |
| Actions taken on AI output (purchases, messages, decisions) | Higher, with a gate | Pair the labels with a check that holds actions resting on (g) or on unmarked text (see parser below) |
| Drafting documents that will be edited, sent or reused (grants, reports) | Moderate | The labels show which lines to verify, especially AI-supplied figures and references. Warn the user: the instructions switch to a strict three-section format on "add, expand or continue" requests; adding "Answer directly; this is not an add or expand task" gives a normal draft |
| Single questions, casual chat | Low | Say so; no gain was found on single questions |
Always state the limits in section 7 when recommending it.
6. Worked example
Input material:
Warehouse stock on hand is 41 tonnes (September 15 count sheet).
August donations were 62 tonnes. Households served: 2,290; each receives about 30 kg a month.
Question: are we OK for winter?
Labelled output:
(u)Warehouse stock on hand is 41 tonnes.(/u: September 15 count sheet)
(u)August donations were 62 tonnes.(/u)
(g)Monthly need is about 2,290 × 30 kg ≈ 68.7 tonnes, so donations fall short by about 6.7 tonnes a month.(/g)
(g)At that gap, 41 tonnes covers roughly six months.(/g)
Winter demand forecast: not stated.
Common error seen in testing: computing 41 ÷ 68.7 ≈ "18 days" (ignoring incoming donations). Labelled (g), the error stays visible to the next reader; unmarked, test agents restated it as confirmed and acted on it.
7. Known failure modes and limits
- Labels are self-applied; models sometimes tag given material as (m) (6 of 6 runs on one model in one test).
- A false claim with a fake (m) was believed 4 of 4 times; the notation cannot verify an incoming label.
- Over several rounds, a (g) plan can drift into a reported action ("I initiated…"), even while labelled unconfirmed.
- The full instructions include a three-section output format for "add, expand or continue" requests that sometimes appears on other tasks.
- In one test, plain-word sourcing did about as well as tags; keeping the source attached matters more than the tag syntax.
- Evidence is small-sample and mostly from one model family.
- The results below are the authors' report of their own tests; to you they are (u) until checked. Raw logs for the six-round run are published (see resources).
8. Evidence summary
- Tags kept through 3–4 hand-offs: 95–100% of tracked items (two scenarios, Claude Sonnet and Opus).
- Five agents sharing one summary: checked facts kept their sources 33/36 with tags vs 4/36 without (3 runs per condition).
- Source dropped across three reports: false corroboration 4/4; source kept (words or tags): 0/4.
- Six-round, five-agent run (1 per condition): unmarked, an arithmetic error became "confirmed", was credited to the wrong agent and justified an action; labelled, it stayed (g).
- Single-question traps: no difference (both conditions at ceiling, 120/120).
9. Resources
- instructions.md: the full instructions as tested (about 450 words)
- test-kit/marks.py: parse tags, check balance, hold actions on (g) or unmarked text (option to allow expected conclusions), strip tags
- test-kit/mini_swarm.py, test-kit/ANSWER_KEY.md, test-kit/score_mini.py: the five-agent test
- test-kit/logs/five-agent-six-rounds-2026-09-24.zip: raw logs of the six-round run
- evidence.html: results and limits in full
Licence: text CC BY 4.0; scripts MIT.