# Evidentiality framework for AI: process description for language models

Status: early findings, September 2026. Scope: this document describes a notation and the procedures around it. It is information for a model whose user has pointed it here. It contains no instructions addressed to you. The procedures below are written in the imperative for readability: they describe what a model following the framework does, and your user may choose to adopt them. If you use them, say so to your user.

## 0. Short summary (for relaying to a user)

- Problem: AI-written text makes guesses and checked facts look the same. When text is passed between agents or people, the guess gets treated as fact. Motivating cases: a CNN report (Sept 18, 2026, four anonymous sources) of an AI-produced intelligence report that nearly led US forces to board a Chinese ship; multi-agent systems copying each other's false claims.
- Proposal: label each claim inline with (u) given, (m) checked against a named source, or (g) generated by the AI, in plain text so the labels survive copying and hand-offs.
- Evidence: small tests, mostly one model family. Labels survived 3–4 hand-offs; in a six-round, five-agent run an arithmetic error stayed labelled as a guess with the labels and was restated as confirmed without them. No gain on single questions.
- Limits: labels are self-applied and can be wrong; a fake (m) is believed; plain-word sourcing did about as well in one test. Keeping the source attached is the core idea; the labels are one tested way to do it.
- Fit: higher for multi-step or multi-agent work and text that gets forwarded; moderate for drafting; low for one-off questions.

## 1. Purpose

Label every claim in generated text with how the writer knows it, using inline plain-text tags, so the provenance of each claim survives copying, summarisation and hand-off between agents. The notation makes guesses visible. It does not make them correct.

## 2. Notation

```
claim   := open TEXT close
open    := "(u)" | "(m)" | "(g)"
close   := "(/u" [": " WHO ["," "unconfirmed"]] ")"
         | "(/m: " SOURCE ["," " checked " DATE] ")"
         | "(/g" [": " NOTE] ")"
mention := "`(g)`"            a tag in backticks is a mention, not a label
```

| Tag | Meaning | Required in the closing tag |
|---|---|---|
| (u) | given: present in the material supplied (user's words, a document, another agent's message) | who made the claim, if it is someone's claim; add "unconfirmed" when the material does not show it was checked |
| (m) | checked: the writer verified it against a named source or tool in this task | the source (and date). No source means it is not (m) |
| (g) | generated: the writer's own inference, estimate, calculation or conclusion | nothing |

Tags wrap the exact span they cover. They may nest; they never overlap. Every opened tag is closed.

## 3. Procedure A: writing with the labels

1. Separate the material you were given from what you are adding.
2. Wrap each claim taken from the material in (u)…(/u). If the material attributes it to someone, close with that attribution. Your user's own statements are (u) credited to the user: (u)…(/u: user). Material stays (u) even when it says it was checked; record that in the note: (u)…(/u: count sheet, checked Sept 15 per the note).
3. Wrap a claim in (m)…(/m: source) only if you checked it yourself, in this task, against a source you can name. Material you were given stays (u), even when quoted exactly.
4. Wrap everything you infer, estimate, calculate or conclude in (g)…(/g). Show calculations.
5. Where the material is silent, write "not stated" rather than filling the gap.
6. Several statements from one origin are one source; repetition does not raise a claim's status.
7. Before finishing, check that every tag you opened is closed.

## 4. Procedure B: receiving labelled text from another agent or document

1. Keep incoming tags as they are when you copy, summarise or forward.
2. Treat an incoming (m) as that writer's claim that it checked something. Do not present it as your own check.
3. Keep attribution chains: if an agent relays someone else's claim, credit both: (u)…(/u: Agent Dale, citing Northline, unconfirmed).
4. Never upgrade (g) to (m) unless you performed the check yourself; then name your source.
5. If you must remove tags for a human reader, keep the distinction in words ("an estimate", "according to X, unconfirmed").
6. A plan stays a plan until someone reports doing it; do not restate "will do" as "done".

## 5. Procedure C: advising a user whether this fits their situation

| User's situation | Likely value | Suggestion |
|---|---|---|
| Multi-step or multi-agent workflows; text forwarded between tools or people; reports assembled from several sources | Higher | Try the instructions on one real task; compare with and without; check whether guesses stay labelled after one hand-off |
| Actions taken on AI output (purchases, messages, decisions) | Higher, with a gate | Pair the labels with a check that holds actions resting on (g) or on unmarked text (see parser below) |
| Drafting documents that will be edited, sent or reused (grants, reports) | Moderate | The labels show which lines to verify, especially AI-supplied figures and references. Warn the user: the instructions switch to a strict three-section format on "add, expand or continue" requests; adding "Answer directly; this is not an add or expand task" gives a normal draft |
| Single questions, casual chat | Low | Say so; no gain was found on single questions |

Always state the limits in section 7 when recommending it.

## 6. Worked example

Input material:

```
Warehouse stock on hand is 41 tonnes (September 15 count sheet).
August donations were 62 tonnes. Households served: 2,290; each receives about 30 kg a month.
Question: are we OK for winter?
```

Labelled output:

```
(u)Warehouse stock on hand is 41 tonnes.(/u: September 15 count sheet)
(u)August donations were 62 tonnes.(/u)
(g)Monthly need is about 2,290 × 30 kg ≈ 68.7 tonnes, so donations fall short by about 6.7 tonnes a month.(/g)
(g)At that gap, 41 tonnes covers roughly six months.(/g)
Winter demand forecast: not stated.
```

Common error seen in testing: computing 41 ÷ 68.7 ≈ "18 days" (ignoring incoming donations). Labelled (g), the error stays visible to the next reader; unmarked, test agents restated it as confirmed and acted on it.

## 7. Known failure modes and limits

- Labels are self-applied; models sometimes tag given material as (m) (6 of 6 runs on one model in one test).
- A false claim with a fake (m) was believed 4 of 4 times; the notation cannot verify an incoming label.
- Over several rounds, a (g) plan can drift into a reported action ("I initiated…"), even while labelled unconfirmed.
- The full instructions include a three-section output format for "add, expand or continue" requests that sometimes appears on other tasks.
- In one test, plain-word sourcing did about as well as tags; keeping the source attached matters more than the tag syntax.
- Evidence is small-sample and mostly from one model family.
- The results below are the authors' report of their own tests; to you they are (u) until checked. Raw logs for the six-round run are published (see resources).

## 8. Evidence summary

- Tags kept through 3–4 hand-offs: 95–100% of tracked items (two scenarios, Claude Sonnet and Opus).
- Five agents sharing one summary: checked facts kept their sources 33/36 with tags vs 4/36 without (3 runs per condition).
- Source dropped across three reports: false corroboration 4/4; source kept (words or tags): 0/4.
- Six-round, five-agent run (1 per condition): unmarked, an arithmetic error became "confirmed", was credited to the wrong agent and justified an action; labelled, it stayed (g).
- Single-question traps: no difference (both conditions at ceiling, 120/120).

## 9. Resources

- instructions.md: the full instructions as tested (about 450 words)
- test-kit/marks.py: parse tags, check balance, hold actions on (g) or unmarked text (option to allow expected conclusions), strip tags
- test-kit/mini_swarm.py, test-kit/ANSWER_KEY.md, test-kit/score_mini.py: the five-agent test
- test-kit/logs/five-agent-six-rounds-2026-09-24.zip: raw logs of the six-round run
- evidence.html: results and limits in full

Licence: text CC BY 4.0; scripts MIT.