Beyond "Made with AI": Visualize Evidence, Not Authorship

Beyond "Made with AI": Visualize Evidence, Not Authorship

A "Made with AI" label answers one question:

Was AI involved in producing this?

But that is not necessarily the question a reader cares about most.

When I read a technical article, research summary, news report, or AI-generated explanation, what I really want to know is:

Which parts of this can I actually verify?

That distinction is the starting point of a new paper, "Beyond 'Made with AI': Visualizing Provenance Density to Mitigate the Transparency Penalty." The work was submitted to arXiv in September 2026 by researchers affiliated with the University of Tokyo, KAIST, Georgia Institute of Technology, and Sony CSL Kyoto.

The paper proposes Provenance Density: instead of treating AI authorship as the primary transparency signal, show readers how much of the generated text is supported by external evidence.

And the interesting part is that the researchers did not simply argue for the idea. They tested both the underlying metric and how people react to the visualization.

The Fluency Trap

Generative AI has made polished writing extremely cheap.

That creates a problem.

For a long time, fluent and well-structured writing could act as a weak signal of competence. Writing something coherent required knowledge, education, editing, and time.

LLMs changed the economics.

A system can now produce something that sounds authoritative without necessarily having a reliable relationship with the truth.

The paper calls this the Fluency Trap.

People can trust a fluent hallucination because it reads well. At the same time, they may become more skeptical of accurate information simply because it carries an AI-generated label.

That creates a strange situation:

Fluency can increase trust even when it should not, while AI disclosure can decrease trust even when the underlying information is correct.

The researchers tested this in their user study.

In the control condition, participants had no additional signal. They rated hallucinated summaries at 6.28/10 and ground-truth summaries at 5.78/10.

The difference was not statistically detectable (p = .43).

In other words, participants were not reliably distinguishing the fabricated material from the factual material based on the text alone.

The problem with "Made with AI"

The obvious response is disclosure.

Put a label on it:

AI-generated content

Transparency achieved.

Except the label tells you something about the production process, not the truth of the individual claims.

The study found that the binary label changed how participants evaluated the content:

The binary label therefore improved discrimination compared with the control condition, but the effect was still relatively blunt.

The researchers describe this broader phenomenon as a Transparency Penalty: disclosure can affect perceived credibility without telling the reader which specific claims are actually supported.

That is the problem Provenance Density tries to address.

So what is Provenance Density?

The basic idea is simple:

Don't just tell me that AI wrote this. Show me how much of what it says is supported.

The paper formalizes this as:

D(T) = (1 − P_int) · tanh(
  (1/β) · Σ_c ( Σ_s w(s,c) )^λ
)

The notation looks intimidating, but the architecture is easier to understand.

The score combines three broad ideas:

1. External evidence
2. How strongly the evidence matches each claim
3. Whether the model remains internally consistent

That last part turns out to be particularly important.

The Consistency Veto

The most interesting component is what the paper calls the Consistency Veto.

Instead of generating an answer once, the audit generates five stochastic samples.

An NLI model then compares the samples to estimate how consistently the system produces compatible answers.

If the model keeps producing materially contradictory answers, the Provenance Density score is penalized.

Formally, the paper defines:

P_int = 1 − ρ_consistent

and uses:

1 − P_int

as a gate on the final score.

So a response can have plenty of retrieved sources and still receive a low score if the model cannot maintain a consistent answer.

There is an important limitation here:

Consistency is not truth.

A model can consistently repeat the same false answer.

The paper explicitly acknowledges this. The Consistency Veto is designed to detect instability, not magically establish factual truth.

Evidence quality matters too

The second major component is the weighting of evidence.

The paper defines:

w(s,c) = Reputation(s) · (MatchRatio)³

where s is a source and c is a claim.

The important detail is the cubic relevance term.

A source that merely shares some keywords with a claim should not contribute much.

For example:

MatchRatio = 0.5

0.5³ = 0.125

So weak semantic overlap gets heavily suppressed.

The reference implementation also uses domain reputation priors:

• 1.0 for high-trust sources such as government, educational, Wikipedia, major news, and scientific domains
• 0.1 for lower-trust sources such as Reddit, Quora, Medium, and X/Twitter
• 0.5 for other domains

This is intentionally simple rather than pretending that domain reputation can perfectly determine whether a source is trustworthy.

It is a prior, not a truth oracle.

The surprising result: more evidence is not necessarily better

This is where the paper becomes much more interesting.

You might assume that the main job of Provenance Density is simply:

Find more sources → verify more claims → detect hallucinations.

The technical audit says it is not that simple.

The researchers evaluated 200 samples using a mixture of TruthfulQA and dynamic FreshQA-style questions.

The results were:

Evidence density by itself was approximately chance overall.

On the dynamic subset, it was actually anti-discriminative.

That makes sense once you think about it.

A search engine can find lots of sources supporting a wrong answer.

If a misconception is widespread, retrieval can happily return pages repeating the same misconception.

And when information is changing rapidly, multiple sources can disagree while all appearing authoritative.

So:

More citations ≠ more truth.

The retrieval layer needs another signal.

Why the Consistency Veto performed better

The Consistency Veto performed particularly well on the dynamic subset, reaching an AUC of approximately 0.92.

But this number needs a very large asterisk.

The FreshQA subset contained only 50 items, of which just 3 were labeled hallucinations.

That makes the estimate extremely sensitive to a tiny number of examples.

The paper itself says the dynamic-query result should be interpreted as suggestive rather than definitive.

This is an important distinction.

It would be easy to write:

"The system detects hallucinations with 92% AUC."

That would be misleading.

A more accurate statement is:

In this particular small dynamic-query evaluation, the Consistency Veto produced an AUC of about 0.92, but the subset contained only three hallucinated examples.

That is still interesting.

It is just not proof of a production-ready hallucination detector.

The Indonesia capital example

The paper gives an interesting example involving Indonesia's capital transition.

A retrieval system can find credible-looking evidence supporting competing answers because the underlying information is time-sensitive.

That creates a problem for a simple evidence counter.

Both answers might appear to have plenty of supporting material.

The consistency component provides another question:

Is the model itself stable when answering this question?

If different stochastic generations produce conflicting answers, the system can reduce confidence in the overall provenance signal.

This is particularly relevant for:

• breaking news
• elections
• stock prices
• policy changes
• scientific developments
• rapidly changing technical information
• facts near a model's knowledge cutoff

In these situations, uncertainty is not necessarily a failure.

Sometimes the world itself is changing.

Does the interface actually help people?

The researchers then tested the visualization with 81 participants.

The interface displayed a provenance indicator such as:

5/5 Claims Verified

The idea was to give readers a claim-level evidence signal instead of a generic AI warning.

The result was a +4.15 point discernment gap between factual and fabricated content, with a reported effect size of d = 1.82.

That compares with a +1.66 gap for the binary AI label.

The difference is substantial.

But there is an important methodological detail here.

The Oracle problem

The user study did not run participants against a completely live provenance pipeline.

Instead, it used an Oracle / Wizard-of-Oz protocol.

The researchers already knew which summaries were grounded and which were fabricated.

They then displayed the corresponding provenance signal:

Grounded summary
        ↓
High-density indicator

Fabricated summary
        ↓
Null-density indicator

That was deliberate.

The researchers wanted to answer:

If the provenance signal is correct, does showing it actually help humans distinguish truth from fabrication?

The answer appears to be yes.

But that is a different question from:

Can a real-world retrieval and verification pipeline reliably generate the correct signal?

The technical audit is supposed to address that second problem, and its results are much less impressive.

This distinction is probably the most important caveat in the entire paper.

The cold-start problem

There is another interesting consequence.

The system is deliberately conservative.

In the technical audit:

• TruthfulQA/static knowledge had an average D(T) around 0.79
• Dynamic/FreshQA knowledge averaged around 0.64

That means newer or less-settled information naturally receives weaker provenance.

This can be useful.

But it can also produce false alarms.

Imagine a major event happens today.

There may be only a handful of reliable sources available.

A provenance system could correctly recognize that the evidence is immature while the underlying event is completely real.

So:

Low provenance does not mean false.

It may simply mean:

"We don't have enough independent, reliable evidence yet."

That is a very different statement.

The latency problem

There is also a practical problem.

The audit took approximately 17 to 26 seconds per document.

That is nowhere near the latency users expect from ordinary text generation.

A system that verifies claims by:

1. generating the response,
2. splitting it into claims,
3. searching for evidence,
4. scoring sources,
5. generating multiple samples,
6. running NLI comparisons,
7. aggregating everything,

is doing significantly more work than simply generating the answer.

That makes synchronous verification difficult.

The paper therefore discusses asynchronous patterns such as background checking or deeper verification modes.

That sounds much more realistic.

For example:

Generate response
      ↓
Show response immediately
      ↓
Background verification
      ↓
Update provenance indicators
      ↓
Allow user to inspect evidence

That is probably a better product model than blocking every response for 20+ seconds.

This is not a replacement for fact-checking

There is a temptation to turn Provenance Density into another magic number:

D(T) = 0.87 → trustworthy

That would completely miss the point.

A provenance score is better understood as an epistemic signal.

It tells us something about the evidence supporting a generated answer.

It does not establish that the answer is true.

A high score can still happen when:

• many sources repeat the same misconception,
• authoritative sources contain an error,
• retrieval finds outdated information,
• the model consistently repeats a false answer,
• the underlying event has changed,
• the retrieval system misses the best evidence.

Likewise, a low score does not necessarily mean the answer is false.

It may mean the evidence is new, sparse, contradictory, or difficult to retrieve.

That makes Provenance Density much more interesting as a calibration mechanism than as a binary truth detector. The same instinct applies to publishing: a human review gate checks the evidence behind each claim rather than trusting the fluency of the draft.

From AI disclosure to evidence disclosure

This is where I think the paper's strongest idea lies.

The question should not necessarily be:

"Was this made by AI?"

It should increasingly become:

"What evidence supports this claim?"

These are different dimensions.

An entirely human-written article can contain false information.

An AI-generated summary can accurately describe a well-established fact.

A binary authorship label cannot distinguish those cases.

A claim-level provenance system at least tries to expose the missing layer:

Who produced it?
        +
What supports it?
        +
How consistent is the generation?
        +
How mature is the evidence?

That is much closer to what a reader actually needs when deciding whether to trust information.

What this means for AI products

If this approach survives further testing, I can see several practical applications.

AI assistants

Instead of:

"This answer was generated by AI."

show:

12/14 claims supported

with expandable evidence for each claim.

Search engines

Instead of simply ranking pages, expose evidence quality around generated summaries.

Enterprise AI

For internal copilots, provenance could show whether an answer came from:

• internal policy documents
• approved databases
• external sources
• conflicting documents
• weak or incomplete evidence

News and research tools

Dynamic information could be explicitly marked as:

Evidence still developing

rather than simply labeling the article "AI-generated."

AI governance

Regulators could eventually distinguish between:

authorship transparency

and

epistemic transparency.

Those are not the same thing.

And there is a security angle

This matters beyond hallucinations.

Any provenance system eventually becomes a target.

If users learn that:

High provenance = trusted
Low provenance = suspicious

then attackers have a new objective:

Manipulate the provenance signal.

That could mean:

• publishing large amounts of low-quality supporting content,
• creating authoritative-looking websites,
• poisoning retrieval results,
• exploiting source-reputation assumptions,
• manipulating search indexes,
• generating mutually reinforcing fake sources.

In other words, the provenance layer itself becomes part of the attack surface.

The paper acknowledges adversarial gaming as an important direction for future work.

And this is exactly why I would be cautious about turning a provenance score into a simple "truth score."

The bigger idea

The paper is not really saying:

"Made with AI" labels are useless.

The argument is more subtle.

Authorship disclosure and evidence disclosure answer different questions.

A label tells me:

How was this produced?

Provenance tells me:

What supports this?

For an AI-generated paragraph, both can be useful.

But if the objective is trust calibration, evidence is much closer to the actual decision a reader needs to make.

That is the interesting shift.

We have spent years trying to detect whether something was written by a machine.

Maybe the more important problem is figuring out whether what the machine — or the human — wrote is actually supported.

My take

I like the direction of Provenance Density, but I would not call it a solved hallucination-detection system.

The technical results are mixed.

Evidence density alone performs poorly. The Consistency Veto looks promising, especially on dynamic queries, but its strongest result comes from a very small hallucination subset.

The human study is much more encouraging, but it tests an idealized signal rather than a fully autonomous verification pipeline.

So the real contribution is not:

"We invented a better hallucination detector."

It is:

"Maybe transparency should expose evidence instead of merely exposing authorship."

That is a much stronger and more defensible idea.

And if AI-generated text is going to become indistinguishable from human writing, this distinction will become increasingly important.

"Made with AI" tells us who — or what — produced the words.

Provenance tries to tell us why we should believe them.

For the usage side of the same question — what people actually do with AI systems once they are deployed — see how AI use looks in the wild.

---

References

• Zhang, Q., Huang, Y., Lee, J., Starner, T., & Rekimoto, J. (2026). Beyond "Made with AI": Visualizing Provenance Density to Mitigate the Transparency Penalty. arXiv:2609.03460.
• Zhang, Q. et al. (2026). Provenance Density: IJCAI-ECAI 2026 Open-Science Release.
• TruthfulQA — Lin et al. (2022).
• FreshQA — Vu et al. (2024).

Paper: https://arxiv.org/abs/2609.03460

Open-source implementation and research data: https://github.com/artisticsciencex/ijcai-ecai-2026-provenance-density

Sources

• arXiv: Beyond "Made with AI": Visualizing Provenance Density to Mitigate the Transparency Penalty
• GitHub: Provenance Density open-source implementation and research data