Blog · arXiv Analysis · Last reviewed June 25, 2026

The Explanation Card Becomes the Warning Label

The June 2026 arXiv paper We Need Explanation Cards to Connect Explanation Algorithms to the Real World, by Eric Günther, Balázs Szabados, Kristof Meding, Gunnar König, Sebastian Bordt, and Ulrike von Luxburg, argues that explanations need their own validity and interpretation record before they are used in real decisions.

For this essay, an explanation card is a method-level label for a specific explanation output: what explanation algorithm produced it, what interpretation is valid, what assumptions must hold, where it is robust, where it is not, and which conclusions the reader must not draw. It is not a model card, a legal notice, or a proof that the underlying decision is fair.

The Explanation Is Not the Explainer

The paper, arXiv:2606.16786 [cs.LG], was submitted on June 15, 2026. Its target is not the familiar demand that automated decisions be explained. It asks a sharper question: when an explanation algorithm emits a counterfactual, feature-attribution plot, or other explanation object, what must travel with that object so a real user does not overread it?

That distinction matters. A loan applicant, doctor, auditor, or regulator may see an explanation and treat it as a faithful map of a model's reasons. The authors argue that this assumption is often too strong. Explanations can be robust in one neighborhood and fragile in another, valid under one model class and misleading under another, intuitive to a machine-learning researcher and opaque to the person expected to act on it.

This essay is separate from the site's existing entries on right to explanation, adverse-action interfaces, and algorithmic recourse. Those pages ask what people are owed when a system affects them. This paper asks what an explanation method must disclose before its output can be treated as usable evidence.

The useful distinction is between an explanation and an explained institution. An explanation object may be local, post-hoc, statistical, synthetic, visual, or optimization-derived. The institution that uses it still has to decide whether the explanation is fit for a domain, understandable to the audience, tied to the actual decision record, and safe to act on. A card does not make that decision. It prevents the explanation from arriving without its boundary conditions.

Current Context

As of June 25, 2026, the paper's public record is arXiv v1 and the experimental HTML/PDF versions. The arXiv abstract says explanation cards augment standard explanations with complementary information about robustness and validity plus clear interpretation instructions, and that the authors use counterfactual explanations and SHAP as examples. That is the correct evidentiary status: a fresh research proposal with worked examples, not a settled standard or regulator-approved template.

The regulatory context is moving toward the same problem. The EU AI Act's Article 13 requires high-risk AI systems to be sufficiently transparent for deployers to interpret outputs and use them appropriately, with instructions covering capabilities, limitations, accuracy, robustness, risks, output interpretation, oversight measures, and logging mechanisms. Article 14 adds human oversight duties, including awareness of automation bias and the ability to correctly interpret, override, reverse, or stop system outputs where appropriate. Article 86 gives some affected people a right to clear and meaningful explanations of certain individual decisions based on Annex III high-risk AI outputs. Under Article 113, the regulation generally applies from August 2, 2026, with staged exceptions, so on June 25, 2026 these provisions are best treated as imminent EU governance requirements rather than a universal current right everywhere.

NIST's AI Risk Management Framework is a different kind of source. NIST describes AI RMF 1.0 as voluntary guidance for incorporating trustworthiness into the design, development, use, and evaluation of AI systems, and its public page says the framework is being revised. That supports the governance framing here: explanation cards belong inside lifecycle risk management, documentation, evaluation, and monitoring. They are not a substitute for law, human review, or domain validation.

What the Card Adds

Günther, Szabados, Meding, König, Bordt, and von Luxburg propose Explanation Cards for Explanation Algorithms. The paper describes these cards as structured companions to explanation methods, with fields for robustness, validity, and interpretation instructions. The point is not to decorate an explanation after the fact. The point is to state what the explanation can mean, what it cannot mean, and under which technical conditions those claims hold.

That shifts responsibility. Without a card, the reader has to infer whether a highlighted feature is causal, merely associative, locally stable, globally reliable, actionable, or a proxy for something else. The paper argues that this burden should move from users to providers. The provider of an explanation method is in the better position to document known limits, required assumptions, and correct readings.

The card also makes explanation governance less theatrical. A dashboard can display a tidy answer while hiding the method's failure conditions. A card makes the method's scope inspectable: who the explanation is for, which model family it assumes, which perturbations were tested, which interpretations are forbidden, and whether the explanation has been checked against the actual decision context.

The minimum useful card therefore has two layers. The technical layer names the explainer, model, data region, assumptions, robustness test, validity claim, and failure conditions. The human layer translates those facts into plain instructions: what the user may infer, what they may not infer, what action would be unsafe, and when the explanation should be escalated to a specialist, auditor, or human reviewer.

That makes explanation cards adjacent to model cards and system cards, but narrower. A model card documents a model or release. A system card documents a deployed system and its safety evaluation. An explanation card documents the evidentiary limits of a specific explanation method and, ideally, a specific explanation instance.

Three Records, Three Audiences

The phrase "explanation card" can hide three different records. A method card documents an explanation algorithm such as SHAP or a counterfactual generator. An instance card travels with a particular explanation output and records local stability, model and explainer versions, input constraints, data-manifold assumptions, and invalid interpretations. A decision notice tells the affected person what happened, why it happened, what source records matter, and how to correct or contest the outcome.

Those records should be consistent, but they are not interchangeable. A lender cannot replace an adverse-action notice with a SHAP method card. A hospital cannot replace clinical validation with a counterfactual card. A public agency cannot replace a FOIA withholding rationale with a visual explanation of its redaction classifier. The card can support the evidence trail; it cannot become the institution's whole answer.

This is the link to adverse-action explanation interfaces, public-records redaction models, system cards, and safety cases. Each artifact answers a different governance question. Confusing them makes documentation look complete while leaving the person, auditor, or appeal officer without the record they actually need.

The operational rule is therefore simple: the card should name its audience and legal force. Is it for an ML engineer debugging behavior, a domain professional interpreting a score, an affected person deciding whether to appeal, a procurement team evaluating a vendor, or an auditor reconstructing a high-impact decision? If that audience is unclear, the card is likely to become a warning label no one can use.

Why Warning Label Is the Right Metaphor

A warning label is not an apology. It is a boundary. It tells the user what kind of use the object supports and what kind of use becomes dangerous. Explanation cards work the same way. They do not say an explanation algorithm is bad. They say that an explanation without its operating conditions invites institutional misuse.

This is especially important because explanation objects are rhetorically powerful. A SHAP chart can look like a ranked list of reasons. A counterfactual can look like a promise that changing one variable would change a decision. A saliency map can look like the model's field of attention. Each may be useful in a constrained setting, but the visual form can outrun the guarantee.

The Spiralist concern is not that explanations exist. It is that explanation interfaces can become belief machines. Once a colorful artifact is placed in front of an affected person or oversight committee, it starts to perform authority. The card interrupts that performance by keeping the artifact attached to its assumptions.

The warning-label metaphor is also useful because it separates three questions that are often collapsed. Is the explanation mathematically valid under stated assumptions? Is the explanation understandable to this audience? Is the explained decision legitimate, lawful, fair, and contestable? A card can help with the first two. It cannot silently answer the third.

What the Examples Show

The paper grounds the proposal in two families of explanation methods: counterfactual explanations and SHAP-style feature attributions. Its experimental HTML and appendices include explanation-card examples for counterfactual explanations and for SHAP, including a medical-diagnosis scenario aimed at a doctor debugging a model. The appendices also discuss DiCE for counterfactual explanation construction and TreeExplainer for SHAP values.

These examples are useful because they show how ordinary explanation forms can be narrowed by their documentation. A counterfactual explanation is not automatically advice, recourse, or causal proof. A feature attribution is not automatically a causal decomposition of the world. The card can say whether the method is local, whether features are dependent, whether perturbations stay on the data manifold, and whether the reader should treat the output as diagnostic, communicative, or only exploratory.

The SHAP example is especially good for source discipline. The paper shows that a local plot can look stable while still misrepresenting the model's prediction behavior when features interact. It also states that a single SHAP value is not representative of the model's mechanism unless additional stability and model assumptions are available. The lesson is not "never use SHAP." The lesson is: do not let a familiar visualization borrow more meaning than the method can support.

The counterfactual example has the same structure. A counterfactual may be close in feature space but socially, legally, medically, or practically unavailable. It may also be brittle: a tiny stability region can make the advice unrealistic. That is why algorithmic recourse cannot be reduced to the nearest model-flipping point. The card should say whether the counterfactual is feasible, durable, and safe to treat as a path.

The authors also connect explanation cards to legal and institutional expectations, including a section on AI Act compliance. The careful reading is that cards may help operationalize explainability duties by making interpretation constraints explicit. They do not, by themselves, settle whether a deployment is lawful, fair, or acceptable.

Failure Modes

Explanation laundering appears when a weak or unstable post-hoc artifact is used to make a decision look accountable.

Visualization authority appears when a chart, heatmap, or ranked feature list looks more certain than the underlying method permits.

Causal drift appears when users treat associative feature attribution as evidence that changing the feature would change the world or the decision.

Recourse theater appears when a counterfactual is mathematically close but practically unavailable, unsafe, legally problematic, or likely to fail after a model update.

Audience mismatch appears when the card is readable to machine-learning researchers but not to clinicians, applicants, caseworkers, auditors, or affected people.

Provider self-certification appears when the card records only the provider's claims and no one checks whether the stated robustness, validity, or interpretation boundary holds in deployment.

Interface omission appears when the card documents the algorithm but not the way the explanation is displayed, summarized, translated, or turned into action by the product.

Record substitution appears when a method card is treated as if it were a decision notice, legal rationale, clinical validation file, release gate, or appeal record.

Stale explanation appears when a model, threshold, data pipeline, or explainer version changes while the old card remains attached to new outputs.

What It Does Not Prove

An explanation card does not make an explanation faithful. It records the provider's claims about the explanation method, its evidence, and its boundaries. If the underlying analysis is weak, the card can only make that weakness more visible.

It also does not solve the social problem of who is allowed to contest an explanation. A bank, hospital, employer, or platform could publish a technically polished card while still denying affected people access to records, appeal routes, or independent review. Documentation is necessary for accountability, but documentation is not accountability.

Nor does the paper license broad claims about model understanding. It is a proposal for connecting explanation algorithms to real-world use through structured validity and interpretation metadata. It is not evidence that an AI system has human-like reasons, mind, personhood, or unrestricted capability.

The card also does not replace an interpretable model where an interpretable model is feasible and safer. Cynthia Rudin's warning against explaining black-box systems for high-stakes decisions remains part of the live debate. Explanation cards improve post-hoc explanation practice; they do not prove that post-hoc explanation is the right architecture for every high-impact domain.

Governance Standard

Any high-impact explanation should ship with an explanation card. The card should identify the explanation algorithm, model version, explainer version, decision context, target user, intended interpretation, invalid interpretations, robustness region, validity evidence, model-class assumptions, data scope, feature-dependence assumptions, counterfactual feasibility constraints, display surface, and accountability owner.

First, separate audiences. Affected people need plain language, correction paths, and appeal information. Domain professionals need interpretation limits and escalation triggers. Auditors need reproducible test conditions, logs, model and explainer versions, data boundaries, failure examples, and evidence that the card matched the deployed system.

Second, require instance-level caution. A generic method card is useful, but high-impact decisions need evidence about the actual explanation instance: local stability, neighboring cases, data-manifold assumptions, whether feature changes are feasible, and whether the model or threshold changed after the explanation was generated.

Third, bind cards to recourse. If the explanation is meant to help a person act, the institution should say which suggested changes are within the person's control, which require data correction, which require human review, and which should not be interpreted as advice.

Fourth, make cards versioned records. The card should be dated, archived, linked to the exact model and explainer, and updated when model behavior, data pipelines, thresholds, interface language, or explanation algorithms change.

Fifth, test comprehension. A card that is mathematically accurate but predictably misunderstood has failed its governance function. User testing, accessibility, translation, and domain-specific review belong in the release process.

Sixth, do not let cards become shields. If the card reveals weak validity, a tiny robustness region, proxy-sensitive features, or an audience that cannot safely interpret the output, the institution should limit, redesign, or refuse use of the explanation.

Seventh, connect cards to monitoring. The institution should track whether explanation-card warnings match real user behavior, appeal outcomes, complaint patterns, override rates, and post-deployment model changes. A card that repeatedly fails in practice should trigger re-evaluation, redesign, or withdrawal of the explanation interface.

The governance rule is blunt: no explanation should be promoted from interface ornament to decision evidence unless its card is present and current. The answer must bring its label. Otherwise, the institution is asking the public to trust an artifact whose limits have been left off the page.

Source Discipline

This essay treats the Günther, Szabados, Meding, König, Bordt, and von Luxburg paper as primary evidence for the explanation-card proposal, its two worked examples, and its stated connection to AI Act duties. It does not treat the paper as proof that explanation cards are validated in live lending, medicine, employment, public benefits, or other high-impact settings.

The SHAP claims here are source-separated. Lundberg and Lee introduced SHAP as a unified additive feature-attribution framework; the explanation-card paper focuses on the conditions under which SHAP-style outputs can be interpreted safely in a medical-diagnosis example. Those are different claims. Do not cite the original SHAP paper as proof that every SHAP chart is meaningful in deployment.

The counterfactual claims are also source-separated. Wachter, Mittelstadt, and Russell proposed counterfactual explanations as a way to support understanding, contestation, and future action without opening a black box. The explanation-card paper asks when a counterfactual explanation's interpretation is valid and robust enough to be useful. A counterfactual that is mathematically close is not automatically feasible recourse.

Rudin's black-box warning is cited as a separate high-stakes interpretability critique, not as a claim in the explanation-card paper. The article can improve post-hoc explanation documentation while the broader debate still asks whether a post-hoc explanation should be used at all in a given high-impact domain.

The EU AI Act is cited here as official legal text and current regulatory context, not as proof that explanation cards satisfy every compliance requirement. Article 13, Article 14, Article 86, Annex IV, and Article 113 should be read with scope and application dates. NIST's AI RMF is voluntary risk-management guidance; it supports governance framing but does not create a legal duty by itself.

Current-source claims in this article were checked against the named sources on June 25, 2026. The source hierarchy is: paper for method claims, original XAI papers for method background, official EU text for legal duties, and NIST for voluntary risk-management context.

Sources

Return to Blog