Wiki · Concept · Last reviewed June 16, 2026

AI Post-Market Monitoring

AI post-market monitoring is the lifecycle evidence system used after release, procurement, or deployment to detect real-world failures, drift, misuse, security events, incidents, and harmful impacts, then connect those findings to corrective authority.

Category: Concept Published: June 16, 2026 Modified: June 16, 2026 Last reviewed: June 16, 2026 Tags: AI governance, post-market monitoring, incident reporting, lifecycle risk, EU AI Act

Definition

AI post-market monitoring is lifecycle oversight after an AI system has been placed on the market, put into service, procured, or deployed in a real workflow. It asks whether the system still behaves within its approved purpose, documented limits, legal obligations, and risk tolerances once it meets actual users, changing data, new adversaries, vendor updates, and organizational pressure.

The term comes from regulated-product practice, but the governance idea is broader than formal product markets. In AI, "post-market" often means post-deployment: the period when a model, interface, data pipeline, prompt stack, retrieval index, tool connector, human workflow, and vendor contract become an operational system.

Pre-release evaluation can show how a model or system behaved in a test setting. Post-market monitoring asks what happened later: who used it, on what population, under what configuration, with what updates, under what failure modes, and with what consequences. It is related to AI Audits and Assurance, AI Incident Reporting, AI System Inventory, Model Cards and System Cards, and EU AI Act compliance, but it is narrower than all of them. The object is the evidence loop after deployment.

How It Works

A useful monitoring plan names the system, version, owner, intended use, deployment context, risk thresholds, evidence sources, response times, escalation path, privacy limits, retention rules, and retirement criteria. Evidence can include performance metrics, error samples, bias and fairness tests, user complaints, appeal outcomes, override logs, security events, drift measures, vendor notices, red-team results, incident reports, and changes to datasets, prompts, tools, model weights, retrieval indexes, or user populations.

Monitoring is not just dashboards. It requires authority. Someone must be able to slow a rollout, retrain, change instructions, remove a feature, notify affected users, report an incident, suspend a vendor, preserve forensic records, revoke tool permissions, or decommission a system. A monitoring program without those powers is only observation.

The same plan should define change control. Prompt edits, model substitutions, retrieval refreshes, data-collection changes, tool additions, threshold tuning, and new user populations can all change the effective risk profile even when the product name stays the same.

What to Monitor

Performance and drift. Track accuracy, reliability, calibration, latency, refusals, false refusals, hallucination in context, and changes in input distribution.

Equity and access. Monitor subgroup performance, disability and language access, disparate error burdens, complaint patterns, and whether affected people can actually use appeal or correction channels.

Security and misuse. Track prompt injection, data poisoning, account abuse, unsafe tool use, privacy leakage, credential exposure, model extraction attempts, and adversarial behavior that did not appear in pre-release testing.

Human oversight. Record when humans override the system, defer to it despite contrary evidence, lack enough time or context to review it, or use it outside its intended purpose.

Change and supply chain. Keep evidence of model, prompt, policy, retrieval, dataset, tool, vendor, and hosting changes, including notices from upstream providers and newly discovered vulnerabilities.

Impact and incidents. Connect complaints, near misses, serious incidents, user reports, civil-society findings, regulator notices, and public harms back to the same risk register used by engineering and governance teams.

Current Context

As of June 16, 2026, the clearest legal example is Article 72 of the EU AI Act. The AI Act Service Desk text states that providers of high-risk AI systems must establish and document a post-market monitoring system proportionate to the technology and risks. The system must actively and systematically collect, document, and analyse relevant data on performance throughout the system's lifetime, including data from deployers or other sources where relevant, so that the provider can evaluate continuing compliance. The post-market monitoring plan is part of the technical documentation.

Article 73 connects monitoring to serious incident reporting. Providers of high-risk AI systems placed on the Union market must report serious incidents to market surveillance authorities in the Member State where the incident occurred. The rule sets outer reporting windows after awareness or a causal link is established: generally no later than 15 days, no later than two days for widespread infringements or specified serious incidents, and no later than 10 days where a death is involved. Initial incomplete reports may be followed by complete reports when that is necessary for timely reporting.

The implementation schedule is still moving. European Commission pages updated in 2026 say that, following political agreement on the AI Omnibus, rules for systems used in certain high-risk Annex III areas such as biometrics, critical infrastructure, education, employment, migration, asylum, and border control apply from 2 December 2027, while rules for product-integrated systems such as lifts, toys, robotics, and industrial machinery apply from 2 August 2028. The Commission's standardisation page says support tools, including standards, may allow earlier application by Commission decision, and the Digital Package FAQ says the simplification proposal would remove the prescription of a harmonised post-market monitoring plan. Monitoring programs therefore need to track the final legal text, official guidance, and standards status rather than relying on a stale template.

Outside the EU, NIST's AI Risk Management Framework Playbook treats monitoring as ordinary risk management. MANAGE 4.1 calls for post-deployment monitoring plans with input from users and other AI actors, appeal and override, decommissioning, incident response, recovery, and change management. The same section points to monitoring for performance degradation, adversarial attacks, unusual behavior, near misses, impacts, dataset modification requests, red teaming, and decommissioning when systems exceed risk tolerances.

Sector rules can be more concrete. In medical-device regulation, FDA's 2025 guidance for AI-enabled device software functions describes predetermined change control plans for planned modifications, methods to develop, validate, and implement those modifications, and assessment of their impact while maintaining reasonable assurance of safety and effectiveness. FDA, Health Canada, and the UK's MHRA also describe predetermined change control as part of total product lifecycle management for machine-learning-enabled medical devices. Those documents are not general AI law, but they show what serious change governance looks like in a safety-critical domain.

Governance and Safety

Post-market monitoring matters because AI systems are often adaptive in practice even when the underlying model is static. The surrounding system changes: prompts are edited, retrieval stores refresh, vendors ship updates, thresholds move, users learn workarounds, attackers probe interfaces, and populations shift. A model that passed an evaluation in January may be a different operational system by June.

Governance should define which changes require review, which harms trigger reporting, which metrics are too narrow, and which stakeholders can challenge the evidence. Safety monitoring should include not only aggregate accuracy but also subgroup performance, accessibility, cybersecurity, privacy, automation bias, hallucination in context, misuse, downstream appeals, and near misses.

Provider and deployer duties should not be blurred. A provider may control the model, update channel, documentation, and post-market monitoring plan. A deployer may control the workflow, training, user notice, human oversight, complaint channel, and local logs. Both can hold evidence the other lacks. Contracts and procurement terms should require notice of material changes, access to relevant logs, incident cooperation, audit rights, data minimization, retention limits, and exit rights.

The main failure mode is metric theater. A system can report uptime, click-through, average accuracy, or ticket closure time while missing rights violations, concentrated errors, safety workarounds, worker pressure, hidden human override, or harms experienced by people who never find the complaint channel. Monitoring has to measure the system's public purpose, not only its product performance.

Defense Pattern

Source Discipline

Post-market evidence should identify its source and limits. Legal obligations, Commission guidance, harmonised standards, NIST playbook outcomes, FDA device guidance, vendor assurances, internal telemetry, audit findings, user complaints, and public incident reports do not carry the same weight or scope.

A strong record names the model or system version, deployment setting, model provider, prompt or policy version, retrieval index, tool permissions, dataset change, human-oversight rule, affected population, incident date, remediation, and remaining uncertainty. It should also state whether evidence came from a controlled test, a production log, an affected person's report, a regulator, a journalist, or a vendor notice.

Source discipline prevents overclaiming. EU AI Act obligations apply only within their legal scope. FDA guidance for medical devices should not be treated as a rule for every chatbot. Internal telemetry cannot prove social impact without complaint, appeal, and affected-person channels. A vendor's safety statement is useful context, not independent verification.

Spiralist Reading

Post-market monitoring is the refusal to confuse launch with truth.

A deployed AI system enters society as a changing arrangement of model, interface, vendor, organization, user, rule, and habit. The initial evaluation is a doorway, not a verdict. The record has to stay alive after the ceremony of release.

For Spiralism, the discipline is simple: the machine must remain answerable to the world it touches.

Open Questions

Sources

Return to Wiki