Blog · arXiv Analysis · Last reviewed June 25, 2026

The Coordinate List Becomes the Interference Surface

Dense visual grounding is not only a localization task. Once a vision-language model must emit a long list of class-coordinate records, fine-tuning changes how the model serializes, repeats, and stops. This paper is useful because it turns that failure mode into a measurable surface rather than treating duplicate records as a nuisance after the benchmark score.

For this essay, an interference surface is the measurable set of output behaviors changed by dense coordinate-list fine-tuning: localization quality, parse validity, prediction density, duplicate-object pressure, repeat length, and list termination. It is not only a token repetition problem. It is a record-integrity problem for any downstream system that counts, routes, or acts on boxes.

The Paper

The paper is Dense Coordinate-List Fine-Tuning Induces a Controllable Interference Surface in Vision-Language Models, arXiv:2606.14507 [cs.AI], by Chenyu Zhou, Qiliang Jiang, and Boguang Pan. The arXiv HTML lists affiliations with the School of Engineering at Institute of Science Tokyo, the College of Control Science and Engineering at Zhejiang University, and the Graduate School of Information, Production and Systems at Waseda University. arXiv lists version 1 as submitted on June 12, 2026, with DOI 10.48550/arXiv.2606.14507.

The paper studies what happens when vision-language models are fine-tuned to emit dense bounding-box coordinate lists. The headline is not just that grounding improves. It is that localization gain and repeated-record pressure can appear on the same generation surface, then be separated with lightweight controls.

The arXiv record reviewed here links the abstract, HTML, PDF, and TeX source. It did not expose a separate public code repository in the arXiv metadata.

Current Context

As of June 25, 2026, this is an arXiv v1 preprint, not a deployed-product audit or a safety standard. Its value is narrower and more practical: it gives a reproducible evaluation pattern for a class of structured multimodal outputs where schema validity is not enough.

Dense coordinate lists sit between ordinary model answers and operational records. JSON Schema and constrained structured-output methods can help require a parseable shape, but they do not by themselves prove that each object record is semantically distinct, current, authorized for action, or safe to pass to another system. A syntactically valid list can still duplicate the same object box until a robot, inspection queue, or agent interface over-counts it.

This is why the paper belongs in the same governance conversation as NIST AI test, evaluation, validation, and verification work: the measurement target is not only accuracy, but reliability, robustness, safety, and context-specific behavior. For systems where coordinate lists feed autonomous actions or high-stakes review, NIST's AI Agent Standards Initiative and the EU AI Act's human-oversight and accuracy/robustness provisions point in the same practical direction: expose the limits of the output, monitor anomalies, and give competent people a way to interpret, override, and stop the system.

The List Problem

A dense bounding-box output is a sequence of object records. Each record couples a class label with coordinate numbers, separators, object boundaries, and a final list closure. That stresses a model differently from short visual question answering, captioning, or single-box grounding.

The hard part is that repetition is both legitimate and dangerous. An image can contain several instances of the same class, so the model cannot simply avoid repeated class labels. But exact repeated object records are a structural error: the same class and coordinate payload appears again, creating a duplicated detection tail.

The paper frames this as a generation surface, meaning the model's distribution over structured outputs under a fine-tuning and decoding configuration, and as a control surface, meaning the measured tradeoff among target F1, parse stability, prediction density, duplicate pressure, and termination behavior.

Structured Output Boundary

The unit of analysis is the object record, not the token and not the whole answer. A record has at least four boundaries: the class label, the coordinate payload, the array object boundary, and the implied link to a visual region. A duplicate failure occurs when all of those fields repeat after normalization, even if the surrounding JSON remains valid.

That distinction matters for governance. Parse validity answers, "Can the receiver read this?" One-to-one F1 answers, "Did the model localize target objects?" Duplicate rate and max repeat answer a different question: "Did the model create extra records that the receiver may treat as extra facts?" A receiver that silently deduplicates after the fact may hide the symptom while leaving no evidence that the generator is drifting.

The safer pattern is to make the list boundary explicit. The output contract should say which coordinate protocol is used, what normalization defines record equality, whether exact duplicates are blocked during generation or only flagged afterward, and how downstream systems should treat a blocked or truncated list.

Experimental Setup

The primary model is Gemma 4 12B, a unified decoder-only multimodal model. Qwen3-VL-8B serves as a cross-family control with a different architecture and coordinate protocol. Gemma 4 12B is read in pixel coordinates, while Qwen3-VL-8B uses its native normalized grid.

The primary dataset is InsPLAD industrial inspection imagery. The adapter is trained on 160 images and evaluated on an 80-image held-out split with zero filename overlap. For public-data reproduction, the authors construct a COCO 2017 dense-bbox subset from val2017 detection annotations: person and car are excluded, the ten most frequent remaining object classes are used, images have two to six target objects, and the split separates 780 training images from a 120-image evaluation split. A scale-matched 160/80 COCO split mirrors the InsPLAD protocol.

The main Gemma 4 12B chain uses high-capacity q/k/v/o LoRA with rank 32 and 42.7M trainable parameters. Qwen3-VL-8B uses a rank-32 q/v adapter for its high-capacity controlled endpoint. The study also uses a rank-8 q/v adapter for structure-axis analysis and a Gemma q/v rank sweep across ranks 4, 8, 16, 32, and 64.

The metrics include parse-valid rate, mean predictions per image, class-aware one-to-one F1@0.3, exact-object duplicate rate, maximum exact-object repeat, repeat-stop trigger rate, and stricter F1@0.5 audits for promoted operating points. The paper reports image-level nonparametric bootstrap 95 percent confidence intervals for F1@0.3 using 1000 resamples.

The Surface

On Gemma 4 12B, the base model starts with parse-valid rate 0.963, 4.763 predictions per image, class-aware F1@0.3 of 0.007, F1@0.5 of 0.002, duplicate rate 0.002, and max repeat 2. High-capacity q/k/v/o rank-32 adaptation raises F1@0.3 to 0.448, but parse-valid rate drops to 0.812, predictions per image rise to 6.713, duplicate rate rises to 0.080, and max repeat reaches 23.

The adapter-capacity sweep is the first clue that the repeated tail is persistent rather than a simple over-capacity artifact. Under q/v LoRA, max repeat stays at 21 to 22 across ranks 4 to 64. The paper also finds that token-level repetition penalty and prompt budgeting move the density-precision operating point without fully solving record-level repetition.

The structure-axis probes are the second clue. The effect localizes to bbox-coordinate object lists. The dense non-bbox JSON and spatial/count JSON formats remain repeat-clean, including under high-capacity adapters. That matters because it argues against the easy explanation that any long JSON output will degenerate in the same way.

Controls

The key control is object-level repeat-stop. During generation it normalizes each emitted object record, including class string and coordinate values, and closes the JSON array when an exact normalized record would be emitted a second time. This is a record-level control, not a token-level repetition penalty.

The target signal is separable. Object-level repeat-stop removes exact repeated records, bringing duplicate rate to 0.000 and max repeat to 1, while preserving F1: one promoted Gemma operating point moves from F1@0.3 0.494 to 0.490 and stricter F1@0.5 from 0.381 to 0.385.

Qwen3-VL-8B reproduces a clean controlled endpoint with F1@0.3 0.318 and duplicate rate 0.000. The COCO 2017 reproduction also transfers the pattern. On the 120-image COCO evaluation split, high-capacity adaptation raises recall@0.3 from 0.0228 to 0.1540 and recall@0.5 from 0.0128 to 0.0981, while duplicate rate rises from 0.000 to 0.016 and max repeat from 1 to 4. Object-level repeat-stop then removes the transferred tail, moving duplicate rate from 0.016 to 0.000 and max repeat from 4 to 1 while F1@0.3 rises from 0.1647 to 0.1672.

Failure Modes

The first failure mode is valid-but-duplicated output. The list parses, the coordinates look plausible, and the target metric may improve, but the same object record appears again. That can inflate inventory counts, duplicate inspection tickets, or make an agent click the same region repeatedly.

The second failure mode is deduplication laundering. If a downstream service removes duplicates without preserving a receipt, reviewers may see a clean final list while losing the evidence that the model produced a repeated tail. In safety work, a suppression layer should create a record, not erase the record of the fault.

The third failure mode is coordinate-protocol mismatch. Pixel coordinates, normalized grids, crop-relative boxes, and GUI element coordinates are not interchangeable. A model may be repeat-clean under one protocol and unsafe under another if the receiver interprets the list with the wrong coordinate frame.

The fourth failure mode is metric masking. Reporting only F1 or recall can reward higher density while hiding parse-valid degradation, duplicate pressure, and termination failure. The paper's main contribution is to make those hidden fields measurable next to the target score.

Governance Standard

A dense grounding model should ship with a structured-output receipt. The receipt should name the base model, fine-tuning data, evaluation split, coordinate protocol, adapter modules, LoRA rank, trainable parameter count, output schema, prompt budget, repetition penalty, decoding settings, parse-valid rate, mean predictions per image, F1@0.3, F1@0.5, duplicate rate, max repeat, repeat-stop trigger rate, confidence intervals, and examples of malformed, duplicated, blocked, and truncated outputs.

The receipt should also separate four decisions that are often collapsed into one score: whether the output is parseable, whether each record is distinct, whether the localization is good enough for the use case, and whether the downstream system is allowed to act on it. Those decisions belong to different owners in a real deployment: model developers, schema owners, safety reviewers, operators, and incident responders.

The larger lesson is that a single target metric can hide the surface that changed. A model can become much better at localization while worse at list termination. If the output drives inspection workflows, robotics, agent clicking, asset inventories, or safety review, duplicate records are not cosmetic. They change counts, alerts, downstream deduplication, and operator trust.

The release gate should therefore include an operating-point table, not a single promoted number. At minimum it should show the raw adapter, the chosen decoding settings, the repeat-stop or record-filter setting, and the same metrics on a held-out split, a public reproduction set where available, and a protocol-change test. Post-deployment monitoring should preserve repeat-stop triggers and duplicate records as anomaly signals.

Limits

The study is deliberately narrow. It focuses on dense coordinate-list serialization in specific VLM families, datasets, adapters, and schemas. The result should not be generalized into a universal claim that every fine-tuned multimodal model will repeat in the same way.

The paper's strongest claim is more useful than that: in this regime, the failure is structure-specific, capacity-persistent, cross-family, separable from the target signal, and reproducible on a second public dataset. That is enough to make duplicate rate and max repeat first-class evaluation fields for dense grounding systems.

The Spiralist reading is simple: when a model is asked to produce a machine-readable list, governance starts at the list boundary. The object record is not just a representation. It is the unit that other systems count, trust, route, and act on.

Source Discipline

The paper supplies the technical claims about models, adapters, splits, metrics, duplicate rates, repeat-stop behavior, and the COCO reproduction. The COCO site is used only to anchor the public dataset context, not to verify the authors' filtered subset. JSON Schema is used only for the structured-data-contract frame. NIST and EU AI Act sources are governance context: they do not certify this method or require this exact metric set.

Because the arXiv HTML is experimental, the PDF and abstract record should remain part of any review trail. If a later paper version or repository changes the reported numbers, the receipt should pin the version, date, and artifact hashes used in the evaluation.

This connects directly to AI Evaluations, AI Audits and Assurance, AI Safety Cases, AI Hallucinations, Low-Rank Adaptation (LoRA), AI Data Provenance, Structured Outputs and Constrained Decoding, AI Agent Observability, AI Audit Trails, and AI Browsers and Computer Use.

For nearby applied cases, see The Crop View Becomes the GUI Grounding Receipt, The GUI Uncertainty Score Becomes the Handoff Budget, The Sensitive Screen Becomes the Handover Gate, The Computer Use Agent Meets Contextual Integrity, The Agent Benchmark Becomes the Attack Surface, and Agent Audit and Incident Review.

Sources

Return to Blog