Wiki · Concept · Last reviewed July 10, 2026

Vision-Language-Action Models

Vision-language-action models, often abbreviated VLA models, are learned robot policies that connect visual perception, language instructions, and executable actions. They are a central technical pattern in the attempt to move foundation-model capability from screens into physical work.

Category: Concept Published: May 20, 2026 Modified: July 10, 2026 Last reviewed: July 10, 2026 Tags: Robotics, VLA Models, Physical AI, Multimodal AI, Robot Control

Definition

A vision-language-action model is a robotic policy that takes visual observations and language instructions as input, then produces actions for a robot body. Those actions may be represented as discrete action tokens, continuous motor commands, end-effector poses, gripper commands, or a lower-level control interface.

The phrase matters because it names a specific bridge. Vision-language models can describe images and answer questions. VLA models add action as an output modality, turning perception and instruction into movement. A VLA system is therefore not only multimodal; it is operational.

VLA models sit inside the broader fields of embodied AI and robotics, but they are narrower than both. The defining feature is the learned mapping from seeing and reading to doing. A robot that uses a language model for planning is not automatically a VLA unless the learned policy or action head produces robot actions.

A VLA model is also not the whole robot. It normally sits inside a stack of sensors, state estimation, low-level controllers, safety interlocks, robot middleware, human interfaces, logs, and site-specific operating rules.

Snapshot

Lineage

Early robotics often relied on hand-engineered perception pipelines, task-specific controllers, and narrow policies trained for particular environments. Foundation-model robotics tries to reuse the representational power of large models: the system may inherit knowledge about objects, language, spatial relations, and common tasks from web-scale pretraining, then learn how those representations connect to robot data.

PaLM-E, introduced in 2023, helped frame the problem as embodied multimodal language modeling: a large language model could be grounded in sensor inputs and used across robotics and visual-language tasks. RT-2 then made the VLA term prominent by translating web and robotics data into generalized robotic control.

Open X-Embodiment and related datasets changed the data story. Instead of training only on one robot's demonstrations, researchers began pooling episodes across many robot forms and institutions, making cross-embodiment transfer a central question.

Current Context

As of July 10, 2026, VLA research has moved beyond the initial RT-2 framing into a broader ecosystem of open research models, proprietary robotics systems, fine-tuning recipes, and humanoid foundation-model projects. The term is now used for robot policies that differ substantially in action representation, embodiment coverage, release status, and safety evidence.

Open X-Embodiment remains a key data reference: its project page describes data from 22 robots collected through 21 institutions, demonstrating 527 skills and 160,266 tasks, with more than one million real robot trajectories across 22 embodiments. OpenVLA is the canonical open VLA reference in the page's source set: the paper describes a 7B-parameter model trained on 970,000 robot demonstrations from Open X-Embodiment.

The open stack has diversified. OpenVLA-OFT studies optimized fine-tuning for speed, latency, and bimanual ALOHA tasks. Physical Intelligence released pi-zero code and weights in February 2025, then described pi-zero-point-five as a VLA with open-world generalization through heterogeneous co-training. Hugging Face's SmolVLA shows the opposite scale direction: a compact, open VLA intended to run on affordable hardware and public community datasets.

Proprietary and platform releases also shape the field. Google DeepMind's Gemini Robotics 1.5 page describes a VLA model that turns visual information and instructions into motor commands, while its model-card page lists Gemini Robotics 1.5 and Gemini Robotics On-Device alongside Gemini Robotics-ER 1.6. The ER line is better read as embodied reasoning that can feed robot functions or a VLA, not the same artifact as the action policy itself.

NVIDIA's GR00T line is another current reference point. NVIDIA Research described GR00T N1.5 as an improved open foundation model for generalist humanoid robots, and the public Isaac-GR00T repository currently describes GR00T N1.7 as an open VLA model for generalized humanoid robot skills. Treat those pages as primary evidence for NVIDIA's releases and implementation state, not as independent proof that any particular humanoid deployment is safe.

Architecture Pattern

Inputs. A VLA model commonly receives camera images or video frames, a language instruction, and sometimes proprioceptive state, history, or task context.

Backbone. Many systems adapt a pretrained vision-language model or multimodal foundation model, because those models already encode visual concepts and natural-language structure.

Action head. The action layer translates the model's internal representation into robot commands. RT-2 framed actions as tokens; pi-zero used a flow-matching action expert for continuous control; GR00T-style systems pair a vision-language module with a diffusion transformer action head; other systems use regression heads, action chunking, or hierarchical planners.

Training data. Training usually combines robot demonstrations with broader visual-language pretraining. The hard part is that robot data is scarce, expensive, embodiment-specific, and safety constrained compared with text or image data.

Adaptation layer. A general VLA often still needs fine-tuning or post-training for a new robot, camera placement, gripper, task distribution, or workspace. Fine-tuning results should be reported separately from base-model results.

Safety wrapper. A deployed robot should not let the VLA be the only barrier between a sentence and motion. Runtime systems need collision avoidance, force and speed limits, emergency stop, workspace constraints, permission gates, and safe fallback behavior when perception or language is uncertain.

Deployment loop. A deployed VLA policy must run under real-time constraints, deal with changing scenes, accept corrections, recover from mistakes, and avoid unsafe motion when the instruction, image, or state estimate is ambiguous.

Major Examples

RT-2. Google DeepMind introduced Robotic Transformer 2 in July 2023 as a VLA model that learns from both web and robotics data and turns that knowledge into generalized instructions for robotic control.

Open X-Embodiment and RT-X. The Open X-Embodiment Collaboration assembled robotic learning data across many robot types and tasks, then used that corpus to study policies that transfer across embodiments.

OpenVLA and OpenVLA-OFT. OpenVLA is a 7B-parameter open-source VLA model trained on robot episodes from Open X-Embodiment. OpenVLA-OFT is a later optimized fine-tuning recipe that focuses on action decoding, latency, success rate, and adaptation to new robot setups.

pi-zero and pi-zero-point-five. Physical Intelligence's pi-zero work combines a pretrained vision-language backbone with a flow-matching action expert for general robot control, emphasizing dexterous manipulation and continuous action generation. The pi-zero-point-five work extends the line toward open-world generalization through heterogeneous co-training across tasks, robots, semantic predictions, and web data.

SmolVLA. Hugging Face introduced SmolVLA as a compact open-source VLA trained on public community robotics datasets and designed for lower-cost hardware. It is useful evidence that the field is not only scaling upward; researchers are also trying to make VLA experimentation cheaper and more reproducible.

Gemini Robotics. Google DeepMind introduced Gemini Robotics in March 2025 as a Gemini-based VLA model for direct robot control. Its Gemini Robotics 1.5 materials describe a multi-embodiment VLA model paired with an embodied-reasoning model family; the current Gemini Robotics-ER 1.6 developer docs explicitly warn that physical robots can cause damage and that operators remain responsible for maintaining a safe environment.

GR00T. NVIDIA's GR00T N1, N1.5, and N1.7 releases frame humanoid control as an open robot-foundation-model problem, using vision-language inputs and diffusion-style action generation for generalized manipulation. These releases are important for open robotics infrastructure, but their benchmark and repository claims should not be generalized to untested robot bodies or sites.

Why It Matters

VLA models are important because they convert foundation-model progress into a robotics strategy. Instead of building a separate perception model, planner, language parser, and controller for each task, researchers try to train one general policy that can interpret instructions and act in varied scenes.

The potential upside is large: faster robot training, better generalization to new objects, more natural human-robot instruction, and transfer across robot bodies. The limitation is equally important: physical action exposes mistakes more directly than text generation. A wrong answer can mislead; a wrong robot action can break property or harm a person.

For the AI transition, VLA models mark the route from artificial intelligence as interface to artificial intelligence as labor. They are one of the places where automation leaves the browser and enters warehouses, kitchens, labs, hospitals, factories, farms, and homes.

Limits and Failure Modes

Data bottlenecks. High-quality robot demonstrations are far harder to collect than internet text or images. A model may have broad visual-language knowledge but thin experience with contact, force, deformable objects, clutter, and human movement.

Embodiment mismatch. Skills learned on one robot may not transfer cleanly to another with different arms, grippers, sensors, speed, reach, compliance, or safety envelope.

Semantic overreach. A model may appear to understand a high-level instruction while missing the local physical meaning: which object is fragile, which surface is hot, which person is in the way, or which action violates a norm.

Evaluation difficulty. Static benchmarks do not capture all of the variation in lighting, wear, occlusion, friction, humans, delays, dropped objects, recovery paths, or site-specific hazards. Video demos are weaker evidence than logged, repeatable trials under a named operating envelope.

Latency and control mismatch. A high-level policy may reason correctly but produce commands too slowly, too coarsely, or too inconsistently for a robot that needs stable contact-rich control.

Security coupling. Once language input can cause motion, prompt injection, malicious instructions, poisoned demonstrations, compromised updates, unsafe tool calls, or model-repository attacks can become physical risk.

Semantic safety gaps. A VLA can avoid collisions yet still choose an unsafe action because it misunderstands social context, object affordances, household danger, workplace rules, or a bystander's intent.

Overclaiming transfer. Cross-embodiment transfer is an empirical claim about a tested robot, task, and environment. It should not travel silently from a benchmark to a new body, gripper, sensor package, home, lab, hospital, or factory.

Evaluation and Safety Evidence

A credible VLA result should specify the model version, robot body, cameras and sensors, action representation, control frequency, training data boundary, fine-tuning data, simulator or real-world environment, number of trials, success metric, failure cases, human interventions, and whether the policy was evaluated on the same distribution used for training.

Safety evidence should be layered. A semantic safety benchmark such as ASIMOV can test whether a model recognizes dangerous or undesirable actions, but it does not replace robot-hardware safety, site hazard analysis, emergency stop design, collision avoidance, force limits, operator training, and post-incident review.

Industrial robot deployments should be read against robot-safety standards and local law, not only AI papers. ISO 10218-1:2025 addresses industrial robot safety requirements for the robot itself; ISO 10218-2:2025 addresses industrial robot applications and robot cells. These standards do not certify a VLA model by themselves, but they show that safety is a system-integration problem.

In the EU, an AI system can be high-risk under Article 6 of the AI Act when it is a safety component of a product, or itself a product, covered by Annex I legislation and subject to third-party conformity assessment. That legal classification is deployment-specific: the same VLA research model, industrial cell, humanoid prototype, or medical robot may face different obligations depending on use, role, and market pathway.

Governance and Safety

VLA governance is agent governance with a body. The minimum record should connect the user instruction, sensor input, model version, robot identity, action chosen, safety wrapper, human approval, and incident outcome. See also AI Agent Observability, AI Audit Trails, and AI Safety Cases.

Procurement should ask for more than a demo: model and system cards, robot hardware specifications, training-data provenance, fine-tuning records, safety evaluation, red-team results, update controls, remote-operation policy, logs, emergency-stop behavior, and a clear line between research, trusted testing, and production deployment.

VLA systems used around workers, patients, children, bystanders, homes, public spaces, or hazardous materials also raise labor, privacy, and liability issues. A policy that can see and move may become a surveillance device, a workplace pace setter, or an unsafe delegate if governance treats it as only a model.

Source Discipline

Separate five kinds of claims: a research result, a project page, a model-card limitation, an open repository release, and a deployed robot operating record. They are not interchangeable. A repository can show that weights and code are available; it does not prove safe operation at a customer site. A model card can report intended use and limitations; it does not replace site-specific hazard analysis.

For current VLA systems, record the review date, model version, release status, robot embodiment, data boundary, action interface, evaluation protocol, and whether claims come from an arXiv paper, peer-reviewed venue, official model card, official blog, standards body, regulator, or independent audit.

Avoid broad phrases such as "general-purpose robot" or "understands the physical world" unless attributed to a source and narrowed to the evidence. VLA progress is real, but the article voice should stay with demonstrated capabilities, tested limits, and deployment conditions.

Spiralist Reading

VLA models are the moment language becomes leverage on matter.

A person speaks. A model sees. A machine moves. The loop is simple enough to demo and hard enough to govern. Every step hides interpretation: what the instruction means, what the image contains, what the body can safely do, and which consequence counts as success.

For Spiralism, the warning is delegation by translation. When language is translated into action, responsibility can disappear into the interface. The system did not merely answer the human. It moved on the human's behalf, inside a world other people also occupy.

Open Questions

Sources

Return to Wiki