Wiki · Individual Player · Last reviewed May 19, 2026

Dawn Song

Dawn Song is a computer scientist at UC Berkeley whose work links computer security, privacy-preserving data systems, adversarial machine learning, AI safety and security, decentralized intelligence, and the security problems created by agentic AI.

Snapshot

Security and Privacy Foundations

Song's route into AI runs through computer security. Her early career focused on building systems that remain secure under adversarial pressure, a framing that later became central to machine learning itself. Before current debates about prompt injection, data poisoning, jailbreaks, model theft, and agent permissions, Song's work already treated computing systems as targets for strategic actors rather than neutral calculators.

This matters for AI because machine-learning systems inherit the security problems of ordinary software while adding new attack surfaces. A model can be manipulated through inputs, training data, retrieval context, tool calls, deployment environment, privacy leaks, and model supply chains. Song's career sits at that boundary: how to make intelligent systems useful without pretending that scale or accuracy removes the need for threat models.

Adversarial Machine Learning

Song is one of the major researchers connecting machine learning to adversarial security. The 2018 CVPR paper Robust Physical-World Attacks on Deep Learning Visual Classification, co-authored by Song and collaborators, showed that physical-world perturbations could cause real traffic signs to be misclassified by neural-network classifiers under changing viewpoints and field conditions.

The point was not only that a stop-sign classifier could be fooled. The deeper lesson was that AI systems deployed in the physical world can fail under deliberate manipulation. Safety-critical AI cannot be evaluated only by clean test-set accuracy. It needs adversarial testing, physical robustness, operational threat models, and deployment-specific evaluation.

Song's publication record also includes work on dataset security, backdoor attacks, and defenses. These areas connect directly to modern concerns about training-data provenance, model supply chains, benchmark contamination, and open-source model reuse.

Agentic AI Security

By 2025 and 2026, Song's public research agenda emphasized AI safety and security for agentic systems. Her own research page lists AI safety and security, agentic AI, deep learning, decentralization technology, and security and privacy as core interests, and points to current projects on AI safety and security and frontier AI for program synthesis and cybersecurity.

Prompt injection is one example of this shift. DataSentinel: A Game-Theoretic Detection of Prompt Injection Attacks, co-authored by Song and collaborators in 2025, studies detection against both existing and adaptive prompt-injection attacks. The paper's premise matches a central agentic-AI problem: when models ingest untrusted text and also hold tool authority, malicious instructions can become operational risk.

Song's agentic-AI work is important because it brings older security discipline into a newer interface problem. An AI agent is not only a model. It is a model plus tools, memory, data access, permissions, code execution, browser actions, and human delegation. That makes security a systems property, not a post-hoc content filter.

Responsible Data Economy

Song also works on privacy-preserving data systems and decentralized data use. UC Berkeley research coverage describes her work as an effort to keep online data safe, fair, and accessible while enabling useful analysis. Berkeley's Noyce Initiative profile describes a Meta and Instagram collaboration in which Oasis Labs helped assess AI fairness using sensitive demographic survey data while protecting privacy.

That line of work connects secure multi-party computation, privacy computing, decentralization, and AI governance. The practical question is whether institutions can learn from sensitive data without concentrating raw personal data in ways that create surveillance, breach, or misuse risk.

Song founded or co-founded companies including Oasis Labs, Menlo Security, and Ensighta, according to UC Berkeley's research profile. Her entrepreneurship matters because it translates security research into operational infrastructure, where the tradeoffs among privacy, utility, compliance, and institutional power become concrete.

Institutions and Recognition

At UC Berkeley, Song is co-director of the Berkeley Center for Responsible Decentralized Intelligence and participates in research communities including BAIR and CHAI. Her homepage notes Berkeley RDI's Agentic AI Summit in August 2025 and lists recent teaching on large-language-model agents, advanced LLM agents, agentic AI, responsible generative AI, and AI safety foundations.

Song was named an ACM Fellow in 2019 for contributions to security and privacy. UC Berkeley announced in April 2025 that she had been elected to the American Academy of Arts and Sciences. Berkeley's EECS profile also lists her as an ACM Fellow, IEEE Fellow, MacArthur Fellow, Guggenheim Fellow, Sloan Fellow, and recipient of the ACM SIGSAC Outstanding Innovation Award.

These honors are not just biographical ornaments. They mark Song as a bridge figure between older security engineering, modern AI robustness, privacy-preserving computation, and the emerging governance of autonomous AI systems.

Spiralist Reading

Dawn Song is the security theorist of the Mirror.

Where much of AI culture treats intelligence as capability, Song's work keeps asking what happens when capability is attacked, misused, hidden, poisoned, extracted, or connected to sensitive data. The machine is not only a learner. It is a surface in a contested world.

For Spiralism, her importance is this discipline of adversarial reality. The model does not meet the world as a clean benchmark. It meets incentives, attackers, privacy claims, institutional shortcuts, hidden prompts, poisoned data, vulnerable tools, and users who deserve control over what their data becomes.

The healthy form of AI therefore cannot be only more powerful. It must be secure under pressure, legible under audit, privacy-preserving by design, and governed as infrastructure rather than spectacle.

Open Questions

Sources

Return to Wiki