Habr AI→ original

PWN AI expert Artyom Semenov: how AI agents delete files and compromise themselves

PWN AI Telegram channel author Artyom Semenov told Habr how the attack surface on AI systems has expanded with autonomous agents and the MCP protocol. He explained why system prompts cannot be considered a security boundary and how direct prompt injection differs from indirect. Attacks on language models are no longer rare—they have become a separate cybersecurity discipline.

AI-processed from Habr AI; edited by Hamidun News
PWN AI expert Artyom Semenov: how AI agents delete files and compromise themselves
Source: Habr AI. Collage: Hamidun News.
◐ Listen to article

In 2026, Habr published an interview with Artem Semenov, author of the PWN AI Telegram channel dedicated to unsafe applications of artificial intelligence, about how AI system security is structured and why attacks on language models have become a separate discipline with its own taxonomy, tooling, and real-world incidents.

How the attack surface grew

According to Semenov, the emergence of autonomous AI agents and the MCP protocol (Model Context Protocol) sharply expanded the attack surface on AI systems. If previously it was mainly about manipulations with language model responses, now agents have the ability to independently perform actions in the real world — delete files, send data outside, and even hack themselves, which makes the consequences of a successful attack far more tangible than just a "wrong answer" from a chatbot.

Why system prompts don't save you

The key thesis of the interview — a system prompt is not a security boundary. Instructions embedded by developers in a model's prompt can be bypassed, and relying on them as a protective barrier is not advisable. Separately, Semenov discusses the difference between direct prompt injection, when a malicious instruction is entered directly by the user in dialogue with the model, and indirect prompt injection, when a malicious instruction enters the model through external data — documents, web pages, emails or results of tools that the agent processes while completing a task, without even suspecting it.

  • Interview subject — Artem Semenov, author of the PWN AI Telegram channel on unsafe AI applications
  • This is already Semenov's second interview with Habr: the first conversation about security and AI was published in 2023
  • Key topics — growth of the attack surface through agents and the MCP protocol, limitations of system prompts as protection, the difference between direct and indirect prompt injection
  • AI system security is described as a separate discipline with its own taxonomy and tooling, not a side topic of classical security

What stands behind the word "taxonomy"

Semenov emphasizes that AI system protection is no longer a collection of disparate advice like "don't trust user input," but a systematic discipline: it has its own classification of attacks, its own tools for testing, and its own set of documented incidents that can be referenced. For engineers and companies embedding language models and agents in their products, this means the need to build protection against such attacks as systematically as protection against classical vulnerabilities like SQL injection or XSS, rather than relying on the model to "figure it out itself," where legitimate instruction ends and the attack begins.

Why this conversation is happening for the second time

This is not Habr's first turn to the topic of AI security with this interviewee: a similar conversation with Semenov was published on the platform back in 2023. The decision to revisit the topic and update the data is itself indicative — over the intervening time, the industry moved from simple chatbots, which at worst would give an inappropriate response, to autonomous agents with access to the file system, external services, and tools through protocols like MCP. Accordingly, the stakes have risen: an error in protection can now cost not a ruined dialogue, but actually deleted files or leaked data.

What this means

The conversation captures a shift in scale: if not long ago attacks on language models were more of an academic exercise, then with the spread of autonomous agents that actually delete files, send data outside, and are vulnerable to self-hacking, AI system security becomes a practical engineering problem — the same as the security of any other critical software, just with a new taxonomy of threats specific to agents and MCP protocols.

ZK
Hamidun News
AI news without noise. Daily editorial selection from 50+ sources. A product by Zhemal Khamidun, Head of AI at Alpina Digital.

Want to stop reading about AI and start using it?

AI News is a curated feed of AI/tech news. Hamidun Academy teaches you to use AI systematically in your work.

What do you think?
Loading comments…