Security | Threat Detection | Cyberattacks | DevSecOps | Compliance

Ask AI to Choose a prompt Write a TLDR of this post Explain the security risk Summarize what CISOs should know.

Let’s imagine that a support copilot opens a ticket and processes a hidden instruction alongside the text it was asked to summarize. If the model follows that instruction, the response is manipulated. In an agentic workflow, the same injection can also influence actions carried out through connected tools and existing permissions. OWASP refers to this risk as prompt injection and ranks it as LLM01, the number-one risk for LLM applications.

Dario Amodei asking the AI race to slow down is a little rich. He is not wrong. Our AI pioneers were so preoccupied with whether or not they could, they didn’t stop to think if they should. Anthropic helped push the frontier hard, and now its CEO is warning that the pace is getting dangerous. Security people have seen this pattern. Build thing, connect thing, watch it behave in ways nobody quite expected, then go looking for controls.

There is a difference between watching data go into an AI tool and knowing what is inside it. Most security tools do the first. Few do the second. Employees now keep contracts, source code, and customer records inside their AI workspaces, in chats and projects that build up for months. Cyberhaven's integration with Claude's Compliance API brings that content into view for Claude Enterprise, and ties it back to where it came from.

A SOC analyst spots suspicious activity from an internal IP. They need to understand what is happening. They open their SIEM's built-in AI assistant and type: "Tell me about 192.168.0.10." The assistant checks the entity store. Nothing. The analyst rephrases: "This is in our environment. Can you please check the logs and walk me through what's going on with this device?" A moment later, the assistant returns a summary. The host is involved in Windows file sharing and remote administration.

41% of CISOs said an audio deepfake targeted their organization within the last 12 months, according to a new survey by Gartner. Additionally, 36% of respondents said employees were targeted by deepfake video calls over the past year. Craig Porter, Director Analyst at Gartner, noted that social engineering and human deception remain at the core of these AI-assisted attacks.