Security | Threat Detection | Cyberattacks | DevSecOps | Compliance

Enterprises deploying AI agents and LLMs often look to prompt scanning as their first line of defense against privacy and security breaches. The idea is simple: analyze the text of the user’s prompt before it reaches the model, detect it for sensitive keywords or patterns, and block the sensitive words that may trigger a security or compliance issue. Enterprises thought this was a safe around, till they walked into unexpected issues.

Everyone’s racing to build copilots right now. But making an agentic AI that feels like a trusted teammate—one that understands context, acts safely, and simplifies complex workflows—is harder than it looks. While building Nyx, our agentic AI copilot for security teams, our team spent a lot of time thinking about how to make her an effective team member - skilled and trustworthy.

AI agents are on the rise. They can spin up, act independently, use tools, and make decisions—often without real-time human oversight. They promise incredible productivity but also introduce new risks and challenges that can’t be ignored. As these agents become more autonomous and integrated into enterprise operations, they blur the lines between human and machine responsibilities. This raises critical questions: How do we ensure they act ethically?

Over the last two years, various systems and applications have been integrated with generative artificial intelligence (gen AI) capabilities, turning regular applications into gen-AI powered applications. In addition, retrieval augmented generation (RAG)-which is the process of connecting gen-AI and large language models (LLMs) to external knowledge sources-and other agents have been incorporated into such systems, making them more effective, accurate, and updated.

Some of you may remember the early days of security, when setting up a firewall or antivirus felt like enough. It was simple and gave us a sense of control. But over time, we learned that security is a moving target. What once felt sufficient quickly became just the starting point. In today’s agentic AI era, many treat their Model Context Protocol (MCP) setups the same way. If it’s running and returning results, it feels good enough. But the AI landscape is evolving rapidly.