AI agents select tools based on natural language descriptions from shared registries, but these descriptions aren’t always verified by humans, exposing critical security vulnerabilities. Initially reported in the CoSAI secure-ai-tooling repository, these vulnerabilities split into two categories: threats during tool selection (like impersonation and metadata manipulation) and threats during execution (such as behavioral drift and contract violations).
Current software supply chain protections—code signing, SBOMs, SLSA, and Sigstore—address artifact integrity but fail to secure behavioral integrity, which ensures tools behave as promised. An adversary can exploit this by manipulating tool descriptions, which the agent’s language model may interpret as instructions, or by changing tool behavior post-publication without altering the artifact’s signature.
The proposed solution is a runtime verification proxy within the Model Context Protocol (MCP) framework that validates each tool invocation through discovery binding, endpoint allowlisting, and output schema validation. This proxy ensures tools act as specified, preventing bait-and-switch attacks and unexpected network activity, and detecting prompt injection payloads.
A layered security approach combining provenance verification and runtime checks is essential. Rolling out endpoint allowlisting first offers immediate protection, followed by output schema validation, discovery binding for sensitive tools, and, ultimately, comprehensive behavioral monitoring scaled to the risk level.
This strategy enhances the security of enterprise AI platforms by closing gaps left unaddressed by existing artifact integrity measures.
