I was wiring up an agent for a client project last month. A dozen tools: email, calendar, a Postgres database, the usual suspects. Somewhere around tool number eight I had a thought that ruined my evening. I’d spent hours reviewing the system prompt for safety and exactly zero minutes reviewing the tool definitions.<\/p>\n
Turns out I had it backwards.<\/p>\n
A paper that landed on arXiv last week put hard numbers on something I’d only half-suspected: the same model that refuses a sketchy request in plain chat will often execute it without complaint once it’s running as an agent. The model didn’t change. The tool schemas changed how it reads the request. If you’re building agents in 2026, this is the AI agent security problem hiding in the one part of your codebase nobody audits.<\/p>\n
The paper is Tool Specifications Matter: Uncovering and Mitigating Safety Risks in AI Agents<\/a>. The authors started from a known but badly explained fact: LLMs get substantially less safe when deployed as agents. Ask a model to do something harmful in a chat window and it refuses. Wrap the same model in an agent loop with tools and the refusal rate craters. In their tests, the average refusal rate for harmful requests sat at 23.8 percent. Not 90. Not 70. Twenty-three.<\/p>\n What’s new here is the why. Using white-box analysis of the models’ internal representations, they traced the degradation to the schema-formatted tool specifications themselves. The JSON blobs describing your tools weaken the model’s internal refusal signals. The structured format nudges the model into execution mode before it has decided whether it should act at all.<\/p>\n I’ll admit that surprised me. My working theory was that agent safety problems came from long contexts, or from the feedback loop of observations pouring back in. Nope. The spec format alone does damage.<\/p>\n If that sounds abstract, make it concrete. An agent holding a shell tool, an email tool, and a database connection is one bad decision away from being an insider threat with API access. The gap between “refuses in chat” and “executes as a tool call” stops being academic the day the tool call drops a table or mails your customer list to a stranger.<\/p>\n Here’s a tool spec in the shape most LLM tool calling APIs expect, more or less identical to ones I’ve shipped:<\/p>\n Nothing wrong with it. But remember where it goes: straight into the model’s context, on every single turn. Both Anthropic’s tool use docs<\/a> and OpenAI’s function calling guide<\/a> are upfront about the fact that tool definitions are injected into the prompt. We treat schemas as configuration. The model treats them as instructions. That gap is where the trouble lives.<\/p>\n And it cuts both ways. If schemas can suppress refusal behavior by accident, they can do it on purpose. A tool description is an injection point, the same class of problem as OWASP’s LLM01 prompt injection<\/a> entry. I wrote about a related trick, where a poisoned GitHub issue steered a coding agent into doing an attacker’s errands, in my post on poisoned issues and coding agents<\/a>. The schema version is nastier, because nobody reads schemas with a skeptical eye. We read prompts. We skim specs.<\/p>\n The paper’s mitigation, called SafeKeep, is a two-pass trick. Pass one: ask the model whether the request is safe, but describe the tools in flat plain text instead of JSON schemas. Pass two: if the request passes, execute with the original schemas, which your API needs for actual tool calls anyway.<\/p>\n The flattening step is the kind of thing you can write before your coffee cools:<\/p>\n The safety pass then sees this instead of JSON:<\/p>\n Same information. Different format. In the paper’s experiments across four models and two benchmarks, that change lifted the average refusal rate for harmful requests from 23.8 to 70.6 percent, and dropped the attack success rate for observation-level prompt injection from 25.6 to 2.5 percent. If you care about prompt injection protection, that second number is the headline. A one-file change cut attack success by an order of magnitude.<\/p>\n Is it bulletproof? No. A 70.6 percent refusal rate still means unsafe requests get through, and I wouldn’t ship it as my only defense. But as a cheap extra layer, it embarrasses a lot of what currently gets sold as agent security tooling.<\/p>\n My review process changed after reading this paper. Descriptions come first: I read every tool description as if an attacker wrote it, because in a world of third-party plugins and MCP servers, sometimes one did. Schemas come second: an agent that only needs to read a table has no business holding a tool that can write to it. Most AI agent security vulnerabilities I’ve met in the wild were ordinary permission problems wearing a fancy hat.<\/p>\n Then outputs. Tool results flow back into the context on the next turn, so anything a tool returns is another injection path. Scraped web pages, file contents, even error messages. And I validate every argument server-side before execution, because the model producing well-formed JSON says nothing about the JSON being safe. I covered that gap in my structured outputs post<\/a>: strict mode guarantees shape, not sense.<\/p>\nThe tool schema is part of your prompt, whether you meant it or not<\/h2>\n
{\n "name": "send_email",\n "description": "Send an email on behalf of the user",\n "input_schema": {\n "type": "object",\n "properties": {\n "to": { "type": "string" },\n "subject": { "type": "string" },\n "body": { "type": "string" }\n },\n "required": ["to", "subject", "body"]\n }\n}\n<\/code><\/pre>\nThe fix is almost annoyingly simple<\/h2>\n
def flatten_tool_spec(tool):\n schema = tool["input_schema"]\n required = set(schema.get("required", []))\n lines = [f"Tool: {tool['name']}. {tool['description']}."]\n for arg, prop in schema["properties"].items():\n status = "required" if arg in required else "optional"\n lines.append(f"- takes {arg} ({prop['type']}, {status})")\n return "\\n".join(lines)\n<\/code><\/pre>\nTool: send_email. Send an email on behalf of the user.\n- takes to (string, required)\n- takes subject (string, required)\n- takes body (string, required)\n<\/code><\/pre>\nThe AI agent security risks I actually check for now<\/h2>\n