Short version for the impatient: I run two things on every AI-written PR before I even read it myself, plus one human pass. Semgrep with a small custom ruleset, a second LLM as reviewer with a structured prompt, and a hard rule that I read every external URL and import path with my own eyes. If you want to know why I bother, read on.<\/p>\n
I almost shipped a phishing URL last month. The PR was a small one, mostly written by Cursor, fixing a broken redirect in a side project. The code looked fine. Tests passed. I was about to merge when I noticed the redirect target had a Cyrillic “\u0430” inside the domain. I’m not going to publish what the URL was. I will say that the Stack Overflow answer the model had clearly cribbed from did not contain that character.<\/p>\n
Two days later I read the Scam2Prompt paper<\/a> and felt extremely seen. The authors ran developer-style prompts through GPT-4o, GPT-4o-mini, Llama-4-Scout, and DeepSeek-V3 and found that 4.24% of the resulting code contained a live scam or phishing URL. The benchmark they built on top of that, Innoc2Scam, gets the rate higher across several other models. None of those prompts were adversarial. They were the kind of thing I type into Cursor every day.<\/p>\n So I tightened my workflow. Here’s what I actually do now.<\/p>\n Before any tool runs, I look at five things in the diff, in order:<\/p>\n That’s it. Maybe 30 seconds for a small PR. It catches the dumbest 20% of issues for free and primes my brain to be skeptical of the rest.<\/p>\n Semgrep<\/a> is my main static analyzer. The community rules are fine. The magic, for me, is writing a small number of my own rules tuned to the bugs I keep seeing. Two I find myself reaching for most often:<\/p>\n These two rules took about an hour to write and tune, and they catch a class of issue that almost no off-the-shelf tool flags. Semgrep runs in pre-commit and again in CI on every PR. Total CI cost is a few seconds.<\/p>\n I do not run “AI-powered” static analyzers as part of CI. I tried two of the big ones last year. Both had false-positive rates high enough that engineers learned to ignore the warnings, which is worse than not running them at all.<\/p>\n This is the controversial layer. I have one AI assistant write the code (usually Cursor or Claude Code), and a different model review the diff before I look at it.<\/p>\n The prompt I use is boring on purpose:<\/p>\n The “do not invent issues” line cuts the rate of hallucinated bugs by a lot. The structured categories keep the reviewer focused on the boring stuff a human is bad at scanning for.<\/p>\n Why a different model? Because the two systems make different mistakes. If Cursor is running on Claude in my setup, I will not ask the same Claude to review it. I use GPT-5 or DeepSeek for the second pass, depending on which I have credits with. Agreement between models on what’s a real issue is a useful signal. If both flag the same line, I read it carefully.<\/p>\n This is also why I don’t fully trust GitHub’s built-in code security tooling<\/a>. The signal is real, but it’s one signal from one provider trained on roughly the same internet as the model that wrote your code.<\/p>\n I know. Boring. The layers above are pre-filters, not replacements.<\/p>\n What I’m actually doing when I read an AI-written diff is different from when I read a human one. With a human PR I’m checking intent and style. With an AI PR I’m doing forensics. Did the model invent an API that doesn’t exist? Did it answer a question I didn’t ask? Did it copy a pattern from a different version of the framework I’m on?<\/p>\n The thing that helps most is having a clear mental model of which parts of a codebase the AI is good at and which it isn’t. Cursor is fine on glue code. It’s bad on anything to do with auth, payments, or our internal queue abstraction. I’m not going to merge a PR touching those without reading every line myself, no matter how green the CI run is. I wrote about how I pick between assistants in my Cursor vs. Copilot vs. Claude Code post<\/a> if you want the long version.<\/p>\n I want to be honest about the gaps, because this stuff sounds tidier on a blog than it does in practice.<\/p>\n Tests that pass for the wrong reason are still a problem. The model writes a function, writes a test for it, and the test is shaped exactly like the function. Both are wrong in the same direction. CI is green and I don’t catch it until something breaks later. My current half-fix is to ask the reviewer LLM to verify the test, but that’s the same model-bias problem in a different coat.<\/p>\n Prompt injection in code comments is another thing none of my layers catch reliably. If an AI assistant pulls context from an upstream README or issue that contains an instruction, the resulting code can get steered by an attacker. I haven’t been bitten by it yet, but it feels like a matter of time.<\/p>\n Subtle data-flow bugs across files are the third gap. Semgrep can do taint analysis for some languages, but in Go and TypeScript with our messy codebase it’s noisy enough that I’ve turned it off for now. I just accept that this is the class of bug that will reach production and we’ll catch it with observability instead of review.<\/p>\n For my own sanity, I added the following to the PR template on a couple of projects. The line that matters most is the second one.<\/p>\n Reviewers do not approve until the checkboxes are real. Yes, you can lie on a checkbox. I have to trust my team. But forcing the second-model output to be pasted into the PR makes it inconvenient to skip the step, which is most of what a checklist does anyway.<\/p>\n Pick one rule from the Semgrep snippet above, drop it into your CI, and see what fires on your last month of merged PRs. Even if you don’t keep the rule, the audit is interesting. The first time I ran the homoglyph rule across a backlog at a previous client I found three URLs with mixed scripts. None of them had been exploited. All of them had been merged by competent engineers.<\/p>\n If you build a lot of LLM-driven tooling, the case for taking this seriously is in the Scam2Prompt numbers<\/a> and your own gut. If you’re a one-person side project shop, the 30-second human scan plus a Semgrep run probably gets you 80% of the way there.<\/p>\nThe 30-second human scan I do first<\/h2>\n
\n
http:\/\/<\/code> and https:\/\/<\/code> string. Stupid simple, but it would have caught my Cyrillic incident.<\/li>\nfetch<\/code>, axios<\/code>, http.Get<\/code>, or requests.post<\/code> that wasn’t there before. If the model added a network call I didn’t ask for, I want to know why.<\/li>\n\/\/ safe to ignore<\/code>, \/\/ works locally<\/code>, or \/\/ fix this later<\/code> in code that touches auth or payments.<\/li>\n<\/ol>\nLayer one: Semgrep with a small custom ruleset<\/h2>\n
rules:\n - id: suspicious-unicode-in-url\n message: URL contains a non-ASCII character (possible homoglyph attack)\n languages: [generic]\n severity: WARNING\n patterns:\n - pattern-regex: 'https?:\/\/[^\\s"'']*[^\\x00-\\x7F][^\\s"'']*'\n\n - id: hardcoded-redirect-domain\n message: Hardcoded external domain in a redirect target\n languages: [javascript, typescript, python, go]\n severity: WARNING\n pattern-either:\n - pattern: redirect("$URL")\n - pattern: location.href = "$URL"\n<\/code><\/pre>\nLayer two: a second LLM reviews the first one’s code<\/h2>\n
You are reviewing a code change written by another AI assistant.\nDo not rewrite the code. Do not summarize it.\n\nList only the following, with file paths and line numbers:\n1. External URLs added or modified. For each, state whether it\n points to a domain you do not recognize.\n2. New dependencies or imports. Flag any package name that looks\n like a typo of a popular package.\n3. Network calls or shell-out calls that were not present before.\n4. Inputs that flow to a sink (file write, DB query, exec) without\n an obvious validation step.\n5. Tests that assert nothing or assert on the wrong thing.\n\nIf you find none of the above in a category, say "none" for that\ncategory. Do not invent issues.\n<\/code><\/pre>\nLayer three: I still read the diff<\/h2>\n
What still slips through<\/h2>\n
A PR checklist I actually use<\/h2>\n
## AI-assisted PR checklist\n- [ ] I have read every external URL in this diff\n- [ ] I have eyeballed every new import or dependency\n- [ ] I ran Semgrep locally (`make sec`)\n- [ ] I ran a second-model review and pasted the output below\n- [ ] I have read the diff end-to-end myself\n<\/code><\/pre>\nWhat I’d do this week if I were you<\/h2>\n