I let Copilot write a database helper for me last month while half-watching a match. Twenty seconds of tab-completion, a function that looked fine, and I pushed it to a staging branch without reading it properly. The next morning I actually read it. There was a string-concatenated SQL query sitting in the middle of it, raw user input and all. The exact thing I’d send straight back if a junior wrote it.
That annoyed me enough to spend a weekend auditing the AI-written code across one of my side projects. So when people ask me whether Copilot is safe, here’s my honest answer: the tool is mostly fine, your review habits are the risk. The code it hands you is about as secure as you force it to be, and the default isn’t great.
Let me walk through what I found, what the research actually says, and the one change to how I prompt that fixed most of it.
Two different questions hide inside “is Copilot safe”
When people ask if Copilot is safe they usually mean one of two things, and they’re worth pulling apart.
The first is about your data and your code. Does GitHub train on your private repos, does your proprietary code leak into someone else’s suggestions. On the paid Business and Enterprise tiers, GitHub says your code isn’t used to train the model and your prompts aren’t retained. The specifics live in GitHub’s Copilot documentation. For most teams that’s a contract question, and a fairly settled one.
The second question is the one people skip: is the code it writes for you actually secure. That’s the one that bit me, and it’s the one nobody checks until something breaks in a way that’s hard to explain to your boss.
The vulnerability that keeps turning up
Back in 2021 a group of NYU researchers ran the original study on this, “Asleep at the Keyboard,” and found that roughly 40% of the programs Copilot generated in security-relevant scenarios contained a known weakness. The paper is here. The models have improved a lot since, but the failure mode hasn’t really changed, because it’s baked into how these things predict text.
The one I hit most is SQL built from string concatenation. Ask for a quick lookup function and you’ll often get back something like this:
def get_user(conn, email):
cursor = conn.cursor()
query = "SELECT * FROM users WHERE email = '" + email + "'"
cursor.execute(query)
return cursor.fetchone()
It runs. It passes your tests, as long as your tests only ever use well-behaved input. And it’s a textbook SQL injection hole, CWE-89 if you want the catalogue number from MITRE’s CWE list. Pass email as ' OR '1'='1, and you’ve handed over the whole table.
The fix is the boring, correct version with a parameterized query:
def get_user(conn, email):
cursor = conn.cursor()
cursor.execute("SELECT * FROM users WHERE email = %s", (email,))
return cursor.fetchone()
Same function. One is safe, one isn’t, and the model will happily write either depending on how you ask.
The other one I see a lot is reading a file from a path the user controls. Copilot loves this version:
def read_report(filename):
with open("/var/reports/" + filename) as f:
return f.read()
Pass ../../etc/passwd as the filename and you can walk straight out of the reports folder. That’s CWE-22, path traversal, and it shows up any time the model is asked to “read a file” without being told the input is hostile.
Why the model writes the insecure version first
For a while I assumed the model just didn’t know any better. Turns out that’s wrong, and there’s recent work that explains it well.
A 2026 paper called SPARK (arxiv link) argues the security knowledge is already sitting in the model from pretraining. The bottleneck isn’t knowledge, it’s activation. Without a specific cue in the prompt, the model drifts toward the most common pattern in its training data, and a huge slice of the code on the public internet is insecure tutorial snippets and old Stack Overflow answers. So you get the average of the internet, and the average of the internet has SQL injection in it.
That reframed the whole thing for me. The model isn’t ignorant, it’s just doing what a next-token predictor does: handing you the most likely answer, not the most careful one. Which means the lever I actually control is the prompt.
The prompt change that fixed most of it
Here’s the naive prompt I used to write, and still see people use:
write me a python function to look up a user by email
And here’s the version I use now:
write me a python function to look up a user by email.
use parameterized queries, no string interpolation in SQL.
assume the input is untrusted. flag any CWE-89 risk.
The difference in output isn’t subtle. Once you name the risk, the model surfaces the careful version it already knew how to write. SPARK’s whole point is that a short, explicit cue does most of the work, and that lines up with what I see day to day. I keep a few of these cues in a snippet now and paste them into the system prompt for anything touching a database, a file path, or user input.
This isn’t a substitute for review. It just moves the starting point from “average internet code” to something closer to “code written by someone who remembered security exists.”
What I actually do before merging AI code
A weekend of auditing left me with a short routine, and I’ll admit I worked it out by getting burned, not by being clever upfront.
I read every line the model writes that touches input, queries, file paths, or auth. Boilerplate I skim. Anything that handles untrusted data I read like I’m reviewing a stranger’s pull request, because functionally that’s what it is. I run a static analyzer in CI so the obvious stuff gets caught even when I’m tired and stop paying attention, and the OWASP project keeps a solid list of what to look for in their Top Ten. And I never let “it compiles and the tests pass” stand in for “it’s safe,” because the insecure version usually compiles and passes too.
I wrote a longer piece on the review side of this, the tooling I run before merging anything an AI wrote, over in my post on AI code review tools. The short version: the model is a fast junior who never gets tired and never learns from its mistakes, so the guardrails have to live in your process, not in the model’s good intentions. If you want to see the kind of projects where I’ve had to take this seriously, a few of them are on my work page.
So, is it safe?
The tool is safe enough. The output is exactly as safe as your review process forces it to be, and the default output skews toward whatever was most common in training, which includes a lot of bad code.
One thing you can do this week: open the last pull request where you accepted a big AI suggestion and grep it for string-built queries, os.system, and anything that reads a file path straight from user input. If you find one, you’ve learned more about your own risk than any blog post can teach you. Then add two lines about untrusted input and parameterized queries to your prompt template, and watch how much cleaner the next suggestion comes out.
