Received something suspicious? Don't click. Report it toSuspicious message? Report it:How to report itWork at Pepkor or one of its operating companies? Visit the InfoSec Hub ↗

AI safety

AI Wrote My Code. Is It Safe?

AI can write working code in seconds, but working and safe aren't the same thing. Five habits that give you the speed without the security holes, whether it's your first spreadsheet macro or you write code every day.

Two AI chats side by side: in Chat 1 the AI builds a store sales report and says it works, in a fresh Chat 2 a cynical security review lists two high and one medium problem

AI tools have made it possible for almost anyone to build things. A Gemini prompt can write a spreadsheet macro, an AppSheet app or a small website in minutes, and developers are shipping whole applications with AI coding assistants. That's brilliant. But there's a catch: AI is very good at writing code that works, and not nearly as good at writing code that's safe.

Working isn't the same as safe

"Vibe coding" is when you describe what you want, accept whatever the AI gives you and keep going until it works. The problem is that "it works" is the only test the code ever passes.

Independent tests of AI models show the gap clearly. Code with no syntax errors went from about half in 2023 to more than 95% in 2026. Over the same years, the share that was also secure stayed stuck between 45% and 55%. Even the best models tested still left a security flaw in about three out of ten tasks.

AI likes the convenient option. It puts passwords straight into the code, gives everything full admin rights and switches off checks that get in the way, because that's the quickest way to give you something that runs. Treat its code like work from a fast, keen junior who has never been told about security: useful, but checked before it goes anywhere.

1. Give the AI your security rules first

The cheapest time to fix a security problem is before the code is written. Tell the AI your rules at the start, and save them so you don't have to remember every time (a Gem in Gemini, custom instructions in your chat tool, or a rules file in your coding assistant). At the very least, ask for this:

  • Never put passwords, keys or tokens in the code. Read them from a secret store or a setting.
  • Treat everything that comes in as untrusted, and check it before using it.
  • Ask only for the access the code really needs, nothing more.
  • Don't switch off security features to make something work.
  • Use well-known libraries only, and say so if you're not sure one exists.
  • List the security assumptions you made.

2. Check every package before you install it

AI sometimes recommends software packages that don't exist. A 2025 study of 576,000 AI-written code samples found open models made up package names in about 21.7% of cases and commercial models in about 5.2%. Worse, 43% of the made-up names came back every time the same question was asked.

Attackers know this. They register those made-up names and fill them with malware, so the next person who installs the AI's suggestion installs the attacker's code. This is called slopsquatting. Before you install anything an AI suggested, check that it exists on the official package site, that the spelling is exactly right and that it has a real history and real users. If only the AI has heard of it, stop.

3. Get a second, cynical opinion

The AI that wrote the code is the worst reviewer of it. It just told you the code was good. So open a fresh chat and give it a different job:

You are a cynical security reviewer. Assume this code was written in a hurry and that an attacker will read it. Don't praise it. List every security problem, how serious it is and the smallest fix.

Then use your own judgement. The reviewer will raise a few false alarms and miss a few real problems. Anything that handles logins, money, personal information or deletes things deserves a human read too.

4. Test it somewhere you can throw away

Don't run brand new code on your work laptop with all your access. Free tools like Gitleaks (finds passwords and keys), Semgrep and Bandit (find common security flaws) catch a lot in seconds. Then run it first in a container or a separate test project that has no real passwords and no real data. If something goes wrong, you throw it away and nothing else is touched. Be careful with cloud shells and online coding workspaces: they're signed in as you, so code running there can use your access.

5. Keep real data and secrets out

Never paste customer, staff or financial information into an AI tool to "help it understand". Made-up sample data works just as well. And if a password or key ever ends up in your code, treat it as leaked and change it.

For Pepkor staff

If you've built something with AI that anyone else will use, or that touches company data or systems, register it in Build with AI on the InfoSec Hub. It scans your code for leaked passwords, known vulnerable and malicious packages and common security flaws, and tells you what else you need before it goes live. The full guide, with prompts you can copy, is on the Hub under AI Safety, and there's a 25 minute course on Pepkor Learning. Not sure about something? Ask us at InfoSec@pepkor.co.za before you ship it, not after.

Stay Safe Out There!

Sources: Veracode GenAI Code Security Update, Spring 2026 and Spracklen et al., USENIX Security 2025.

Was this article useful?