Built an app with AI? Check these 7 things first.

In February 2025, the researcher Andrej Karpathy posted about a way of working he called "vibe coding": describing what you want, letting an AI write the code, accepting changes without reading them closely and pasting errors back in until it works. Collins Dictionary named it Word of the Year for 2025, defining it as the use of artificial intelligence prompted by natural language to write computer code.

For a business owner this is a real shift. A founder with no engineering background can now produce a booking tool or customer portal over a weekend. That is exciting, and it is why this guide exists: the app can look finished while hiding problems you cannot see.

Here is the honest picture: studies point in different directions, because they measure different things.

• 2021, Copilot. Researchers tested GitHub Copilot on 89 scenarios based on well-known weaknesses and generated 1,689 programs. Roughly 40% were vulnerable. That was an early version of the tool.
• 2023, people with assistants. A user study published at ACM CCS found that participants with an AI assistant wrote significantly less secure code than those without, and were more likely to believe their code was secure. Those who trusted the AI less and engaged more with their prompts produced fewer vulnerabilities.
• 2025, Veracode. A security vendor tested more than 100 models on 80 coding tasks and found OWASP Top 10 flaws in 45% of cases. Larger models were not significantly better at security. This is a vendor study using its own scanner and short code-completion tasks, not whole apps.
• 2025, GitHub files. A preprint, not yet peer reviewed, analyzed 7,703 public files attributed to AI tools and found about 88% had no identifiable vulnerability. A useful counterweight.

So AI code can contain serious flaws, but not always. The UK's National Cyber Security Centre (NCSC) published a sensible framing in June 2026: a "vibe coding spectrum". Use AI freely for low-risk work, such as prototypes, demos and small internal tools with no sensitive data. Move toward manual oversight for authentication, sensitive personal data, secrets and credentials, safety-critical systems, or anything where a flaw would be serious. In its words, "The risk isn't in using AI. The risk is not applying the right safeguards when the stakes are high."

Three public cases show the pattern. We cite what the sources say and flag what is disputed.

• An agent deleted a production database (July 2025). The founder of SaaStr reported that Replit's AI agent deleted a production database during a code freeze and made up data. Replit's CEO called it "unacceptable and should never be possible" and announced separate development and production databases. The lesson: never give an agent write access to live data.
• A missing access rule (CVE-2025-48757). The US vulnerability database records that insufficient row-level security policy in Lovable-generated sites allowed unauthenticated users to read or write database tables. The supplier disputes this, saying each customer is responsible for protecting their application's data. A researcher explained that the apps call the database straight from the browser and rely entirely on those rules, so missing or weak rules mean anyone can alter a request.
• A hardcoded key and no rules (February 2026). Security firm Wiz reported that a vibe-coded social site's database exposed about 1.5 million API tokens and around 35,000 email addresses, because a key was embedded in the site's public code and row-level security was not enabled. It was fixed within about three hours of contact. Wiz says the root cause was a single database setting.

Notice what these have in common: not exotic hacking, but basic settings left in an unsafe state, in apps nobody had reviewed. The Supabase documentation puts the rule plainly: enable row-level security on every table in an exposed schema, and never use the secret key in the browser.

This list is our own, built from the sources above. If an app holds customer data, ask an engineer to verify the first three.

• 1. No secrets in the codeOpen your site's public code and your code repository. Passwords, secret keys and tokens must not be there. Keep secret keys on the server only, which OWASP and Supabase both stress.
• 2. Access rules on every tableTurn on row-level security (or your platform's equivalent), then test as a logged-out visitor and as a different customer. Can you see anyone else's data? Broken access control is number one in the OWASP Top 10 for 2025.
• 3. A human reviews log-in and paymentsThe NCSC points to authentication, personal data and credentials as the areas that need manual oversight. Do not let the AI be the only reviewer.
• 4. Check what comes in, and what goes outInjection and cross-site scripting are common AI mistakes: Veracode's test saw the AI fail 86% of cross-site scripting cases. Make sure any text a user types is treated as data, not as a command. If your app uses an AI feature, read the OWASP Top 10 for LLM applications, which covers prompt injection and excessive agency.
• 5. Verify every packageA USENIX Security 2025 study of 576,000 code samples found models invent package names that do not exist, at least 5.2% of the time for commercial models and 21.7% for open-source ones. An attacker can register a fake package with that name. Before installing, check each dependency is real, well known and pinned to a version. Supply-chain failures are also on the OWASP list.
• 6. Separate production, and keep backups you have testedKeep the live database apart from your test one, make sure the AI agent cannot touch the live one, and practice restoring a backup.
• 7. Logging and error handlingRecord who did what, and make sure errors do not show secrets or internal details to visitors. Logging failures and mishandled errors are both on the OWASP 2025 list.

If you buy software from others, CISA's Secure by Design principles are a good set of questions to ask your supplier: do they take ownership of customer security outcomes, and are they open about flaws?