VOOZH about

URL: https://thenewstack.io/dont-trust-security-in-ai-generated-code/

⇱ Don’t Trust Security in AI-Generated Code - The New Stack


TNS
SUBSCRIBE
Join our community of software engineering leaders and aspirational developers. Always stay in-the-know by getting the most important news and exclusive content delivered fresh to your inbox to learn more about at-scale software development.
REQUIRED
It seems that you've previously unsubscribed from our newsletter in the past. Click the button below to open the re-subscribe form in a new tab. When you're done, simply close that tab and continue with this form to complete your subscription.
The New Stack does not sell your information or share it with unaffiliated third parties. By continuing, you agree to our Terms of Use and Privacy Policy.
Welcome and thank you for joining The New Stack community!
Please answer a few simple questions to help us deliver the news and resources you are interested in.
REQUIRED
REQUIRED
REQUIRED
REQUIRED
REQUIRED
Great to meet you!
Tell us a bit about your job so we can cover the topics you find most relevant.
REQUIRED
REQUIRED
REQUIRED
REQUIRED
REQUIRED
Welcome!

We’re so glad you’re here. You can expect all the best TNS content to arrive Monday through Friday to keep you on top of the news and at the top of your game.

What’s next?

Check your inbox for a confirmation email where you can adjust your preferences and even join additional groups.

Follow TNS on your favorite social media networks.

Become a TNS follower on LinkedIn.

Check out the latest featured and trending stories while you wait for your first TNS newsletter.

PREV
1 of 2
NEXT
VOXPOP
As a JavaScript developer, what non-React tools do you use most often?
Angular
0%
Astro
0%
Svelte
0%
Vue.js
0%
Other
0%
I only use React
0%
I don't use JavaScript
0%
Thanks for your opinion! Subscribe below to get the final results, published exclusively in our TNS Update newsletter:
NEW! Try Stackie AI
From clobbered drafts to real-time sync
Apr 14th 2026 10:00am, by David Moore
TypeScript 6.0 RC arrives as a bridge to a faster future
Mar 14th 2026 9:00am, by Darryl K. Taft
Mastra empowers web devs to build AI agents in TypeScript
Jan 28th 2026 11:00am, by Loraine Lawson
2024-10-28 10:00:27
Don’t Trust Security in AI-Generated Code
contributed,
AI Engineering / DevOps / Software Development

Don’t Trust Security in AI-Generated Code

While AI tools like Copilot provide efficient coding solutions, research indicates a troubling increase in security vulnerabilities among AI-assisted code.
Oct 28th, 2024 10:00am by Jerome Robert
👁 Featued image for: Don’t Trust Security in AI-Generated Code
Photo by RoonZ nl on Unsplash.

Speaking from more than 20 years of experience in development and cybersecurity, developers need to use all the cutting-edge, time-saving, and productivity-boosting tools. It’s meticulous, time-consuming work to ensure you commit to high-quality, functional code, and the software development life cycle always demands more.

As such, nowadays, almost all developers use some form of AI-generated code — and they absolutely should. AI tools make developers’ lives easier by leveraging the knowledge cultivated by the development community over time and across the globe to overcome obstacles that, while potentially new and challenging to them, have long been addressed. They can reasonably trust that code to perform the function they want to achieve — and can test it to be sure.

But can they trust that code to be secure? Absolutely not. With all that time and work spent committing functional code, just as much, if not more, is spent navigating the security backlog afterward.

What’s Wrong With AI-Generated Code?

GenAI platforms, such as Copilot, learns from code posted to sites like GitHub and has the potential to pick up some bad habits along the way. It searches for and returns code that, first and foremost, actually works, but security is a secondary objective (if at all). As you’ll see further down in this article, this leads to substantial potential for vulnerabilities.

A pair of studies recently explored the effect of AI on code security. The first was a Stanford University study, “Do Users Write More Insecure Code with AI Assistants?” and the other was a Wuhan University study, “Exploring Security Weaknesses of Copilot Generated Code in Github.”

The Stanford study found the following.

  • Participants who had access to an AI assistant wrote significantly less secure code than those without access to an assistant.
  • Participants with access to an AI assistant were also more likely to believe they wrote secure code, suggesting that such tools may lead users to be overconfident about security flaws in their code.
  • Participants who invested more in creating their queries for the AI assistant, such as providing helper functions or adjusting the parameters, were more likely to eventually offer secure solutions.

The Wuhan study found that almost 30% of Copilot-generated code snippets have security weaknesses. Focusing specifically on Python, 91 of 277 snippets, or 33%, contained security weaknesses; of those 91 snippets, there were 277 instances of security weakness. In other words, the insecure code was VERY insecure.

How Do I Secure My Code?

At this point, it should go without saying that when it comes to using GenAI like Copilot to help with your code, you should never assume that what you’re getting is perfectly secure. You should approach AI-generated code like you approach code written by humans with a keen, skeptical eye and through standard security protocols. However, with the volume and prevalence of potentially vulnerable AI code making its way into software development lifecycles, traditional reactive approaches may not be enough.

Organizations should create and maintain a culture of security that integrates security at every stage of the SDLC and seeks to identify vulnerabilities as proactively as possible. Ideally, vulnerabilities should be identified as developers write the code, ensuring they commit quality, secure code, eliminating backlogs for security operations teams, and making the entire lifecycle more efficient. Addressing vulnerabilities within the IDE during coding is the natural end-point of shift-left and secure-by-design philosophies and the most effective and efficient way to integrate security into the software development lifecycle.

Conclusion

Whether code is manually written or AI-generated, detecting and fixing vulnerabilities as code is written saves time and preserves focus. This also reduces the back-and-forth in peer reviews, making the entire process smoother and more efficient. By embedding security more deeply into the development workflow, we can address security issues without disrupting productivity.

TRENDING STORIES
Jerome Robert is the co-founder and CEO of startup, Symbiotic Security. He has over 20 years of experience in cybersecurity and 15 years as a CxO. Starting his career in mathematics and engineering, Jérôme has transitioned to business leadership, leveraging...
Read more from Jerome Robert
SHARE THIS STORY
TRENDING STORIES
SHARE THIS STORY
TRENDING STORIES
TNS DAILY NEWSLETTER Receive a free roundup of the most recent TNS articles in your inbox each day.
The New Stack does not sell your information or share it with unaffiliated third parties. By continuing, you agree to our Terms of Use and Privacy Policy.