Amazon launches AWS BugBust to spur adoption of AI-powered coding tools

Software failures are expensive — and on the rise. An estimated 19% to 23% of software development projects fail, and Standish Group found that “challenged” projects — i.e., those that fail to meet scope, time, or budget expectations — account for about 52% of software projects. According to a joint project by Undo and Cambridge Judge Business School MBA, the cost of these bugs to enterprises is about $61 billion annually, and a around 620 million developer hours are wasted on debugging.

A partial solution to the quality assurance problem might be machine learning, which could augment developers’ workflows to make it easier to spot critical bugs in software. Amazon’s CodeGuru service takes this approach, drawing on machine learning models trained with millions of lines of code to recommend fixes for developers, troubleshoot performance issues, and detect anomalies.

To draw awareness to CodeGuru and AI-powered tools like it, Amazon today launched Amazon Web Services (AWS) BugBust, an international challenge for developers to fix 1 million software bugs and realize $100 million in technical debt.  Developers from around the world can join the challenge by creating BugBust events for their organization using CodeGuru and compete for prizes on a leaderboard by identifying and fixing bugs in their codebases and apps.

“BugBust lets you create a challenge for your organization to come up with a list of potential problems,” Amazon CTO Werner Vogels told VentureBeat in a phone interview this week.  “It makes bug fixing a bit of a game — who can find the most bugs and get on the leaderboards. The goal is to make squashing bugs and finding performance issue a fun thing to do instead of a chore.”

Pandemic challenges

The pandemic has given rise to software failures that ordinarily might not have happened. Remote learning, online exams, and work-from-home requirements drove a huge influx of online users, putting pressure on software that wasn’t necessarily designed to handle the volume. According to a McKinsey report, as of October, interactions between customers and North American companies are 65% digital compared with 41% pre-pandemic. And nearly two-thirds of knowledge workers have increased their use of collaboration tools since working from home, an Asana survey found.

Complicating matters, there’s a massive shortage of developer talent. It’s estimated that just 2% of the world’s population knows how to develop software, and the global need is anticipated to grow by 24% over the next seven years.

CodeGuru can lend a hand, here, argues Werner, by freeing up developers with limited time to focus on important tasks. “I think that these days, definitely with digital transformations, there’s a pressure to build things,” he said. “[We talk to many organizations] who would love to go back over their code to figure out for a particular app that they built five years ago where the resources are going — for example, CPU bottlenecks, memory bottlenecks, and memory leaks. Organizations usually can’t find the time for it, so we’re trying to automate it further.”

To this end, CodeGuru offers two tools designed for code optimization: Profiler and Reviewer. Reviewer uses program analysis and machine learning to detect potential defects in code repositories and offer suggestions. While it doesn’t flag certain mistakes, Reviewer can identify problems related to best practices, input validation, security analysis, code quality, and more.

As for Profiler, it collects runtime performance data from a live app and delivers recommendations that can to help to fine-tune the app’s performance. Using machine learning, Profiler tries to find the most expensive lines of code and suggest ways that efficiency can be improved, either by eliminating dependency bottlenecks or reducing bloat.

Amazon says that Reviewer, which can scan 1 million lines of code in 30 minutes, has analyzed over 200 lines and produced 165,000 recommendations on fixes for developers — including 25,000 Amazon developers — to date. Moreover, the company says that its internal teams have used Profiler on more than 30,000 apps deployed into production.

The idea behind BugBust, then, is to enable organizations to use CodeGuru to fix bugs. Developers on the US East (N. Virginia) AWS region, with more coming soon, can claim and squash bugs to compete within their organizations and internationally for a chance to win prizes, badges, and a trip to AWS re:Invent, Amazon’s annual developer conference, which will host a live BugBust event.

“[We’re trying to] motivate people to actually use these tools not just as part of their daily new code building, but also to start looking at old code — code that they’ve had for a long time to see whether they actually adopt and optimize that,” Vogels said. “It doesn’t happen enough and it doesn’t have priority in all organizations.”

Looking ahead

Despite their potential, coding assistance tools like CodeGuru have their limitations. Reviewer only supports code written in Python or Java and can’t spot syntactical mistakes, for example. And recent research suggests that even the best models can’t learn to generate answers to difficult coding problems without syntax errors.

But Vogels notes that CodeGuru is designed to self-improve from customer feedback, which serves as a signal in the bug-spotting models’ retraining. That’s arguably superior to manual code review processes, which only a minority of software developers say that they’re satisfied with, according to a SmartBear study.

“It’s based on both rule mining and machine learning — it’s combination of logistic regression and neural networks,” Vogels said. “[CodeGuru is] going to get better and better over time, over more and more languages … It’s a new set of rules that are being created every time we run this.”

Ultimately, Vogels says, the goal with CodeGuru and BugBust is to augment developers — not replace them. While simple functions might eventually be handled by code-correcting engines, the need for programmers who write higher-level apps and APIs won’t go away in the foreseeable future.

“Programming is an artistic profession,” he said. “We tend to create new things every day, and to be able to focus on the creation parts and automate some of the more boring but very important performance and security parts is a key thing.”