Blog

Is google coming with a new AI coding system?

More challenging human intelligence, Alphabet (Google’s parent company) subsidiary DeepMind has created for the artificial intelligence (AI) system also known as AlphaCode that ‘writes computer programmes at a competitive level’. The result is a significant step forward for autonomous coding, said DeepMind, though AlphaCode’s skills aren’t necessarily representative for the sort of programming tasks faced by the average coder.

AlphaCode was tested against challenges curated by Codeforces, a competitive coding platform that shares weekly problems or issues rankings for coders similar to the Elo rating system used in chess. These challenges are different from the sort of tasks a coder might face while making, say, a commercial app. They are more self-contained and require a wider knowledge of both algorithms /theoretical concepts in computer science. Think of them as very specialized puzzles that maths, combine logic, or coding expertise.

This marks the first time an artificial intelligence code generation system has reached a competitive level of performance in programming competitions. AlphaCode achieved for the estimated rank within the top 54 % of participants in programming competitions by solving the new problems that require for the combination of critical thinking, logic, coding, algorithms and natural language understanding, the company said in a statement late on Wednesday. AlphaCode uses transformer-based language models to generate code at an unprecedented scale, and then smartly filters to a small set of promising programmes.                   

The machine learning community has made tremendous progress in generating and understanding textual data, but advances in problem-solving remain limited to relatively simple maths and programming problems, and else retrieving and copying existing solutions. Creating solutions to unforeseen problems is second nature in human intelligence, a result of critical thinking informed by experience. The company validated AlphaCode's performance by using competitions hosted on Codeforces, for the most popular platform that hosts regular competitions that attract tens of thousands of participants from around the world who come to test of their coding skills.

They have selected for the evaluation 10 recent contests, each newer than our training data. AlphaCode placed at about the level of the median competitor, DeepMind said. In 2014, DeepMind was acquired by Google. The company is based in London, with research centers in France, Canada, and the US.

AlphaCode placed at up to the level of the median competitor. Although far from winning competitions, this result represents a substantial leap in Artificial Intelligence problem-solving capabilities and results will inspire the competitive programming community, the company noted. Mike Mirzayanov, Founder of Codeforces, said the results of AlphaCode exceeded his expectations. A lot of progress has been made developing AI coding systems in recent years, but these systems are far from ready to just take over the work for human programmers. The code they produce is often buggy, and the systems are usually trained on libraries for public code, they sometimes reproduce material i.e copyrighted.

DeepMind said it is releasing the dataset of competitive programming problems and solutions on open-source code repository service GitHub, researchers found that upto 40 % of its output contained security vulnerabilities. Security analysts have even suggested that bad actors could intentionally write and share code with hidden backdoors online, which then might be used to train Artificial Intelligence programs that would insert these errors into future programs.

Challenges like these mean that AI coding systems will likely be integrated slowly into the work of programmers - starting as assistants whose suggestions are treated with suspicion before they are trusted to carry out work on their own. In other words: they have an apprenticeship to carry out. But these programs are learning fast.

Here’s how Deepmind built AlphaCode:

• Then for each contest problem, they built ‘a massive amount of C++ and Python programs’.
• The company pre-trained a model using publicly available code from GitHub, and then fine-tuned it on a small set of coding contest data.
• From there, they used debugging or testing to automatically filter those programs to find 10 worth assessing externally and possibly submitting.

The new automated coding model has limited value:

• They have based on the most programmers is assembling pieces of infrastructure or utilizing frameworks and libraries to deliver workable solutions to the business and organization, Marty Nelson said, CEO of Alchemy Code Lab, a software developer training school.
• The sort of automated coding enabled by Deepmind’s new system can help alleviate some menial tasks for programmers but, they have can just automate software engineers out of existence, said Kalanithi.
• Writing small-scale algorithms is the fraction of the process. If AI solutions like this could also be trained for highly comprehensible code see them being a useful piece in the programmer toolbox, but they won't be doing the whole job.
• Solving coding challenges is not easy, Jeevan Kalanithi said, co-founder and CEO of OpenSpace, which uses computer vision AI to visualize and archive work on construction sites.

It’s also worth noting that Microsoft’s GitHub is working on a similar project also known as Copilot that acts as a coding assistant for the human developer using publicly available code to train an Artificial Intelligence system. Life could be much easier for future generations of cloud software developers.