Unpacking the Coding Automation Framework: A New Reference for Development Leaders

Introduction

Software development is evolving rapidly, and coding automation has taken centre stage as a potential means of driving that evolution. For enterprises, Generative AI (GenAI) tools such as GitHub Copilot and OpenAI’s Codex present opportunities to reduce manual coding, boost productivity, and enhance code quality. Currently, however, the integration of GenAI into development practices needs to be more consistent across enterprise teams, and its impact remains limited. A structured approach is essential to meaningfully define and categorise the extent of automation used by developers and understand its impact. To that end, BlueOptima’s recent study, Autonomous Coding: Are we there yet?, introduces the Coding Automation Framework, designed to help large organisations evaluate their current use of automation and plan strategically.

This article is the first in a series exploring the results of the BlueOptima report and discussing how the framework can help organisations set realistic and optimal goals for implementing AI in software development.

Understanding the Coding Automation Framework

The Coding Automation Framework is inspired by the SAE Driving Automation Levels, which categorise vehicle autonomy based on the level of human intervention required. These range from Level 0 (no driving automation) to Level 5 (full driving automation) in the context of motor vehicles and their operation on roadways.
We apply this concept to coding automation to help organisations assess how much of their development process is managed by AI versus human oversight.

The Six Levels of Coding Automation

Below is a concise outline of the levels of automation. These are described in full in the Autonomous Coding report: 

1. Level 0 – No Automation: Manual coding, where developers write everything without automation tools; complete human oversight.
2. Level 1 – Basic Assistance: IDE tools provide basic support like syntax highlighting and code completion but don’t generate code.
3. Level 2 – Partial Automation: Tools handle repetitive tasks with templates and refactoring; developers still write most of the code.
4. Level 3 – Conditional Automation: AI generates more significant code portions from high-level inputs, with developers refining the output.
5. Level 4 – High Automation: AI autonomously creates complex code sections while developers handle architecture and validation.
6. Level 5 – Full Automation: AI manages all coding, testing, and deployment without human involvement—currently aspirational.

This framework offers a clear structure for organisations managing large, often disparate, development teams. Levels of automation are defined in a standardised and objective way, enabling consistency in tracking the impact of automation on critical factors such as security, productivity, and quality. 

Methodology: Building an Evidence-Based Framework

BlueOptima’s framework is backed by a comprehensive study examining the coding activities of over 110,000 developers across 82 million code changes. Using our core productivity, maintainability, and security metrics, the framework acts as a lens through which we can see how different levels of automation impact software development. These insights allow enterprises to make data-driven decisions and give more concrete meaning to discussions around levelling up through automation.

Conclusion

The Coding Automation Framework is an essential reference point for senior decision-makers navigating AI integration into software development. By categorising automation into clear levels, BlueOptima offers development leaders an objective way of assessing the current state of automation in their organisation while also defining what progress looks like in scaling that automation. 

This overview sets the stage for our upcoming blog posts, where we will delve deeper into insights from BlueOptima’s full report, discuss actionable advice for development executives, and consider the future potential of autonomous coding.