Exploring DORA's Take on Generative AI for Software Developers

UV PantaUV Panta
7 min read

You've likely been hearing a fair bit about generative AI (gen AI), those clever tools that can whip up code and text. Well, it's causing quite a stir in the world of software development. The latest DORA report confirms this, showing that a whopping 89% of organisations are prioritising the integration of AI into their applications, and 76% of technologists are already using AI in some part of their daily work. This "AI moment" is backed by serious investment, with leading tech giants expected to pump around $1 trillion into AI development over the next five years.

However, it's not all plain sailing. Developers naturally have some valid concerns, such as worries about job displacement, security risks, and the potential for AI to eat into the time they spend on truly rewarding work. This guide aims to look beyond just adopting AI and focus on how to integrate it responsibly and effectively throughout the software development lifecycle, maximising the good bits while keeping the risks in check.

How AI is Shaking Things Up for Developers

The initial news for individual developers looks largely positive:

  • More Time in Flow: Developers who use gen AI more extensively report experiencing a more frequent flow state.

  • Increased Job Satisfaction: They also report higher overall job satisfaction.

  • Boost in Productivity: Gen AI use is linked to increased productivity. The findings suggest that a 25% increase in AI adoption by an individual could lead to an approximate 2.1% increase in productivity.

  • Reduced Burnout: Interestingly, those using gen AI more also report less burnout.

However, here's where it gets a bit more complex. One of the big hopes for AI was that it would free up developers from routine tasks to focus on more valuable work. Yet, the data indicates that increased AI adoption might actually lead to less time spent on work developers consider valuable, while time spent on tedious, "toilsome" work seems to remain largely the same.

The researchers came up with the "vacuum hypothesis" to explain this. Essentially, by boosting productivity and flow, AI helps people complete valuable work more efficiently, creating extra time. However, AI isn't yet tackling those less enjoyable but still necessary tasks like meetings and bureaucracy. It's worth noting that even with this shift, developers' well-being hasn't been negatively affected.

Impact on Teams and Organisations

From an organisational standpoint, the influence of AI appears quite promising in several areas:

  • A 25% increase in AI adoption is associated with a 7.5% increase in documentation quality.

  • Code quality is also likely to see a 3.4% increase.

  • Code review speed could improve by around 3.1%.

  • Approval speed for code changes might see a modest 1.3% increase.

  • Code complexity is estimated to decrease by 1.8%.

These improvements suggest that AI is helping people get more value from their codebases and documentation, and is also speeding up the code review and approval processes.

However, and this is crucial, despite these positive impacts on development processes, the findings indicate that AI adoption is negatively impacting software delivery performance. For every 25% increase in AI adoption, there's an estimated 1.5% reduction in delivery throughput and a more significant 7.2% reduction in delivery stability. The researchers hypothesise that the increased speed of code generation due to AI might be leading to larger change sizes, and as DORA research has consistently shown, larger changes are slower and more prone to instability. So, even with AI, the fundamental principles of successful software delivery, like small batch sizes, remain vital.

What Developers Truly Value in Their Work

To better understand the impact of AI on developers' perception of their work, the research delved into what developers actually consider "valuable". They identified five key perspectives:

  • Utilitarian Value: The feeling that their work has a positive impact on the world. Gen AI can potentially boost this by speeding up development.

  • Reputational Value: Being recognised for the work they've done. AI could increase this by improving the impact of their work, but it could also reduce it if AI gets the credit.

  • Economic Value: The pay and benefits associated with their work. AI might increase this through higher productivity, but some worry about potential reductions in workforce or paid hours.

  • Intrinsic Value: The inherent worthwhileness of the development work itself, often linked to learning and traditional skills. AI is anticipated to have a neutral impact as new skills like prompt engineering become important.

  • Hedonistic Value: The enjoyment derived from performing certain development tasks. AI could make enjoyable tasks more accessible but might also make some obsolete. Allowing developers to choose not to use AI for tasks they enjoy is important.

Building Developers' Trust in Gen AI

For developers to embrace and benefit from AI, trust is paramount. However, research suggests that developers' trust in gen AI output is currently relatively low. Organisations can foster this trust through several strategies:

  • Establish a Clear Policy on Acceptable Gen AI Use: Providing explicit guidelines encourages responsible use and can alleviate fears of unknowingly acting irresponsibly. Organisations with more transparent AI use policies see higher levels of trust.

  • Double-Down on Fast, High-Quality Feedback: Robust code review and automated testing processes assure developers that errors introduced by AI-generated code will be caught. Interestingly, AI adoption can actually make code reviews faster and improve code quality.

  • Provide Opportunities for Developers to Gain Exposure with Gen AI: Familiarity increases trust. This is particularly true when developers can use AI in their preferred programming languages, where they have the expertise to evaluate its output.

  • Encourage Gen AI Use, But Don't Force It: While leadership encouragement is effective, developers need to maintain control over when and how AI is used. Building community structures to share knowledge organically can be a good approach.

  • Help Developers Think Beyond Automation: Addressing fears of job displacement requires envisioning the future role of developers working with AI at a higher level of abstraction, focusing on innovation and user value.

Practical Strategies for Adopting Gen AI

Moving from isolated AI experiments to widespread adoption requires a strategic approach. Here are four research-backed strategies for organisations:

  1. Share and Be Transparent About How Your Organisation Plans to Use AI: Open communication about the AI mission, goals, and policies can alleviate apprehension and position AI as a tool to help everyone focus on more valuable work. Organisations that do this can see an estimated 11.4% increase in team adoption of AI.

  2. Address Developer Concerns About AI's Impact: Directly addressing anxieties about job displacement can enable developers to focus on learning how to best use AI. Organisations that alleviate these concerns are estimated to have 125% more team adoption of AI.

  3. Allow Ample Time for Developers to Learn How to Use AI: Providing dedicated time for experimentation and integration leads to significantly higher adoption rates. Simply giving developers dedicated work time to explore AI tools can lead to a 131% increase in team AI adoption, and actively encouraging integration leads to a 27% increase.

  4. Create Policies That Govern the Adoption of AI: Clear guidelines on appropriate use cases, ethical considerations, and potential risks can reduce uncertainty and encourage responsible experimentation. Organisations with AI acceptable-use policies show a 451% increase in AI adoption.

Measuring the Success of AI Adoption

To understand the impact of gen AI, it's crucial to establish baseline measurements and track progress at the team, service, and organisational levels. Some key metrics to consider include:

  • Code assistant metrics: Licenses allocated, daily active users, code suggestions generated and accepted, lines of code accepted.

  • Fast-feedback metrics: Tests on commit, daily tests, daily builds, test confidence, time to fix broken builds.

  • Team-level metrics (gathered through surveys): AI task reliance, AI interactions, perceived AI productivity, trust in AI output, organisational trust, flow, job satisfaction, valuable work, burnout, code review time, documentation quality, technical debt.

  • Service-level metrics: Code complexity, code quality.

  • Organisational metrics: Customer numbers, market share, overall performance, profitability, customer satisfaction.

Gathering feedback from developers through regular surveys, team retrospectives, and communities of practice is also essential for refining the AI adoption strategy.

Key Takeaways for the Future

For Leaders:

  • Prioritise transparency: Clearly communicate your AI strategy, directly address job security concerns, and establish clear policies for responsible AI use.

  • Invest in your people: Provide dedicated time, training, and resources for developers to learn and experiment with AI. Foster a culture of learning and psychological safety.

  • Measure and iterate: Track key metrics (code quality, developer satisfaction, delivery performance) and be prepared to adjust your approach based on data.

For Practitioners:

  • Embrace experimentation: Don't be afraid to try new AI tools and explore different use cases within established guidelines.

  • Become AI-fluent: Master prompt engineering, understand AI limitations, and integrate AI into your workflow.

  • Own the output: Always review, test, and refine AI-generated code and documentation. Your expertise remains critical.

The integration of gen AI into software development is a significant and ongoing journey. By taking a thoughtful, data-driven, and human-centred approach, organisations and developers can collectively harness the power of AI to create a more productive, fulfilling, and innovative future.

0
Subscribe to my newsletter

Read articles from UV Panta directly inside your inbox. Subscribe to the newsletter, and don't miss out.

Written by

UV Panta
UV Panta