Skip to content

AWS re:Invent 2025 - The state of AI in software development: Insights across 400+ organizations (AIM126)

This presentation by Justin Reock, Deputy CTO at DX, shares a data-driven "state of the union" on AI in engineering, based on insights from over 400 organizations and 135,000 developers (0:30-0:42). The research reveals both the promise and challenges of AI adoption in software development, highlighting significant variability in outcomes across organizations.

Here are the key takeaways:

1. Current Impact of AI in Software Development (2:33)

Mixed Productivity Results

While Google reported a 10% increase in productivity due to AI use (2:33-2:37), other studies showed contrasting results. The "infamous MER study" showed a 19% decrease in productivity (2:47-3:00), though developers qualitatively felt more productive with AI (3:11-3:16).

Modest but Positive Gains on Average

Broader studies show consistent but modest improvements: - 7.5% increase in documentation quality (4:09-4:12) - 3.4% increase in code quality (4:12-4:15) - 3.1% increase in overall code review speed (4:20-4:24) - 2.6% gain in change confidence (5:18-5:21) - 2.2% gain in code maintainability (5:36-5:42) - 1% reduction in change failure rate (6:00-6:04)

High Volatility at Company Level

Individual companies show significant volatility, with some experiencing over 20% increases and others experiencing over 20% decreases in metrics like change confidence, code maintainability, and change failure rate (6:32-7:58). This highlights that AI acts as an accelerant, accelerating both good and bad practices (8:14-8:20).

Outcome: 90% overall adoption of AI across sampled companies (8:27-8:32), with regular AI users saving about 3.8 hours per week from code completion (9:06-9:10).

Junior Engineers Lead Adoption

Junior engineers are using AI the most, as they are still learning new skill sets and are less reticent about adopting new tools (13:29-14:44). However, staff engineers save the most time despite lagging in overall adoption, because they are generally better at understanding well-written code and identifying issues (14:58-15:15).

The "J Curve" of Adoption

There's a clear J-curve trend: productivity and quality initially decrease during the "none to light adoption" phase due to experimentation and learning, before significantly increasing with moderate and heavy adoption (15:27-16:05). Leaders need to be prepared for this initial dip and provide education and time for experimentation (15:52-16:02).

Enterprise Advantages

Traditional enterprises have higher daily AI usage, likely due to already established clear AI policies and effective change management practices (17:01-17:47). They also save the most hours per week once they reach heavy adoption (20:13-20:21).

Outcome: Daily AI users are shipping 60% more PRs (10:47-10:50), with approximately 22% of code across the sample now authored by AI that makes it to production (9:54-10:32).

3. Language and Onboarding Impact (23:37)

Better Gains in Modern Languages

AI delivers bigger gains in modern languages like Golang, Python, Rust, and JavaScript compared to older languages like Perl and COBOL, primarily due to the training data and domain-specific semantics (23:37-25:47).

Transformative Onboarding Impact

The average time to onboard a new resource into an organization or project has been more than halved (26:02-26:24). Companies like Zapier have reduced onboarding time from 30 days to two weeks by effectively using AI agents (26:48-26:57).

Redefining "Builder"

AI is enabling traditional non-builders, such as engineering managers, project managers, and designers, to ship more code and contribute to technical aspects (28:11-30:11).

Outcome: Significant shadow AI usage exists, where developers use personal AI licenses (e.g., ChatGPT, personal Cursor licenses) that aren't tracked by enterprise APIs (21:00-22:01).

4. Remaining Challenges and Key Insights (30:21)

AI is Not a Silver Bullet

The time savings from AI are eclipsed by other pain points for engineers (30:21-30:33). Major time sinks include:

  • Interruption Frequency/Context Switching: The biggest detractor from productivity, significantly outweighing AI time savings (30:53-30:57, 31:52-32:00)
  • Meeting-Heavy Days: Too many meetings continue to be a major time sink (31:01-31:05)
  • Build and Test Time & Developer Environment Toil: Traditional pain points that compound to reduce overall productivity (37:04-37:26)

Context Switching Impact

Studies show a dramatic difference in productivity based on how often engineers are interrupted (33:34-34:15). The neurological impact of context switching leads to cognitive fatigue and reduced creativity (35:39-36:20).

Outcome: Focus on minimizing context switching and encouraging flow state to maximize overall productivity, as these issues still eclipse the gains from AI code generation (36:21-36:46).

Presenter: Justin Reock - Deputy CTO at DX