Microsoft’s 2026 CLI-agent rollout: more merged PRs, not yet measured value

Source. Emerson Murphy-Hill, Jenna Butler and Alexandra Savelieva, Adoption and Impact of Command-Line AI Coding Agents, posted July 1, 2026 (Microsoft-affiliated preprint). The full paper covers Claude Code and Copilot CLI, not the earlier autocomplete product.

Question and design. Does CLI-agent adoption change accepted output? For Microsoft engineers active in PR work (at least two merged PRs in a four-week pre-rollout window; trimmed at the P5–P95 pre-period), researchers compared early adopters during January 5–11, 2026 to a modelled non-adopter counterfactual built from daily merged-PR histories; pre-period October 1, 2024–January 4, 2026; follow-up January 5–April 29, 2026. A merged PR is counted when merged within 28 days of creation. Includes separate within-person weekly tool-use/PR associations.

Result. Estimated +24.0% merged PRs per engineer per day for adopters over 115 days (95% model credible interval +14.5% to +33.7%). Authors find no statistically distinguishable fade between February (+29.4%) and March–April (+20.0%), but have only four months of observation. Within-person dose-response supports an association, not a randomized causal estimate.

Why it matters. One of the more directly relevant 2026 at-scale signals for agentic rather than autocomplete tools, measured downstream of generation. But +24% PR count cannot be exchanged for +24% business value or 24% engineering cost savings. Early adopters self-select; group-specific shocks may survive the counterfactual. Smaller PRs mechanically raise counts; quality, follow-up fixes and token costs were not quantified. Authors explicitly note their affiliation with a vendor and Azure DevOps-only PR coverage.

Question to bring to a team. Can we pair merged-PR rate with independent checks on post-merge defects, review time, customer outcomes and inference spend, and observe the distribution by engineer and task?

Compare the causally different METR repeated field experiment and the older autocomplete RCTs. Synthesized in productivity evidence.