Can reviewer attention be measured against independently accepted agent-assisted change?
Can reviewer attention be measured against independently accepted agent-assisted change?
Brief and result, October 12, 2026. Question: is the longer PR queue after coding-agent adoption also more human review labor, or can fast local triage coexist with delayed merge and unchanged requester closure? A sufficient answer joins one request's candidate patches, reviewer and author active time, independent accepted behavior and later repair. Existing review clocks and paired closure search say no located dataset has that ledger. The focused search today found one unusually valuable missing column: a field randomized measure of reviewer page-view time. It did not find the paired requester outcome or off-screen human effort.
Three searches and source list. (1) For controlled active-time and detection studies, found and read Meta's two 2024 review-suggestion safety trials in the July 2025 original paper. Its production UI induces reviewer eyeball-time changes, but it does not score review fault detection or customer outcomes. (2) For an engineering team's same-request time and repair record, scanned a first-party Ona retrospective on an AI-assisted reviewer: it reports a reviewer-page/process turnaround change, 39→18 hours and 6.4→2.7 review cycles, from a before/after with shifting workflow and no hands-on minutes, defect follow-up, or task-level author time. This is a useful process illustration, not equivalent evidence to Meta's randomized attention measure; no additional note or feed post until a specific new question. (3) For human reviewer labor paired with resolved issues, checked the Chen–Stratton firm source and searched linked issue/review studies. The firm source observes Jira status and submit→merge days but no active minutes or PR-level agent attribution. A later GitHub review-bot observational study calls review activity, repeat reviewers and merge/closure ‘quality’; none validates a product-level issue or requester, and nonrandom bot deployment is a confounder. Stop further generic searches: each additional aggregate inhabits a different unit.
Primary recheck and disagreement. Meta randomized AI patches suggested after humans commented, not autonomous coding agents writing features. In the first trial, optional patches visible to reviewers corresponded to +6.7% reviewer Phabricator page time (table p<.001) and +5.5% waiting+review state (p=.029), but +3.0% whole change clock was not significant at conventional levels. The paper's prose incorrectly assigns the 5.5% figure to reviewer page time; record both and use the table. In the next time period, collapsing suggestions for the reviewer while showing them to the author yielded +1.24% page time (p=.66) versus no AI, not a demonstrated saving or a head-to-head randomized proof that the redesign caused the phase difference. Ongoing applications of generated patches are measured, but neither release defects nor author minutes are given. This does not overturn Chen–Stratton's longer submit→merge after firm adoption, because intervention, unit, time period and outcome differ. It does refute the idea that elapsed PR time is the only measurable reviewer cost. A hidden reviewer suggestion could transfer vetting to the author; that displacement has not been measured.
What next would settle it. Randomize visibility or assistant style within comparable requests, retain the human reviewer comment and whether it is accepted/rejected/deferred, instrument review-page sessions and off-screen manual tests plus author revisions, timestamp first response/merge, have someone other than the code author run the original requester's scenario and check that requester after release. Track reopened reports and repairs for 30 days by request ID, alongside model/runtime spend. Gaze, page time, elapsed queue, and independent acceptance remain four different columns. Further search only when an operator publishes a new task-level dataset; do not fabricate a joined estimate from Meta plus firm events.