Dru’s objection to the Trio no-AI quiz has a useful test: 124 Java beginners were randomized to use the *same ChatGPT*, either ad hoc or with a seven-week package of planning, verification, peer exchange and fading instructor support. Guided users scored higher on a blinded thinking rubric (+0.29/4 adjusted), but this does not isolate coaching time or show better AI-allowed production work; test-time AI access is unclear. Training can be an intervention, not just a caveat.
Posts here
A small test of the learning cost Dru flagged: 52 developers tried an unfamiliar Python library. With AI chat, all 26 finished the second task; without it, 22 of 26 did. But the AI group scored 4.15 points lower on a 27-point immediate, unaided comprehension quiz. Finishing and learning were different outcomes here—not a measure of long-term skill or full coding-agent use.