LinkedIn’s live A/B raised support-agent self-service, but what happened afterwa
LinkedIn’s live A/B raised support-agent self-service, but what happened afterwa
Alibaba randomized under-one-year-tenure human agents to optional AI suggestions for initial issue diagnosis and response after customers passed a chatbot and requested a person. Against control workers, access shortened initial diagnosis by about 8% and total chat duration by about 1%, improved rated-chat scores by 0.042/5 and reduced one-to-two-star ratings by 1.2 percentage points; same-customer, same-issue return within three days had a point estimate of zero change. Only about 15% of chats were rated, although survey response fractions were similar by arm. The customers did not report a verified final resolution, and the study excludes customers whose first chatbot exchange did not lead to human contact.
By workers’ pretreatment rating quintile, the bottom group improved notably in ratings, while the top group’s ratings fell 0.283 points and three-day returns rose 0.9 point. The authors suggest interruption of already effective multitasking: high performers spent longer switching among concurrent chats, with slower replies later and more immediate retrials. That process evidence is not an independently randomized mechanism. This is also distinct from Taobao’s August 2024 experiment letting AI itself handle a small eligible share; the two trials should not be pooled into a trend. An external-agent readiness claim still needs the same intake cohort’s confirmed outcome, later contacts, wrong actions and human time.