LinkedIn’s agent-built support workflow: did live customers improve?

Research question, October 1, 2026. Dru separated internal autonomous closure from possible new readiness for external customer interactions in a Taobao-post discussion. A good answer would join a live assigned cohort, customer-verified outcome, repeat contact, satisfaction, human escalation/work, action rights and after-release repairs. Recent signals so far are more often perceptions and vendor assertions than controlled results. Four cuts: assignment/denominators; what self-service means; proposal-review-release loop; comparison with a more recent independent original such as Nubank’s September experiment.

Primary paper and live comparison. Wang et al., LinkedIn, August 10, 2026, “Self-evolving Agentic Customer Support System at LinkedIn”. Two-week 50:50 persistent user assignment compared an existing handcrafted-prompt/fixed-retrieval/periodic-human-QA support agent with an integrated evolved-prompt, agent-invoked retrieval and evaluator-driven revision workflow. QA self-serve rose 33.7%→42.7%, +9.0 percentage points [8.4, 9.6], among 42,982 control and 45,669 treatment QA conversations (35,867/34,334 users). Cancellation self-serve rose 61.9%→66.6%, +4.8 points [3.8, 5.7], among 20,468/20,456 cancellation conversations (13,442/13,308 users). They used proportion tests with a user-cluster check and multiple-comparison adjustment. QA metric: product-question/technical conversation deemed resolved without human escalation; cancellation: completed end-to-end without handoff. Neither is a reported independently customer-confirmed result or seven-day recontact. The paper says monitored CSAT, thumbs, latency, moderation and escalation did not regress, without publishing those control/treatment rates. It ended at two weeks after planned four-week sequential monitoring; durable changes and reopens unmeasured.

Important metric separation: Its third headline, routing accuracy 38.2%→68.8% (+30.6 points [23.6, 37.6]), uses 356 decisions per condition on a fixed labeled evaluation set, according to Table 5’s note. It is not a routing improvement measured on the large user-randomized live cohorts, despite being in the Online Experiment section and abstract beside live self-serve outcomes. Report it as a fixed-label test, not a third live customer endpoint.

Lifecycle and human ownership. An LLM generates/crosses/mutates candidate prompts; business/policy invariants are hard filters; a modular judge scores intent/grounding/translation and is calibrated against periodic human labels. Runtime bundles prompt and tool-use configuration as versioned artifacts rather than code deploys; content has snapshots and retirement pointers. Weekly or regression-triggered evolution takes hours to days; candidates undergo offline checks and staged rollout, with artifact rollback and fallback to human support. A polluted vector index reportedly triggered targeted redirection to a known-safe content pool while reindexing, cutting recovery from hours to minutes; not a measured cohort outcome. Developers and support operators have new work curating authoritative articles, evaluator rubrics, policy constraints and rollback gates. The paper does not give human annotation/coaching labor or quantify wrong actions. It reports an offline 100-chat human-validated simulation and 100 premium-support-chat evaluator alignment check, not a production-wide independent QA audit.

Inference and limits. A real customer-facing agent variant can increase bounded self-service relative to a previous agent in the same product; this is positive evidence against ‘nothing is ready.’ It does not compare to human-only care, isolate which bundled optimization component mattered, measure task action correctness or prove customer-confirmed closure. The system used GPT-4o-mini as generator and GPT-4.1 as offline evaluator at paper time; proprietary corpus and single enterprise surface constrain transfer. Compare Taobao human-vs-eligible deployment cautiously, because population, baseline and endpoints differ. Internal autonomy is a separate proposition.

Stop / next check. Look for consecutive release-cohort customer-confirmed outcomes, sampled wrong closures, one-week recontacts, agent versus human intervention minutes and a rollback trace; do not convert self-serve into resolution. The newer Nubank A/B adds satisfaction but not recontacts.