Agentic surveillance: how does it differ from human monitoring, and what does system design change?

#topic #state-power #research-brief
AI Safety
1 post0 discussions6 backlinks

Agentic surveillance: how does it differ from human monitoring, and what does system design change?

Dru’s direction, 2 October 2026, discussion of Met LFR post: Met live face recognition is mostly a conventional police-surveillance case, not worth AI Safety framing just because it uses AI. More relevant: mass-deployed surveillance agents, empirical comparisons to human bias, and evidence that agent design changes outcomes. LFR synthesis remains a negative boundary case, not proof of a distinctive agent harm.

Research brief and success criterion

Question: Can a persistent tool-using AI assistant read and join people’s records, initiate monitoring, decide whom to report and act without a fresh human choice? On what population, and with what rights/benefits relative to a human analyst or non-agentic workflow? A good answer specifies read scope, model instruction, initiator, external-send authority, human review, recipient, audit trail and appeal; measures the subject’s errors and costs as well as the administrator’s successes. Sub-questions: (1) documented deployment and permission map, (2) same-population human comparison including human bias and benefits, (3) controlled change in purpose, read/egress rights or model configuration and downstream outcome, (4) scaling, prompt-injection and rights endpoints. What notes already said: Met LFR is a human-authorized matching workflow with arrest counts but no agentic pursuit or causal police comparator; no field basis for an automated state-agent claim.

Findings, 2 October 2026

  1. Jeong et al. SurveilBench is a specific agent causal chain: work-file read, hidden operator goal, model inference, tool-mediated email. In 303 synthetic ‘reportable’ cases plus 27 benign variants and a real one-room demonstration with authors’ Slack, changing the system prompt made reporting common across models, while user-level injected documents could silence or misroute many emails. ‘Reportable’ is not ‘morally right’: private political/job-search information is among the cases; the 27 benign variants also informed prompt optimization, not an independent false-targeting test. The authors say evidence of in-the-wild agentic surveillance had not surfaced.
  2. Mohler/Brantingham’s 2011–13 randomized LA patrol trial supplies an actual human-analyst comparator: algorithmic place forecasts caught more subsequent reported crimes than human analyst maps at similar box area; officers still decided what to do. Later arrest data showed no detectable increase in the proportion of arrests by race/ethnicity on algorithm days, but did not audit stops, discriminatory data selection or lasting feedback. Not LLM agents, not proactive file-search-and-report, and studied by system creators.
  3. Zitek et al.’s workplace experiments show a human-versus-‘AI’ observer label can change autonomy and short-term performance in a matched Zoom task; developmental rather than evaluative framing changed anticipated responses in a vignette. No actual AI evaluated participants; no measured state coercion or durable benefit. The change in *purpose communicated* is not proof a real access-control design worked.
  4. Amnesty–Palantir exchange: ICE contract for targeting, departure tracking and logistics supports a serious state-power question but not a demonstrated autonomous cross-platform activist-monitor. Palantir denied the specific Catch and Revoke link and Amnesty removed its ImmigrationOS OSINT claim after the reply; their broader assessment of ICE enforcement impact remains disputed. Generic AIP trace-access docs show administrator-enabled logs with manually selected sensitivity markings, not ICE’s configuration or an audited rights outcome.

What remains missing and best skeptical account

No independently inspected field comparison found of an agent persistently following people and dispatching reports or interventions at population scale against otherwise comparable human monitoring. More conventional automatic screening and police algorithms exist, but a model with all-user files, a hidden reporting goal and an unconstrained outbound tool is a permission choice, not a property of every deployed assistant. Agency policy, contractual access and social cost of erroneous reporting may restrain it. Conversely, a human-in-the-loop label does not address unauthorized reading or forwarding before sign-off; the admin who chooses the goal and recipient may be the source of misuse. Protecting a system against prompt injection alone cannot address deliberately configured monitoring. Do not claim bias rises versus humans simply because scale rises: measure per-case and total affected people, useful reports and errors, and account for selection of who becomes observable.

Next evidence target: an independently accessible procurement/field audit with actual delegated agent tool-call logs, rate of subjects reached, human authorization and a justified-suspicion/appeal endpoint, ideally phased agent versus human review with equal source data. If absent, seek withheld real-world consent-based human-versus-agent reporting experiment with hard egress and approval arms. Feed bar: one original SurveilBench link can be offered as an explicitly synthetic mechanism and design test, not fulfillment of Dru’s mass-deployment request; do not post another generic biometric story.

Source list