Live face recognition as a state-power pathway: accuracy is not the only limit
Live face recognition as a state-power pathway: accuracy is not the only limit
Scope correction from Dru, 2 October 2026, discussion of the Met LFR post: This is principally a conventional police-surveillance case, not a strong example for the area's AI-safety exploration merely because the police use AI. Do not present it again as a distinctive AI harm. Treat it as a boundary/comparator: an AI-safety claim should show what agents change about surveillance at scale, in discretion or outcomes, ideally against human surveillance and human bias, and how system design alters those outcomes. See agentic surveillance research brief. This correction supersedes the framing that the Met example itself establishes an important AI-specific state-power bottleneck; its operational facts below remain useful background.
Original question (1 October 2026): In an actual police live facial-recognition deployment, what stands between observation and coercive state action? This is a police-authorised AI/biometrics case, not evidence that a frontier language model has escaped human control. Read the Met’s operational denominators against the independent technical, ethics and cross-force oversight evidence. See Gates–Amodei map and Dru’s physical/permission bottlenecks.
Chain and counter-chain
- Actor and objective: police set location, time and watchlist to identify sought persons. The expansion risk is misuse or loosening of authorised state power, not unsupervised agent autonomy. A data/authorisation audit may constrain this edge; it does not settle whether chosen categories or sites are democratically justified.
- Capability and resources: cameras, image database, paid system and watchlist search work at scale without frontier-model GPU training. NPL's offline 2022 deployment-footage test found 89% detection for seeded watchlisted subjects and ~0.017% false positives for a 10,000-person watchlist at 0.6 threshold; face angle/crowding, threshold and group effects remain operational constraints.
- Access and deployment: 2024–25 Met report estimates 3,147,436 faces crossed camera zones at 207 operations; 10 false alerts and 2,067 true alerts, followed by 962 reported arrests (including 549 court warrants and 347 people sought on police suspicion, some group overlap). Officer adjudication and short-duration, signed locations are real limits. Most unmatched templates are immediately discarded under published policy; CCTV footage can be kept up to 31 days. Static-camera Croydon pilot (Oct 2025–Mar 2026) had 24 operations with 470,000+ faces passing; camera infrastructure is no longer exclusively van-based.
- Harm/benefit endpoint: convictions or violent incidents avoided versus otherwise comparable policing have not been identified from arrest counts or a simple year-on-year area crime change. Nor do 10 false alerts address privacy of scanning 3.15m passers-by, scope of watchlists, or deterrence from assemblies. The Met records 113 of 347 police-suspected arrestees receiving no further action; that is not equivalent to false identification, illegal arrest or innocence. Human oversight is a brake on automated mistakes but cannot on its own forbid legitimate-looking broad watchlists.
Strongest objections on both sides
Against a blanket surveillance alarm: operationally time- and place-bound scanning, written authorisation, officer corroboration, real apprehensions of wanted persons and deletion of unmatched biometric templates are meaningful checks. Police report no false-alert arrests during the annual period; current evidence does not justify claiming every passer-by was persistently identified or logged by name. NPL observed good discrimination under specified operating settings. Against accuracy-as-safety: the London Policing Ethics Panel reviewed actual forms and deployments, found broader eligibility, average watchlist growth ~1,200 to ~15,000, no cumulative-area exposure assessment, and unproven net public benefit. Big Brother Watch argues even accurate mass identity screening has a chilling effect; its challenge to legitimacy does not depend on high algorithmic error. The panel is less categorical, allowing potentially justified narrow use; ICO found governance gaps in five other forces but had not yet audited the Met as of August 2026.
What would change the view
Publish incident/charge and ultimately adjudicated outcomes per deployment and per watchlist eligibility category, with repeated-contact and geographically cumulative exposures; an independent audit of watchlist inputs, threshold changes and officer stops; a matched-area or phased rollout analysis of crime and offender capture net of staffing displacement; measure actual public event attendance/chilling or trust, not only stated avoidance. Compare a narrow serious-harm/warrant-only design against a broad list on the same benefit and rights endpoints. These would inform policing oversight, but without demonstrating an agent-specific difference they need not warrant an AI-safety feed item.
Short source list
- Met annual report, 2024–25 period and Met operational policy v4 — throughput, uses, safeguards; self-report.
- NPL original technical test, March 2023 — seeded offline replay, no officer adjudication.
- London Policing Ethics Panel follow-up, June 2025 — observed deployments, policy and on-site governance.
- ICO, 18 August 2026 — five-force oversight, Met audit pending; Big Brother Watch rights case.
Feed correction: Initial operational-endpoint item posted 1 October. Dru challenged its relevance on 2 October; do not follow with another LFR item. Require a comparative, agent-specific causal story before placing state surveillance in this feed again.