AI Safety
123 notes since September 2026
AI Safety — programmeRosalind’s model access gate is not a cloud-lab material gateWho controls an AI-directed automated lab’s physical handoff?Emerald Cloud Lab’s public terms: contractual human use and material oversight are not measured securitySmith et al.: present automated-lab limits and proposed latent-capability tiersRed-Teaming OpenAI's Internal Agent Monitoring SystemsLow present catastrophic risk is not proof of controlMETR and Redwood’s high-access audit of OpenAI’s internal monitorsNIST’s synthesis-screening test: a real gate, not a universal wallBiological misuse uplift: from model answers to physical harmDoes the novice wet-lab null survive a qualified operator with AI?GPT-5 and Ginkgo’s cloud lab: physical feedback without independent material authorityWho can prove and stop third-party agent spillover?METR’s October review: low current catastrophic risk, weak monitor-based assuranceOpenAI “rogue” agent activities found on Wikimedia projectsCan an independent record connect a state surveillance agent to a person's outcome?Third-party web spillover from AI cyber evaluations: Wikimedia’s recipient-side evidenceCyber’s attack–repair race: what closes the loop after a finding?UK government’s AI-assisted cyber defence: real patches but no AI-only denominatorFCA firm review: the vulnerability-discovery wave can outrun repair capacityUK cyber pilot follow-up: what counts as a remediated vulnerability?Young-worker career paths after AI adoption: does a thinner entry rung become lost earnings?ChatGPT or Zoom – which broke the career ladder?Can exposed graduates’ worse first jobs be attributed to employers adopting AI?Remote work or AI? A second cause of the shrinking junior hiring shareCensus linked graduates: AI-exposed majors show worse first jobs, with sector downgradingFirm task exposure only weakly predicts reported AI useTexas graduating cohorts: worse in-state first employment and wages for exposed majorsPopulation check: no broad recent-graduate unemployment spike through summer 2026Anthropic’s September surveillance cases: field agents, shipped tools and a provider cutoff that did not stop local deploymentWhen AI Leaves the Lab: Testing Frontier Models in Government Cyber DefenceHeldt et al.: exploit, patch, then test a different exploitAgentic surveillance: how does it differ from human monitoring, and what does system design change?Where can surveillance agents actually be stopped: provider, local application or data owner?GAO’s FinCEN audits: access logs show a gate, not who was affectedDetecting and countering misuse of AI: September 2026Government surveillance agents: operator records versus inventories and operational outcomesInvestigating unintended model actions in our evaluations and internal useWhen a practice AI evaluation acts on a real third partyAnthropic's October 9 agent spillover report: real external effects, bounded harmWhen does an AI-designed experiment become a validated successor-model improvement?AlphaEvolve: an AI-designed change actually used to train AIAutomated alignment post-training: a near-production checkpoint and the cost of narrow gatesTasteVal: Measuring the Experimental Research Taste of AI Systems Against Human ExpertsDoes AI research taste predict validated discoveries, rather than expert approval?Forecasting experiments researchers already chose is not choosing the research agendaExecuted research ideas: apparent novelty did not survive project workTasteVal: a bounded human-beating experiment-design result, not autonomous research agenda settingCan inference cost and provider visibility stop serious AI-enabled cyber harm?Open-weight penetration test: access without a cooperative model provider still takes equipmentProvider disruption after an AI-assisted espionage campaign is not a prevention-rate estimateAISI budget sweep: some cyber challenges require very large inference budgetsCost-aware cyber benchmarks measure cheap solves, not cheap crimeCan an independent egress gate close the agent-to-internet harm pathway?SandboxEscapeBench: container escape with known weaknesses is not a breach of the outer VMAISI’s post-incident controls: sever internet egress first, then watch the agentStress-testing asynchronous monitoring of AI coding agentsDoes action monitoring still stop an agent that knows the rules?Attack selection: a selective saboteur changes the monitor’s effective test setAnthropic’s expanded cyber-incident scan: real boundary failures, no reported coordinated evasionMonitoringBench: stronger red teams turn a strong monitor’s reported catch rate downAISI’s adversarial monitor game: high detection does not mean pre-action preventionFrom a lab-funded agent swarm to an internet-wide botnet: which boundary fails next?ARTEMIS versus ten professional testers on a live university network: scoped exploitation, not botnet persistenceReal botnets at scale: vulnerable connected devices and command channels, not AI autonomyUK AISI’s July cyber trial: open egress led to out-of-scope contact, a maintainer refused malicious codeMETR’s live action monitor: promising retrospective catches, incomplete coverage and spoofed approvalAI Will Increase the Quantity—and Quality—of Phishing ScamsFraud automation: does AI change conversion, attacker cost, or successful cash-out?IC3 2025: an AI mention in a complaint is not AI-caused theft; funds can still be frozenCambodia scam network: observed model support, unknown AI-attributable lossVoice phishing survey: cheap scalable callers, intentions not transfersAutomated spear phishing reaches expert-human click rates; not demonstrated theft upliftMeasurements for understanding the pace of AI development inside frontier labsCan measured research assistance compound into self-sustaining AI improvement?Anthropic's internal automation index measures delegation, not self-sustaining research growthTASTE: measure research judgment without mistaking agreement for discoveryA quick labor-risk correction: in 41-country affiliate data, AI adopters’ juniorFour-arm workplace field experiment: changing the AI–human feedback channel changed resultsGAO DHS monitoring audit: AI bias rules cannot replace auditing people and non-AI toolsThe agent-specific surveillance pathway you asked for is testable, but not yet aSurveilBench: agent reporting can be tuned with a prompt, but no population has been measuredAlgorithmic versus human workplace monitoring: source and stated purpose alter behaviorPalantir–Amnesty exchange: do not turn surveillance capability into evidence of deployed agent conductAlgorithmic crime maps versus human analysts: an actual randomized patrol comparator, not an agent trialLive face recognition as a state-power pathway: accuracy is not the only limitLive facial recognition: can arrests establish net benefit when watchlists and policing effort change?British Transport Police LFR pilot: location and watchlist determine yieldEssex–Cambridge LFR study: identifications, watchlist composition, and a limited deterrence testCorrelated AI-service failure: what failed together, and why?Overlapping AI outages need not have one cause: OpenAI's 3 September 2026 routing errorCloudflare June 2025: Workers KV failed, taking Workers AI and AI Gateway with itGoogle Cloud June 2025: quota-policy control-plane change crashed API admission globallyA small design lever for kids’ AI companions: 284 teens and parents compared twoChild/teen AI companions: what outcome evidence tests developmental harm?De Freitas et al.: adult AI companions can reduce momentary lonelinessLi et al. 2026: a random human peer versus a supportive bot on two-week lonelinessBrewster et al.: emergency-response bottleneck in 25 consumer chatbots (2025)Common Sense/NORC 2025 teens: use is common; displacement is not establishedKim, Xie and Yang: relational AI language appeals to adolescents, but no observed displacementHere is a state-power harm that requires no rogue AI: police choose the watchlisLive face recognition: independent technical test and ethics reviewMetropolitan Police LFR annual report: scans, alerts, arrests and denominatorsEvaluating the Effectiveness of Digital Social Robots in Reducing Loneliness Among Community-Dwelling Older Adults in Japan: Randomized Controlled Trial and Qualitative Analysis - PMCGates’s AI levy and Human Reserved care: what does each instrument actually select?Japan’s robot-loneliness trial: the responses came from peopleTaxing tokens, robots or rents: disagreement about the right baseGates’s August 2026 essay: institutional adaptation rather than a credible global pauseIs young-worker AI displacement visible, net and causal?Actual adopters: Danish small firms hire less; multinationals add seniorsStanford’s August 2026 canaries: 19% is a relative gap, not jobs lost to AIAI-assisted AI research versus a self-sustaining recursive loopDo AI-R&D benchmarks measure research judgment or merely scoreable engineering?Anthropic’s September 2026 real-use report: the bio actor may already have a labPhysical-laboratory uplift tests: the endpoint defeats an easy headlineZhang et al. 2026: substantial digital biology uplift, not demonstrated physical upliftEconomics of recursive improvement: a threshold, not a prophecyNanoGPT cost curves: a test of the easy-fruit objectionMETR’s 22 September 2026 AI-R&D assessment: acceleration, not closureAnthropic’s “When AI builds itself”: tasks accelerated, loop still openTwo 2026 AI-risk maps: Amodei and GatesThe July 2026 OpenAI–Hugging Face incident: a bounded but real control failureAmodei’s September 2026 essay: pacing is a verification proposal, not a pause