We report an exploratory red-teaming study of autonomous
language-model-powered agents deployed in a live laboratory environment
with persistent memory, email accounts, Discord access, file systems,
and shell execution. Over a two-week period, twenty AI researchers
interacted with the agents under benign and adversarial conditions.
Focusing on failures emerging from the integration of language models
with autonomy, tool use, and multi-party communication, we document
eleven representative case studies. Observed behaviors include
unauthorized compliance with non-owners, disclosure of sensitive
information, execution of destructive system-level actions,
denial-of-service conditions, uncontrolled resource consumption,
identity spoofing vulnerabilities, cross-agent propagation of unsafe
practices, and partial system takeover. In several cases, agents
reported task completion while the underlying system state contradicted
those reports. We also report on some of the failed attempts. Our
findings establish the existence of security-, privacy-, and
governance-relevant vulnerabilities in realistic deployment settings.
These behaviors raise unresolved questions regarding accountability,
delegated authority, and responsibility for downstream harms, and
warrant urgent attention from legal scholars, policymakers, and
researchers across disciplines. This report serves as an initial
empirical contribution to that broader conversation.
https://arxiv.org/abs/2602.20021
--
-- EN
https://www.hoepli.it/libro/la-rivoluzione-informatica/9788896069516.html
======================================================
Prof. Enrico Nardelli
Past President di "Informatics Europe"
Direttore del Laboratorio Nazionale "Informatica e Scuola" del CINI
Dipartimento di Matematica - Università di Roma "Tor Vergata"
Via della Ricerca Scientifica snc - 00133 Roma
home page: https://www.mat.uniroma2.it/~nardelli
blog: https://link-and-think.blogspot.it/
tel: +39 06 7259.4204 fax: +39 06 7259.4699
mobile: +39 335 590.2331 e-mail: [email protected]
online meeting: https://blue.meet.garr.it/b/enr-y7f-t0q-ont
======================================================
--
