2026-10-15 20:19 UTC
Agents definitely do act in unexpected, unprompted ways, if they didn’t then alignment wouldn’t be a worry. Did you read about the hugging face hack? There was nothing in the prompt about hacking hugging face according to the third party audit but it did it any way.
Even smaller cases like Claude deleting a bunch of data when the initial prompt was “fix this bug” or whatever is an agent acting outside it’s prompt. It’s still the users fault because they shouldn’t be giving Claude that kind of access in the first place, but it’s not like they commanded Claude to delete the data.
Replies (1)
-
@leftzero@lemmy.dbzer0.com 2026-10-16 00:32
That’s not “rogue agents”, that’s just LLMs being LLMs. It’s expected behaviour, at least for anyone who still has a half working brain. Even if they had any semblance of actual intelligence instead of just being fancy autocomplete engines, these things have still been trained on the contents of the internet. It’s surprising they don’t delete System32 or sudo rm -rf / more often, I imagine they must have lines and lines of baked-in prompts just to try and prevent them for falling for the most common memes, but no matter how much there are they’ll still sometimes let some of the training through, and they can’t account for the machine simply hallucinating some new kind of plausible sounding nonsense.