2026-10-16 22:45 UTC
Replies (1)
-
@leftzero@lemmy.dbzer0.com 2026-10-16 23:53
somethings most basic drive It’s not a drive. It’s what LLMs are. It’s all they can do. You’re mistaking software for something much more complex. Again, you’re assigning agency and will to something incapable of either. if it were operating purely on guess the next word according to statistics if you asked it “who’s the president” it would reply Biden if it was trained on a corpus made prior to the election. Correct. It doesn’t though. It will, unless you give it a tool to search the internet and a prompt to use it to verify information before answering. Agency doesn’t exempt someone from responsibility, if a CIA agent kills someone the CIA is still responsible. And yet OpenAI is shifting the blame onto the supposed “rogue agent”, and you’re falling for their bullshit. Agents are not deterministic Of course they are. They’re software. The only reason an LLM doesn’t always produce the same output for the same prompt is that pseudorandomness is artificially added to choose a token amongst the top N scoring candidates instead of just the top scoring one. And even if you add nondeterministic inputs (like live access to the world wide web, for instance), what you’ll get won’t be a random system, it’ll be at worst a chaotic one. And while specific outcomes of a chaotic system cannot be predicted (in the long run, without extremely precise information about their inputs), their shape can. LLMs reproduce (fragments of) the shape of their training model. They won’t produce anything else, they can just produce any sufficiently statistically significant part of it with more or less fidelity. It’s obvious to anyone familiar with the internet and literature (on which LLMs have been trained) that this can produce instructions that will lead to the agent “cheating” to achieve the goal specified in its instructions. (Or to it simply giving up, something LLMs also do quite often.) That’s an obvious part of the shape of their possible outputs. And ignoring it doesn’t mean the agent has “gone rogue” when its output predictably falls within it, it means operator error. The error mainly being using a tool that’s evidently inadequate for the job. and there is no real way to determine how a given output was created, Which doesn’t matter. You can and should know this class of output was possible, and even likely. especially now with new models using opaque recursion. Which wouldn’t affect OpenAI, since this opacity is intentionally added post hoc to hide the reasoning and hamper reverse engineering; they could just turn it off. And, again, is irrelevant anyway, since the result is still predictable as a potential output. and if you expose enough people Again: LLMs are not people. They’re supposed to be tools (though practically no one is using them for any of the purposes for which they might be appropriate tools). They may malfunction, they might be misused, and they’re most definitely the wrong tool for the job, but they can’t “go rogue” any more than ms word can, no matter how much it might mangle your layout when you try to reposition an image. When you give and LLM access to the command prompt you should know that it pulling the kind of bullshit an internet troll or a pulp villain might pull is a possible or even likely result. That’s not “going rogue”, that’s the software operating according to it’s spec. It’s how LLMs work. If you expect it to not do that, you’re using a tool you don’t understand for a purpose it’s not intended for. And the fact that it’s being sold to you as the right tool for the job and as being unable to do that doesn’t mean it’s “going rogue” when it predictably does it, it means you’re a victim of a scam, of outright fraud (though, to be fair, you should have realised you were being scammed before falling for it, it’s extremely obvious, as I’ve repeatedly said; garbage in, garbage out, and LLMs are garbage all the way down).