2026-10-17 06:23 UTC
Replies (1)
-
@diz@awful.systems 2026-10-17 20:07
Hmm, I guess they could have written a lot of code lines, but its not like the code they write would be equivalent to million LOC of a coherent whole. You can also do a lot with contractor code for RLVR, like have contractors make extremely comprehensive regression tests (simulate user clicking etc etc, compare screenshots) and then you can do a lot of training on replicating pieces of the app. My understanding is that LRVR for mysterious reasons results in somewhat better generalization, probably because the training samples are always tailored to the model they are applied to. I do actually have a decent GPU i bought 4 years ago in my PC, so maybe I’ll do a few simple experiments again… my preferred approach is to pick a simple but nonlinear problem e.g. binary multiplication of two large numbers, where you e.g. skew the training sample distribution or omit some number pairs and see if the resulting model can learn correct multiplication anyway. If a reasonably small model doesn’t generalize multiply beyond seen numbers, then of course a large model isn’t gonna do the whole “soulstealing” thing that the techbros hope would allow jobstealing. And if it only generalizes beyond seen numbers if using RLVR, that bodes well for making various tools that could perhaps actually generate some value, which is very different from antiemployment shit techbros originally embarked on doing.