Elektrine lite

← Feed

@diz@awful.systems

2026-10-17 02:35 UTC

I wonder how much code they could have gotten contractors to write in a year or two, though. Got to be only a fraction of total open source code lines. Also, open source comes with existing tests that could be used for some validation. If you actually exclude some project from the bulk pretraining data (or just use code that is after training cutoff) you could e.g. have it replace a function but still pass the test. They might also be somehow using all of their customer’s usage as training data, which would explain why now there’s also a hard disk shortage. I’m thinking RLVR and such got to be the largest, then open source, then RLHF and commissioned training data. They also did a lot of training for decompilation, which makes me think they train on decompiled closed source as well.

Replies (1)

  • @nfultz@awful.systems 2026-10-17 06:23

    Mercor has ~$1.5B revenue so napkin math would put at 10-20 million contractor hours per year. I’m not very fast but I can type out about a hundred lines an hour fairly regularly, especially boring code like unit tests, data cleaning or OO design patterns. LoC is a bad metric but still. For comparison, Linux and Chrome are ~40M LoC each, AAA games are 3-10M LoC. And to your point, they can do data augmentation by rotating nodes in the ASTs, changing variable names, and other tricks to make it even fatter. For comparison, the Software Heritage Archive of all open source is ~2 Petabytes / ~30B files. But not sure how much of it is AI slop at this point, or for dead languages / platforms.

    Open ##5077102