2026-03-26 23:08 UTC
Replies (4)
-
@robryk@social.wuatek.is 2026-03-27 02:22
@mirth @jamey @regehr Also, crashes that are hard to find by coverage-guided fuzzing (e.g. CVE-2023-4863), but easy to detect when they happen -- as crashes are.
-
@mirth@mastodon.sdf.org 2026-03-26 23:09
@jamey @regehr I suspect it's possible to construct ethically trained models capable of doing this work, though no such thing exists today.
-
@jamey@toot.cat 2026-03-26 23:17
@mirth @regehr I meant a different thing. If you have a hypothetical tool that tells you whether your source code meets your requirements, run a fuzzer on that tool: it may eventually find an input program that satisfies your requirements. This is not that different from using an LLM in a feedback loop with such a tool, except a coverage-guided fuzzer can see inside the tool to make potentially better guesses about what result you want. I claim random fuzzing should be "smarter" than an LLM for that purpose.
-
@regehr@mastodon.social 2026-03-26 23:28
@mirth @jamey so here's our paper (the one referenced in the post) using randomized synthesis. this is as good as we can do, so far. we might be able to do better with more work, but I don't know. but regardless, the difference between the LLM and randomized synthesis is night and day, it's not even close, and I strongly doubt we can close this gap. https://users.cs.utah.edu/~regehr/papers/popl26.pdf