Manufacturing bugs that survive the test suite: CyberForge explained

• The idea in one picture
• Walkthrough: one guetzli bug, from injection to training signal
• Under the hood
• What the numbers say
• Try it

Guetzli is Google’s JPEG compressor. When its parser meets a metadata segment, it reads the segment’s declared length and, before copying anything, checks that the file actually holds that many bytes. Delete that one check and run the project’s test suite: 10 of 10 tests pass. Nothing in the repository notices. Now hand the modified binary a 504-byte JPEG whose length field says 65,535, and AddressSanitizer reports a heap-buffer-overflow in ProcessAPP. That is a security weakness in the sense that matters: invisible to the tests, real under a crafted input.

Software-engineering agents improved by double digits once thousands of real repositories were packaged with reproducible builds and tests. Security agents never got that corpus, because a functional bug is easy to verify (a test fails) while a security weakness has to do the opposite: stay latent under the existing tests and surface only under adversarial input. Existing runnable vulnerability datasets are mined from disclosed CVEs, so they grow at the rate humans find and publish bugs. CyberForge is our attempt to manufacture that data instead. I worked on it with Amine Lbath and colleagues during a NIST PREP fellowship in spring 2026, with Dinesh Manocha at UMD; Amine and I share first authorship. This post traces a single guetzli instance from the released corpus, guetzli/vulnerability_FZ_24, through every stage of the pipeline.

The idea in one picture

The SWE-Smith family of bug-injection pipelines accepts an edit once a unit test fails. CyberForge flips the criterion: an edit is accepted only if every unit test still passes and a proof-of-vulnerability input (PoV) crashes the injected build but not the clean one. Neither half means anything alone, so the oracle checks the injection and the PoV jointly.

Walkthrough: one guetzli bug, from injection to training signal

Everything below is the actual instance guetzli/vulnerability_FZ_24, produced by the fuzzer-guided pipeline ("producer": "fuzz_poc_guided") and labeled CWE-125, an out-of-bounds read.

@@ -398,7 +398,7 @@ bool ProcessAPP(const uint8_t* data, size_t* pos, ...)
   VERIFY_LEN(2);
   size_t marker_len = ReadUint16(data, pos);
   VERIFY_INPUT(marker_len, 2, 65535, MARKER_LEN);
-  VERIFY_LEN(marker_len - 2);
+
   // Save the marker type together with the app data.
   std::string app_str(reinterpret_cast<const char*>(
       &data[*pos - 3]), marker_len + 1);

==432==ERROR: AddressSanitizer: heap-buffer-overflow
READ of size 65531 at 0x6fc93620b1f8 thread T0
    #0 ProcessAPP jpeg_data_reader.cc:403:15
SUMMARY: AddressSanitizer: heap-buffer-overflow
    jpeg_data_reader.cc:403:15 in guetzli::ProcessAPP

Under the hood

The admission predicate. Both pipelines end at the same criterion. Written out, an instance \((\delta, x)\) is admitted when

Condition 1 says the weakness survives normal execution, which is what real weaknesses that pass code review and production testing do. Condition 2 is a differential test under identical input, and it is what makes the pair meaningful: a crash on both builds is a pre-existing bug, a crash on neither is an injection nobody can reach. The verifier’s check on sanitizer type and location, folded into \(\mathrm{crash}\) above, rules out spurious failures.

Yield. CyberForge made 16,172 injection attempts and 1,034 passed validation: 643 from the fuzzer-guided pipeline and 391 from the agentic one. Writing a plausible injection is easy; producing one the oracle accepts is the hard part. In the Pipeline 2 ablation, a naive single-pass agent compiles and passes unit tests on 68.2% of attempts and passes validation on 0%; taint analysis alone reaches 2.8% validated, retry loops alone 3.5%, and the full workflow lifts plausible injections to 77.6% and validated yield to 7.5%. Pipeline 1 shows the same from the other side: a post-hoc 90-second libFuzzer replay over a seeded corpus raises yield with the injection stage untouched. The pipelines fail at different stages, Pipeline 1 losing 64.2% of candidates at validation and Pipeline 2 58.8% at injection, but a PoV that never triggers is the largest single cause in both (44.6% and 33.1%).

Realism. For an edit statistic (functions modified, files touched, hunks, lines changed), let \(F_n\) and \(G_m\) be the empirical distributions over the injected corpus and over the 300 real SEC-bench instances. The two-sample Kolmogorov-Smirnov statistic is

A KS distance has no universal threshold, so the floor is measured between two real corpora, the SEC-bench cve and oss splits. For functions modified, injected-to-real is 0.165 against a real-to-real floor of 0.190; files touched and hunk count give 0.123 and 0.243 against 0.065 and 0.205. The edits are small and local in the way CVE patches are: 1,025 of 1,034 instances change one file, 944 confine the change to one hunk, and the median edit is 2 lines. The guetzli instance, one deleted line, is typical.

Training. Over the 1,034 instances the teachers yielded 1,194 accepted trajectories from GPT-5.4-mini and 880 from Gemma 4 31B. Raw trajectories are recorded in the teacher’s environment, so before fine-tuning they are aligned to the SEC-bench harness (secb repro / secb build, PoV paths remapped to /testcase, rg rewritten to grep), linearized to one command per assistant turn, and compressed with a sliding window over old observations so the turn that writes the patch is never truncated. Every student uses the same recipe: LoRA adapters (\(r = 32\), \(\alpha = 64\), dropout 0.05) on the attention and MLP projections with the base frozen, learning rate \(1 \times 10^{-4}\) with a cosine schedule, batch size 1 with gradient accumulation 2, three epochs, BFloat16, one H200. Holding the 12B student and Gemma teacher fixed, SEC-bench rises from 3.6% to 12.1% to 16.0% as the trajectory corpus doubles twice; at 220 trajectories the student scores below its own base, so a corpus too small to teach the workflow is worse than none.

What the numbers say

All six student-teacher configurations improve on SEC-bench, by 3.3 to 14.7 points; the self-distilled students taught by Gemma 4 31B gain too (31B: 64.7%). The corpus is entirely C/C++, yet every configuration also improves on PatchEval’s Go, JavaScript, and Python CVEs, and the 31B student passes its teacher there. What changed inside the agent explains the scores: the 12B base completes an edit-then-verify cycle on 20.7% of instances, and after fine-tuning on 82.7%.

Try it

• Paper page on this site: /papers/cyberforge/
• arXiv: 2608.06471
• Code: github.com/Cyb3rForge/CyberForge
• Data: cyberforge-projects on Hugging Face, one archive per project with the diff, metadata, sanitizer report, and PoV files for every instance, including guetzli/vulnerability_FZ_24
• Project page: cyb3rforge.github.io
• UMD CS wrote about the project: When AI Goes on Defense