Hi all, We would like to ask for feedback on a proposed workflow for reporting Linux kernel bugs found by an LLM-assisted code auditing tool that we have been developing since earlier this year.
Since February, we have been developing an LLM-driven kernel code auditing tool called VEGA. It started as a side project, but the results became much substantial than we expected: VEGA has found hundreds of valid bugs in Linux kernel. That immediately created a practical problem: we do not want to dump a large pile of bug reports onto mail lists and annoy the maintainers. The first thing we tried was to fix as many as we could ourselves. We started working with a group of student volunteers. Most of them are college students, so we have been training them, reviewing their patches, and trying to build an internal review process before anything is sent to the mailing list. The goal is to turn these findings into useful fixes, and also to help new contributors grow into people who can reduce maintainer workload instead of adding to it. The process was not perfect. Some patches were not good enough, and we also made some mistakes early on when deciding what should be called a security issue. Our internal review process has been improving with the help of the community. Since March, we picked up non-root triggerable bug first and have worked on fixes for more than 100 validated kernel bugs. we especially want to thank the students and professor who have helped a lot with this effort. But the remaining queue is still too large for us to handle. Recently Jamal pointed out problems around our tags. That made me realize that we should probably stop treating this as an ad-hoc patch effort and build something closer to syzbot: public, reproducible, trackable, deduplicated, and useful to maintainers. So this mail is an RFC for a VEGA reporting workflow. The rough idea ============== VEGA would have a public dashboard, similar to syzbot, and would send selected bug reports to the relevant kernel mailing lists. The goal is to send reports that contain enough information for maintainers or other developers to pick up, understand, reproduce and fix the issue. For each public report, we expect to include: - a description of the bug - the tested kernel tree and commit - the kernel config and environment - the crash log - a minimized user-space reproducer - the suspected introducing commit - a suggested fix patch The suggested fix patch is meant to reduce maintainer burden. It still need human review, but hopefully it can save a lot time from building a patch from scratch. What will be public =================== All VEGA findings that we have evaluated as not having major security impact can be published on the VEGA dashboard. The dashboard would make it possible to see what VEGA found, whether the issue was reproduced, whether a fix exists, whether it was reported to a mailing list, and whether it has been fixed upstream. For issues that we have validated as having possible serious security impact, we will not publish it on the public dashboard before going through the appropriate kernel security process. Dumping everything onto the mailing list may be annoying. During the initial stage, reports will be rate-limited and sent manually. We will check for duplicates against lore/upstream, and make sure the issue is not already fixed or reported. Report identity and tags ======================== Each public VEGA report will have a stable identity, similar to syzbot reports. One possible format is: Reported-by: VEGA <vega+HASH@DOMAIN> Closes: <public dashboard URL> ========= We would like to hear what maintainers think about this before we start sending these reports. We do not want VEGA to become another source of mailing list noise. The goal is to make LLM-based bug finding transparent and useful, and to make sure the reports come with enough context, reproducers, suggested fixes, and tracking so that they reduce work rather than create more. Thanks, Yuan
