IEEE Symposium on Security and Privacy (S&P) 2027 · SoK

SoK: From Finding to Deployment:
Systematizing the OS Kernel Bug Lifecycle

Luyao Bai1, Gengda She2, Kenan Alghythee1, Hang Zhang2, and Xiaoguang Wang1

1University of Illinois Chicago  ·  2Indiana University Bloomington

Paper (PDF) Dataset SyzFix Code Classification BibTeX
  • 140research efforts & production systems surveyed
  • 6,946syzbot-fixed Linux kernel bugs measured
  • 35 daysmedian crash-to-patch latency
  • 26.7%of fixed bugs have no reproducer
  • ≤ 6%end-to-end LLM repair rate on 80 hard crashes

Abstract

Automated kernel bug discovery has advanced rapidly. Continuous fuzzing and static analysis systems, such as syzbot, now expose Linux kernel bugs at a scale that downstream processes struggle to absorb. Yet a crash report is only the beginning. Before a bug is eliminated, it must be triaged, understood, patched, validated, reviewed, integrated, and often backported. These later stages remain far less automated, creating a persistent gap between bug discovery and patch deployment.

This SoK systematizes the Linux kernel bug lifecycle from discovery to deployment. We organize prior work and production systems into five stages: discovery, triage, patch generation, patch validation, and integration. We explain the resulting automation gradient through kernel-specific challenges such as concurrency, implicit invariants, cross-syscall state, hardware dependence, lack of fault isolation, and architecture/configuration multiplicity. We further ground the analysis in a measurement of real syzbot-fixed bugs. The data shows that the crash-to-patch gap is not merely a backlog of unfixed reports but a structural failure mode of the repair pipeline: even after being fixed, bugs often remain open for weeks, require review-driven patch revisions, or lack reproducers that current repair and validation systems assume. These findings expose a deeper mismatch: today's repair and validation techniques often assume reliable reproducers, localized root causes, and checkable correctness oracles, yet these are precisely the artifacts missing from many real kernel bug reports. Closing the crash-to-patch gap, therefore, requires treating such artifacts as outputs to be produced, not prerequisites to be assumed.

The Kernel Bug Lifecycle

Five stages, from a fuzzer crash to a fix in every affected tree. Automation is mature at the front and thins to manual work at the back: the automation gradient.

  1. A0
    S1 Discoveryfuzzing, static analysis
  2. A1
    S2 Triagededup, impact, root cause
  3. A1/H
    S3 Generationrepair, backport
  4. H
    S4 Validationcorrectness, completeness
  5. M
    S5 Integrationreview, merge, stable

A0 production-deployed · A1 offline/prototype · H human-in-the-loop · M manual. Revisions and re-backports loop S5 back to S3.

A cross-cutting taxonomy applied uniformly to all five stages (Table 1 in the paper).
StageConsumes → ProducesDominant methodAuto.Oracle (“done”)Productive LLM roleChal.
S1 Discoverysource/binary → crash + reportcoverage-guided search, static analysisA0strong: crash / sanitizer tripsgenerator: specs, checkersC1, C3, C4
S2 Triagecrash + report → root cause, severity, dedupsymbolic exec., static, learningA1partial: exploit primitive, dup matchjudge: severity labelsC3, C5
S3 Generationroot cause → candidate patchtemplates, transforms, LLM agentsA1/Hweak: a single reproduceragent: propose patchC2, C6
S4 Validationpatch → correct/complete verdictdirected fuzzing, static, PoCHweak/none: no spec for completenessjudge: correctness (unverified)C2, C5
S5 Integrationpatch → merged + backported fixhuman review, social processMsocial: maintainer acceptancegenerator: review commentsC6

Why Kernel Bugs Are Different

Six cross-cutting properties of the OS kernel shape every stage. They weigh far more heavily on the later stages than on discovery, which helps explain the automation gradient.

C1

Pervasive concurrency

Data races, deadlocks, and concurrent-free UAFs belong to an interleaving, not an input. Most user-space APR and fuzzing assume sequential behavior.

C2

Implicit invariants

Lock ordering, refcount balance, RCU grace periods, no sleeping in atomic context: no test suite encodes them, so generate-and-validate has no specification.

C3

Cross-syscall state

Kernel objects persist across system calls. Triggering a bug needs a precise sequence, and the symptom can sit far from the faulty instruction.

C4

Hardware dependence

Drivers are most of the kernel and depend on devices, MMIO, DMA, interrupts, and firmware that many test environments lack.

C5

No fault isolation

No process boundary contains a fault. A benign-looking WARNING can hide an arbitrary write, and a faulty patch can leave the system unbootable.

C6

Config & tree multiplicity

One source builds for many architectures and thousands of configs, and ships through mainline plus stable and vendor trees. Fixes must hold, and be backported, across all of them.

Measuring the Crash-to-Patch Gap

6,946 Linux kernel bugs that syzbot reports as fixed, each linked through its full lifecycle: crash timestamps, reproducers, the reconstructed patch series (v1 → v2 → …), reviewer threads from lore.kernel.org, and the merged commit.

Left: CDF of crash-to-patch latency on log time, median 35 days, 13% exceed one year. Right: median latency by bug class, from data-race at 15 days to hang at 111 days.
Crash-to-patch latency. Left: CDF on log time; the median is 35 days and 13% exceed one year. Right: median latency by bug class. Semantically diffuse bugs (hangs, UAF) linger; sharp-signature bugs (races, null-deref) close fast.
Stacked bar of mean per-bug share of latency: 51% before first patch, 4% revision churn, 11% review and merge wait, 35% post-merge confirmation lag.
Where the time goes (mean per-bug share, n = 3,371). Just over half precedes the first posted patch. Revision churn is a thin 4% with a long tail, and a third is syzbot's post-merge confirmation lag.

Key findings

  1. Even among bugs that are eventually fixed, the gap is large and heavy-tailed: median 35 days, mean 138, 13% over a year. This is a conservative lower bound.
  2. Since continuous fuzzing matured in 2019, the median has held at about three weeks for six years. The gap is a structural property of the back end, not a transient.
  3. About 51% of measured latency occurs before the first patch, 4% in revision churn, 11% in review and acceptance, and 35% in syzbot's post-merge confirmation.
  4. Latency varies more than 10× across subsystems (median 6–212 days), and reproducer scarcity ranges from 7% to 32%. No single intervention closes the gap everywhere.
  5. 21% of fixes take two or more review-driven revisions, and a long tail draws 40+ reviewer messages.
  6. 26.7% of fixed bugs lack any reproducer and a third lack a C reproducer, yet the typical fix is small (median 1 file, 5 lines). Only 34% are “repair-ready”.
  7. No bug class has dedicated automation across all four technical stages. Coverage is dense at discovery and triage and collapses at generation and validation.

Coverage-gap matrix

Strongest automation per (bug class × technical stage). The empty lower-right is the crash-to-patch gap, drawn at the technique level.

Bug class# fixedDiscoveryTriageGenerationValidation

● dedicated kernel technique   ◐ generic only   ○ none

LLM Patch-Generation Benchmark

Thirteen LLM patch-generation configurations on 80 evolution-stage syzbot crashes (bugs whose first upstream fix was itself revised). Each candidate is scored for localization and, by two semantic judge models with author re-judging of disagreements, for repair.

Localization is far easier than repair. Methods touch a correct file 55–66% of the time, and 95–100% with an oracle. End-to-end repair never exceeds 5/80. Told the exact function to edit, models still fix only 2/80.
MethodLocalization %Localization setRepair (of 80)
anyfilefuncstructmacroFixedPartial

Surveyed Systems

The classification from the paper's appendix (Tables 6 and 7): every discovery system with its type, target, bug class, and technique families, and representative systems for triage, generation, validation, and integration with their automation level.

Download: CSV · JSON
System Stage Classification Venue Year
Loading…
Technique-family abbreviations

Discovery type: F = fuzzing, S = static, H = hybrid, E = empirical study. ★ = LLM-based. (U) = user-space contrast.

Artifacts

Everything behind the paper's figures and statistics is public.

SyzFix dataset

The syzbot-fixed bugs with crash reports, reproducers, patch revisions, reviewer threads, and merged commits. Hosted on Hugging Face.

SyzFix code

The scraper that builds the dataset from syzbot, lore.kernel.org, and kernel git, plus the analyzers behind the measurement section.

Classification

Stage, type, target, bug class, technique family, and automation level of the surveyed systems, as CSV and JSON.

Benchmark verdicts

The judge rubric, per-candidate verdicts, and rationales for the 80-crash patch-generation benchmark.

Ethics: all data comes from public sources on bugs syzbot had already marked fixed. No unfixed vulnerability, proof of concept, or exploit is included, and no LLM-generated patch was sent to kernel maintainers.

Citation

@inproceedings{bai2027sok,
  title     = {{SoK}: From Finding to Deployment: Systematizing the {OS} Kernel Bug Lifecycle},
  author    = {Bai, Luyao and She, Gengda and Alghythee, Kenan and Zhang, Hang and Wang, Xiaoguang},
  booktitle = {Proceedings of the IEEE Symposium on Security and Privacy (S\&P)},
  year      = {2027}
}

This work is supported in part by US NSF grants 2127491 and 2154199, and a Google Cloud gift award.