Transport-level Testing of NVMe Devices using VFIO
summary
Demonstrates a user-space NVMe driver built with the VFIO user API against a QEMU emulated device, implementing the admin IDENTIFY command by mapping BAR0/BAR2, allocating IOVA pages with a simple IOVA allocator, writing 64‑byte SQ entries, ringing 32‑bit doorbells, and waiting on an eventfd for MSI/MSI‑X interrupts. Shows a working path for admin SQ/CQ, acknowledges completions, exercises the controller memory buffer, and surfaces a QEMU bug rejecting IOVA 0; full demo completes in under 45 minutes.
AI-written summary · about
chapters (28)
- 0:00Intro
- 0:31Outline
- 1:11NVMe in N Slides
- 2:55Submission Queue Entry - 64 bytes
- 3:59Command Queue
- 5:43PCI Configuration Space
- 7:12NVMe Controller Registers (MBAR)
- 7:51Setting up the Admin Queues
- 8:26Queue Doorbells
- 9:21PCI Doorbells
- 10:59Completion Queue Entries - 16 bytes
- 13:04Polling for "phase" change
- 14:28Interrupts
- 15:14"Free'ing" Completion Queue Entries
- 16:35Low-level NVMe
- 19:09Putting the Theory into Practice
- 19:51How Low Can You Go?
- 20:12Virtual Function I/O Framework
- 21:22I/O Memory Management Unit
- 23:14VFIO is boiler-plate heavy
- 29:42libvfio-NVMe specific functionality
- 31:00Emulating this stuff
- 33:50VFIO-based NVMe Controller Initialization
- 34:56Identify Controller
- 36:08Submitting command and kicking the queue
- 36:50Wait for completion and ack
- 41:41The Controller Memory Buffer
- 47:33Rounding up
related talks
29mVirtual Device Fuzzing Support in QEMU
21mVFIO: Very Frightening I/O? Taming Wild Guests and their PCIe Config-Space Abuse
30mHA-IOV: Applying Hardware-assisted Techniques to IO Virtualization Framework
27mIOMMUFD Integration in QEMU
39miommufd discussion
29mvfio, iommufd: Enabling user space drivers to vend more granular access to client pro...
41mTowards a Cross-Hypervisor Test Framework for VirtIO Backend Validation Using VirtIO...
21mOak stage0: a minimal firmware for (confidential) virtual machines