1. Three ways to reach a device
Hardware now virtualises the CPU and memory at close to native cost. Input and output is where every hypervisor still makes real design decisions, because a device is shared state that more than one operating system wants to own. The same question appears in a vehicle cockpit that runs an instrument cluster beside an infotainment guest, in an industrial controller that hosts a Linux HMI beside a real-time partition, and in a cloud server running thousands of tenants: who touches the hardware, and what does everyone else see?
A guest operating system can reach a device in one of three ways.
- Emulation. Software pretends to be a real, well-known device, such as a common network card or UART. The guest runs its unmodified driver, and every register access traps to the hypervisor and is handed to a device model. Compatibility is perfect; performance is poor, because a single packet can take many register accesses and therefore many exits.
- Paravirtualisation. The guest knows it is virtualised and uses a driver written for an interface designed for that situation. Requests are batched in shared memory and the device side is notified once per batch. VirtIO is the standard interface of this kind.
- Assignment (pass-through). A physical device, or a hardware-partitioned slice of one, is given to a single guest. The guest's native driver programs it directly; an IOMMU confines the device's memory accesses to that guest. Performance is close to native, at the cost of sharing and portability.
Real systems mix all three. A typical consolidated platform assigns a latency-critical device to the partition that needs it, uses VirtIO for storage, networking and consoles, and keeps a handful of emulated devices (an interrupt controller, a timer, a UART) because the guest expects them to exist.
2. What the hypervisor provides
The old split between type-1 (bare-metal) and type-2 (hosted) hypervisors is less useful than two sharper questions: which code runs at the highest privilege, and where do the device models run? Device models parse data that a guest controls, so their location decides how much a single bug can reach.
On Arm, the hypervisor runs at exception level 2 (EL2), guest kernels at EL1 and applications at EL0. Four mechanisms do most of the work:
- Stage-2 translation. The guest maps virtual addresses to what it believes are physical addresses (intermediate physical addresses, IPAs). The hypervisor's stage-2 tables map IPAs to real physical addresses. Any IPA without a valid mapping, such as an emulated device's register window, faults to EL2. That fault is how a register access becomes a trap.
- Trap controls.
HCR_EL2and related registers decide which instructions and system-register accesses trap to EL2. - Virtual interrupts. The GIC's virtualisation support lets the hypervisor present interrupts to a vCPU through list registers. GICv4 adds direct injection of message-signalled interrupts (vLPIs) into a running vCPU, and GICv4.1 extends it to virtual SGIs, which removes the hypervisor from some interrupt paths entirely.
- Virtual timers. Covered below, because the details are often misunderstood.
KVM is the hypervisor built into Linux. It multiplexes CPUs between the host and guests, saving and restoring each vCPU's state on exit and entry. It does not create devices or load guest images: a user-space virtual machine monitor (VMM) does that through ioctl calls on /dev/kvm. QEMU, crosvm, Firecracker, Cloud Hypervisor and kvmtool are VMMs; rust-vmm is a shared set of Rust crates that several of them are built from, not a VMM of its own.
VHE (Virtualization Host Extensions, Armv8.1, enabled by HCR_EL2.E2H and TGE) lets an unmodified host kernel run at EL2, with its EL1 system-register accesses redirected to the EL2 equivalents. A guest exit then lands directly in the host kernel instead of bouncing through a small EL2 stub. The non-VHE layout has not disappeared: protected KVM (section 11) deliberately keeps the host kernel at EL1 so that the EL2 code can protect guests from it.
Type-1 designs such as Xen, Gunyah, the QNX hypervisor and L4Re keep a small component at EL2 and run everything else, including device models and physical drivers, in a privileged VM or in user tasks.
Virtual timers
The Arm Generic Timer has a system counter, an always-on source that increments at a fixed frequency and is broadcast to every core, plus per-core timers that compare against it and raise an interrupt at a programmed point. Architecturally there is a physical count (CNTPCT) and a virtual count (CNTVCT), where the virtual count equals the physical count minus an offset that only EL2 can set (CNTVOFF_EL2). Guests use the virtual timer, so the hypervisor can hide time spent descheduled or make a migrated guest's clock continuous. Linux on arm64 reads the virtual counter even on bare metal, where the offset is normally zero. FEAT_ECV (Armv8.6) adds an equivalent offset for the physical counter.
A common misreading is that the virtual timer delivers virtual interrupts. It does not. When a guest's virtual timer fires, the core receives an ordinary physical private interrupt, and the hypervisor must turn it into a virtual interrupt for the right vCPU. KVM links the two so that when the guest deactivates the virtual interrupt the physical one is deactivated as well, which avoids a second trap. Timer delivery therefore still has a hypervisor hop, and its latency matters for any real-time guest.
3. VirtIO: device model and transport
VirtIO is a standard, maintained by the OASIS VIRTIO Technical Committee, for devices whose driver and implementation agree on a simple shared-memory interface. Version 1.3 was published in 2023; version 1.4 reached Committee Specification 01 on 8 April 2026 and defines 23 device types, from network, block and console to GPU, sound, SCMI, GPIO, CAN and SPI. It also adds administration virtqueues and device groups, used to manage one device from another.
The specification has two halves, and keeping them apart explains most of what follows.
- The device model is the same for every device: a status field the driver walks through during initialisation, feature bits negotiated between driver and device, a device-specific configuration space (a MAC address, a disk's capacity) and one or more virtqueues that carry requests.
- The transport says how the driver finds the device, reads and writes its configuration, and notifies it. The specification defines three: PCI, memory-mapped I/O (MMIO, common on Arm and embedded systems because it needs no PCI bus) and Channel I/O (s390 mainframes). A fourth, message-based transport is being standardised (section 12).
Nothing in either half requires a hypervisor. The same driver works whether the device is implemented by a VMM, by a backend in another VM, by a network card that speaks VirtIO in silicon, or by a remote processor across a shared-memory link. That independence is why VirtIO became the common device layer for cloud VMs, Android's virtualisation framework and automotive consolidation platforms.
| Linux source | Role |
|---|---|
drivers/virtio/virtio.c | Virtio bus: matches drivers to devices, runs the status and feature handshake. |
drivers/virtio/virtio_ring.c | Virtqueue implementation, split and packed formats. It is not a transport. |
drivers/virtio/virtio_pci_modern.c, virtio_pci_legacy.c, virtio_mmio.c | Transports. virtio_pci is a transport, not a device type. |
drivers/net/virtio_net.c, drivers/block/virtio_blk.c, drivers/char/virtio_console.c, drivers/virtio/virtio_balloon.c, drivers/gpu/drm/virtio/ | Device drivers, one per device type. |
4. Bringing a device up
Every VirtIO device is initialised by the same handshake, whatever its type or transport. The driver resets the device, announces itself, negotiates features, sets up its queues and only then declares itself ready.
FEATURES_OK is how a device refuses a feature combination; a driver that skips it can run with features the device never agreed to.A few feature bits shape everything downstream and are worth knowing by name:
VIRTIO_F_VERSION_1(bit 32) marks a modern, specification-compliant device rather than the pre-standard "legacy" interface.VIRTIO_F_ACCESS_PLATFORM(bit 33) tells the driver that the device's access to memory is limited or translated by the platform, for example by an IOMMU or by a hypervisor that only lets the backend see shared pages. A Linux driver then routes buffers through the DMA API instead of handing the device raw guest-physical addresses. Xen grants and protected guests both depend on it.VIRTIO_F_RING_PACKED(bit 34),VIRTIO_F_EVENT_IDX(bit 29) andVIRTIO_F_INDIRECT_DESC(bit 28) change the queue format and notification behaviour described next.
Initialisation is deliberately chatty. Each register access traps, which is acceptable once per device at boot. The design goal is that steady-state I/O touches registers as rarely as possible.
5. Virtqueues
A virtqueue is a set of ring buffers in the driver's memory that both sides read and write under a strict ownership rule: each field has exactly one writer. The split format, the original and still the most common, uses three areas.
- The descriptor table, written by the driver. Each 16-byte entry gives a buffer's address and length, flags (
NEXTto chain,WRITEif the device writes into it,INDIRECTfor a descriptor that points to a further table) and the index of the next descriptor in the chain. - The available ring, written by the driver: the head indices of chains ready for the device, plus a running
idx. - The used ring, written by the device: the head index of each completed chain and how many bytes the device wrote into it.
next fields, publishes the head index in the avail ring and bumps idx. The device walks the chain, fills the data and status buffers, and returns the head in the used ring with the number of bytes it wrote (4,096 + 1).The driver never performs DMA itself. It fills descriptors and rings with ordinary CPU stores into its own memory, then issues a memory barrier so that the device sees the descriptors before the index that publishes them. The device side reads and writes that memory: by DMA when the device is hardware, through a mapping of guest RAM when it is software.
The packed format (VirtIO 1.1 and later) folds the three areas into one descriptor ring. Each descriptor carries an ID and two flag bits that, together with a wrap counter, say whether it is available or used. Driver and device walk the same ring, which touches fewer cache lines and suits hardware implementations, at the cost of a more subtle ownership protocol.
Notifications are the expensive part, because each guest-to-device kick is a trap and each device-to-guest interrupt is an injection. Both sides can suppress them. With VIRTIO_F_EVENT_IDX, each side publishes the ring index at which it next wants to be woken (used_event, avail_event), so a busy device that is already polling the ring receives no kicks at all, and a driver processing a burst of completions takes one interrupt for many buffers.
6. One request on KVM
Following a single request end to end shows where the costs fall. Take a virtio-blk read in a Linux guest, with the device implemented by a user-space VMM on a KVM host.
ioeventfd, KVM completes the trapped notify write in the kernel and lets the vCPU run on; without it, KVM returns KVM_EXIT_MMIO to the VMM's vCPU thread, which must handle the write before the guest continues. Orange arrows are notifications.Two kernel mechanisms make this path efficient. An ioeventfd lets the VMM tell KVM "when the guest writes to this address, just signal this file descriptor"; KVM completes the trapped write in the kernel without returning to user space. An irqfd works the other way: writing to it makes KVM inject a specific interrupt into the guest. Between them, the vCPU thread and the I/O thread never need to synchronise directly, and the device model can run on a different core from the guest.
Guest RAM is ordinary memory in the VMM's address space, registered with KVM as memory slots. That is what lets the device model walk descriptors and copy data with plain loads and stores. It is also the reason a classic VMM can read and write all of the guest's memory, an assumption that protected guests remove.
7. Moving the backend: vhost, vhost-user and vDPA
The data plane does not have to live in the process that created the VM. Three designs move it elsewhere while leaving the guest driver unchanged.
- vhost moves the virtqueue processing for a device into a host-kernel worker:
vhost-net,vhost-scsi,vhost-vsock. The VMM hands the kernel a description of guest memory (VHOST_SET_MEM_TABLE), the ring addresses and two eventfds per queue, one for kicks and one for interrupts.vhost-netthen moves packets between the guest's buffers and a TAP device without a round trip to user space. It still copies data between guest buffers and socket buffers; zero-copy transmit exists only as an opt-in module parameter,experimental_zcopytx, which is off by default. - vhost-user applies the same split across processes. The backend is a separate user-space program, connected to the VMM by a Unix domain socket. Storage and networking stacks (DPDK, SPDK), file-system sharing (virtiofsd) and the rust-vmm
vhost-devicebackends for I2C, GPIO, SCMI, sound and others use it. A backend written once can serve QEMU, crosvm or Cloud Hypervisor. - vDPA (virtio data path acceleration) puts the data plane in hardware. The device parses standard VirtIO rings, while its control plane goes through a vendor driver on the Linux vDPA bus.
vhost-vdpaexposes it to VMs;virtio-vdpaexposes it to host applications as an ordinary virtio device. The guest keeps a generic driver, so the VM stays portable across hardware.
vhost-user setup shows how the split works in practice: the VMM negotiates, shares memory and passes on the eventfds that KVM already uses for the queue, then steps aside.
SET_MEM_TABLE is the trust decision: unless the platform restricts it, the backend maps all of guest RAM, not only the buffers it needs.Process placement is also an isolation tool. crosvm, originally written for ChromeOS, runs each virtual device in its own process, forked from the main process and confined with minijail namespaces and a per-device seccomp policy. A device process can reach the rest of the VMM only through proxied bus operations, shared memory and the file descriptors its policy allows, so a parsing bug in one device does not hand over the whole VMM.
8. Xen: grants, event channels and IOREQ
Xen is a type-1 hypervisor. A privileged domain, Dom0, normally holds the physical drivers and the toolstack; driver domains can take over individual devices so that a driver failure does not take Dom0 with it. Xen gives domains three primitives for building split drivers:
- Grant tables. A domain grants another domain access to specific pages of its own memory and passes the grant references; the other side maps or copies them with a hypercall. The granting domain decides what is shared.
- Event channels. Lightweight, edge-style notifications between domains or between Xen and a domain.
- Foreign mapping. A sufficiently privileged domain can map any page of another domain without a grant. Dom0's toolstack relies on this to build and inspect guests.
Xen's own paravirtual drivers (netfront and netback, blkfront and blkback) were built on these primitives, with configuration in XenStore, and need no device emulation. VirtIO on Xen is different, because the guest's VirtIO driver expects PCI or MMIO registers. Those accesses trap to Xen, which forwards them as ioreqs to an IOREQ server: a device model in Dom0 or a driver domain that has registered for that address range. QEMU can act as one; so can standalone backend daemons. The IOREQ mechanism, long used on x86, was extended to Arm so that virtio-mmio devices work there too.
Memory access is the sharp edge. A backend that uses foreign mapping can read any page of the guest, so it must be fully trusted, and it must run in a domain privileged enough to do so. Grant-based VirtIO removes that requirement. The guest advertises VIRTIO_F_ACCESS_PLATFORM, and Linux's Xen grant DMA layer turns each buffer the driver hands to the device into a grant reference, so the backend can map only pages the guest has explicitly shared. Linux gained this in 2022, first for Arm and from Linux 6.1 for x86_64 as well.
9. QNX, L4Re and Gunyah
A common principle runs through the hypervisors used in safety- and security-sensitive embedded systems: keep the code that parses guest-controlled data out of the most privileged component, and give each device model as little authority as possible. A malformed descriptor from one guest should be able to compromise, at worst, one deprivileged process.
- QNX Hypervisor. The QNX microkernel (
procnto) acts as the hypervisor, and each guest is hosted by its ownqvmprocess. Virtual devices (vdevs) run inside that process: emulated devices such as timers and UARTs, and paravirtualised VirtIO devices including virtio-blk, virtio-net and virtio-console; VirtIO 1.1 and later is supported. Unmodified Linux and Android guests use their standard VirtIO drivers. Vdevs reach physical hardware through ordinary QNX drivers using the OS's message passing, and devices can also be passed through to a guest. - L4Re. The Fiasco microkernel provides threads, address spaces, IPC and capabilities; the VMM,
uvmm, is a user task per VM. A VirtIO device can be implemented insideuvmmor by a separate server task connected through an IPC capability, with the rings and buffers in a dataspace shared between the two. The NVMe driver server, for example, can export a namespace to a Linux guest as a virtio-blk device, and a virtio network switch connects the network devices of several VMs. - Gunyah. Qualcomm's open-source type-1 hypervisor runs at EL2 and keeps policy out of the hypervisor: a resource manager VM handles VM lifecycle, memory sharing, interrupt routing and capability assignment. VMs communicate through hypervisor-buffered message queues and doorbells, and paravirtualised devices use the virtio-mmio transport with the VMM and backends in a host Linux VM. Linux support has been posted as patch series since 2022 and re-posted as an RFC in April 2025; check its upstream status before depending on it.
qvm process per guest; L4Re lets a virtio device be served by a separate task connected by a capability; Gunyah keeps policy in a resource-manager VM and device models in a host VM.| Stack | Most privileged code | Where virtio backends run | Backend's view of guest memory |
|---|---|---|---|
| KVM + QEMU | Host Linux kernel with KVM | VMM process (one per VM) | All guest RAM |
| KVM + vhost | Host Linux kernel with KVM | Host kernel worker | All guest RAM |
| KVM + vhost-user | Host Linux kernel with KVM | Separate backend process | All guest RAM unless restricted by the platform |
| crosvm | Host Linux kernel with KVM | Per-device sandboxed processes | Guest RAM via shared mappings |
| Xen | Xen | IOREQ server in Dom0 or a driver domain | Any page (foreign mapping) or only granted pages |
| QNX Hypervisor | QNX microkernel | qvm process per guest | That guest's memory |
| L4Re | Fiasco microkernel | uvmm or a separate server task | Dataspaces shared with it |
| Gunyah | Gunyah | Host Linux VM | Memory shared or lent through the resource manager |
10. Device assignment with VFIO
When a guest needs a device's full performance or a feature no VirtIO type models, the device is assigned. On Linux, VFIO is the framework that does it. It binds a device (vfio-pci for PCI, vfio-platform for memory-mapped platform devices) to a user-space owner, exposes its registers and interrupts through file descriptors, and programs the IOMMU so that the device's DMA is confined to memory the owner has mapped. The newer iommufd interface manages those IOMMU mappings.
VFIO does not depend on KVM: user-space drivers such as DPDK and SPDK use it on bare metal. A VMM combines the two, mapping the device's registers into the guest through stage-2 and routing its interrupts through irqfd, or on GICv4 hardware injecting them directly into the running vCPU. A small KVM-VFIO pseudo-device tells KVM which VFIO groups a VM uses.
The trade-offs are structural. Isolation rests on the IOMMU and on how devices are grouped behind it: devices that can reach each other's DMA without passing the IOMMU must be assigned together. A device holding state is hard to live-migrate. Sharing one device among VMs needs hardware partitioning such as PCIe SR-IOV virtual functions or a vendor-specific scheme, as with many GPUs.
11. Protected guests: pKVM and Arm CCA
Classic VirtIO assumes the device side may read and write any guest memory. Confidential and protected virtual machines reverse that assumption: the host, including its VMM and backends, is outside the guest's trust boundary. The device model survives; the memory model changes.
- pKVM (protected KVM) runs KVM in the non-VHE layout with the host kernel deprivileged at EL1. The EL2 hypervisor owns the stage-2 tables for the host as well as the guests, so once memory is donated to a protected guest the host can no longer map it. It has shipped in Android's virtualisation framework for some time. Protected-guest support upstream is arriving incrementally: a January 2026 patch series adds protected guest memory and hypercalls for a guest to share pages back with the host and to unshare them, and states that CPU register state and DMA are not yet isolated from the host.
- Arm CCA (Confidential Compute Architecture) runs guests as realms, with the Realm Management Monitor mediating between them and the normal-world host. In Linux realm guests, DMA always goes through swiotlb bounce buffers that the guest marks as shared with the host, because a VMM-emulated device cannot reach arbitrary realm memory.
Both depend on VIRTIO_F_ACCESS_PLATFORM, which sends the driver's buffers through the DMA API, where the bounce happens. The cost is a copy per I/O and a fixed-size shared pool. The limit is more important than the cost: shared memory is visible to the host in the clear. Confidentiality of the I/O itself needs end-to-end protection above the device (TLS, disk encryption) or a trusted assigned device that the guest can attest and talk to over an encrypted link. The Arm CCA and KVM deep dive covers how authority is split between KVM and the RMM.
12. Beyond VMs: virtio-msg
PCI and MMIO transports assume that one side can trap the other's register accesses. That holds for a hypervisor and its guests, but not for a secure partition behind a firmware interface, a microcontroller beside an application processor, or two systems joined by PCIe. Those environments already have messaging mechanisms, and virtio-msg is a proposed transport that uses them.
virtio-msg keeps the VirtIO device model unchanged and turns control-plane operations into messages: GET_DEVICE_INFO, SET_DEVICE_STATUS, GET_DEVICE_FEATURES and SET_DRIVER_FEATURES, GET_CONFIG and SET_CONFIG, GET_VQUEUE and SET_VQUEUE, and events such as EVENT_AVAIL, EVENT_USED and EVENT_CONFIG. Bulk data still moves through virtqueues in shared memory. A separate bus binding defines discovery, memory sharing and interrupt delivery for each environment; informative bindings cover Arm's Firmware Framework for A-profile (FF-A) and PCIe for asymmetric multiprocessing, and a Xen binding is under discussion.
As of late 2026 virtio-msg is an RFC developed by Linaro and Arm with community contributors; it is not part of VIRTIO 1.4, and the Linux support is experimental. Its significance is architectural. One driver stack and one device model could serve a VM, a secure-world service, a companion MCU and a peer host.
13. Choosing an approach
No single approach wins on every axis. The useful comparison is who processes guest-controlled data, what each I/O costs, and what a failure in that component can reach.
| Approach | Data path cost | Sharing and portability | Who parses guest data | Typical fit |
|---|---|---|---|---|
| Full emulation | A trap per register access | Shareable; any guest with the stock driver | Device model | Legacy guests; boot-time devices |
| VirtIO in the VMM | Kick and interrupt per batch, plus a process hop | Shareable; portable; migratable | VMM process | General-purpose VMs; development |
| vhost | No user-space hop | As above | Host kernel | High-throughput networking |
| vhost-user | No VMM hop; backend may poll | As above; backend reusable across VMMs | Dedicated process | DPDK/SPDK, virtiofs, device-specific backends |
| vDPA | Hardware data path | Guest driver stays generic; needs capable hardware | Device silicon | Cloud and telco networking |
| Assignment (VFIO) | Near native | One owner unless partitioned in hardware; hard to migrate | Guest driver; IOMMU confines the device | Latency-critical or feature-rich devices |
| Xen, grant-based VirtIO | Trap, event channel, grant map per buffer | Shareable | Backend domain, limited to granted pages | Disaggregated, least-privilege systems |
| Microkernel vdev (QNX, L4Re) | Trap plus IPC to the device process | Shareable | Deprivileged per-guest process or server task | Mixed-criticality and certified systems |
Two questions settle most decisions. First, does the timing budget of the most critical consumer survive the notification path and any shared backend, measured rather than assumed? Second, if the component that parses guest-controlled rings is compromised, what can it reach? The answers usually lead to a mix: assignment or hardware partitioning for the few devices with hard deadlines, VirtIO with deprivileged backends for the rest, and explicit sharing wherever the guest does not trust the host.
The first question does not stop at the device. From chain of trust to chain of timing follows a sensor sample across every hop on a consolidated platform and asks the same question of each.
14. References
- OASIS, Virtual I/O Device (VIRTIO) Version 1.4, Committee Specification 01, 8 April 2026; Version 1.3, 2023.
- Linux kernel, drivers/virtio and drivers/vhost/net.c.
- Linux kernel documentation, KVM API (ioeventfd, irqfd, memory slots) and VFIO.
- QEMU, Vhost-user protocol.
- crosvm, Architecture.
- Xen, IOREQ feature (+ virtio-mmio) on Arm patch series; virtio: restrict memory access under Xen using the xen-grant DMA-mapping layer; grant-based VirtIO for x86_64 in Linux 6.1.
- QNX, Hypervisor architecture, Virtual devices and VIRTIO (QNX Hypervisor 8.0).
- L4Re, Using the NVMe server with a Linux guest VM; virtio-net-switch.
- Gunyah, documentation patch in the April 2025 Linux RFC series.
- W. Deacon, KVM: arm64: Add support for protected guest memory with pKVM, January 2026.
- S. Price, arm64: Enforce bounce buffers for realm DMA, 2024.
- Linaro, What is virtio-msg? and How virtio-msg works, 2026.
- Arm, Learn the Architecture: Armv8-A virtualization (stage-2 translation, VHE, virtual interrupts and timers).
About this reference
Written by Lei Zhou. Text and diagrams © 2026 Lei Zhou, licensed CC BY 4.0.
Specification versions and upstream status reflect the position in November 2026 and move quickly. Check the linked primary sources before relying on them.
Corrections, questions or measurements?
If something here is wrong or out of date, or you have measured a notification path that confirms or contradicts it, I would like to hear about it. Comments are hosted in GitHub Discussions and need a GitHub account. Nothing loads from a third party until you click.