About this article. It is drawn from a customer delivery I led as technical delivery lead and architect; the case study covers the outcome.
1. Why gfxstream, and what the platform withholds
When virtual machines share one GPU, every frame a guest renders crosses the hypervisor boundary, so the design of that path decides how much performance the guest loses and how much CPU the host spends. With virgl, an Android guest gets GLES from the host and, on this platform, only software Vulkan. gfxstream works differently: guest-side encoders serialise the guest’s Vulkan and GLES calls, a renderer library on the host decodes them and replays them on the host’s Vulkan driver. The guest gets hardware Vulkan, and GLES can run on top of it.
gfxstream as usually deployed also assumes something this platform does not give it. Its command ring, its host-visible Vulkan memory and its host-allocated buffers all rely on the host mapping its own memory into the guest at runtime. This hypervisor stack cannot do that. crosvm’s own source describes guest-handle memory as useful for zero copy on hypervisors designed around static allocation of guest memory, such as Xen, so the constraint is not unique to this platform. Venus, the other Vulkan path for virtio-gpu, relies on the same kind of mapping: Mesa’s documentation describes host-visible memory exported as a mappable dma-buf and mapped into the guest by the hypervisor. Changing protocol would not have avoided the problem.
The root-cause fix is runtime host-to-guest sharing in the hypervisor stack. That was outside the delivery timeline, so it was recorded as the long-term recommendation, and everything in Part II is the design that works without it.
2. The stack and its three contracts
A gfxstream guest touches the host through more interfaces than the virtio-gpu device itself. Three of them decided the architecture.
- Contract A is the renderer’s C interface: the entry points, structs, parameter keys and flags that rutabaga_gfx calls.
- Contract B is the protocol between the guest encoders and the renderer’s decoders: opcode tables, the transport handshake and the capability structs the guest reads.
- Contract C is the numbering that lets guest userspace discover what the kernel and the device support: virtio-gpu feature bits and DRM parameter ids.
Each contract has a different owner and a different release cadence. The guest’s Android release was fixed here: it was the customer’s functioning production guest, built from an older AOSP release. The VMM was written for a newer renderer. A mismatch on any of the three would have stopped the migration.
3. Contract A: the renderer interface
The renderer interface was renamed as gfxstream evolved. Earlier renderer sources, from the project once called Vulkan Cereal, export most of the virtio-gpu interface as pipe_virgl_renderer_*; later releases export stream_renderer_*, which is what rutabaga_gfx binds. The renderer that matched the older guest came from the earlier line.
The first step was to compare the VMM’s rutabaga_gfx binding, entry point by entry point and with every parameter key and flag, against the earliest release that has the newer interface. Most matched. The binding also expected items that release did not offer in its stable interface, such as the renderer-features parameter key and the snapshot entry points.
A compatibility adapter in the VMM was designed for the older renderer, under rules written before any code:
- contained behind one interface and selected explicitly for older renderers;
- no feature logic inside it;
- independent of the host GPU driver;
- specified by a field-by-field map, with a contract test at every milestone gate.
Then the question changed. If a newer renderer could also serve the older guest, no adapter would be needed. Several later renderer releases were checked against the VMM: every entry point it binds present, and every struct identical in layout on the 64-bit Arm target. Argument layouts were not compared statically; the first runtime milestone was the proof for them. The newest releases were set aside, because they add capability fields, treat one pipe type as fatal and renumber handle types.
4. Contract B: the guest protocol
A renderer that the VMM can load is useless if it cannot decode what the guest sends. The older guest’s encoders and the newer renderer come from different release lines, so the protocol was compared from source as well:
- GLES and the render-control channel: the opcode tables were identical.
- Transport: the pipe handshake was equivalent to the original pairing.
- Vulkan: there was a set of calls the newer renderer cannot decode, all of which the older encoder could in principle send.
The Vulkan gap turned out to be unreachable. The guest’s Android extension allowlists are a subset of the features the renderer supports, so the guest never advertises, and never sends, what the renderer lacks. That was established from the AOSP sources, not by testing until nothing failed.
Two differences were real, and neither needed a change in the guest:
- A struct that differs by one field. The guest and renderer versions of one capability struct have the same size and agree on their first fields. One field the guest reads lies in what is padding on the host, so it always reads zero. It is harmless here only because every reader of that field takes the guest-handle branch, which this design always takes.
- A program-binary gap. The older encoder drops its program state after
glProgramBinary, so the UI’s cached shader binaries drew black. The renderer now reports zero program-binary formats. Skia clears its program-binary support when that count is zero, and then ignores cached binaries, which repairs both new and already-affected caches.
5. Contract C: numbers nobody owns
The guest discovers guest-handle support in two places: a virtio-gpu feature bit offered by the device, and a DRM parameter that the kernel answers. Neither number is standardised:
- The feature bit. The OASIS virtio 1.3 specification defines virtio-gpu feature bits 0 to 4, ending with context initialisation. The guest-handle bit is left to the VMM: crosvm, for example, defines a
CREATE_GUEST_HANDLEbit of its own, outside that range and not in the specification. - The parameter id. Android userspace defines parameter 8 as guest-handle support, for example in minigbm’s copy of the virtio-gpu uapi header. Upstream Linux assigned 8 to a different parameter, the context debug name, in 2023, and never added a guest-handle parameter.
So a guest kernel that offers guest handles cannot follow upstream Linux for that id and also serve its own userspace. It has to agree with the VMM on the bit and with userspace on the id. The fix therefore belongs in the guest kernel’s uapi header, declared as an explicit divergence from upstream for that id, rather than in each userspace project. The rule generalises: when out-of-spec extensions number the same thing, find the component that deviates from the contract its consumers use, and fix it there once.
6. The decision: one renderer, both contracts
The decision was to keep the production guest on its release, changing only its graphics libraries and kernel where the design required it, so that the change surface, and with it the maintenance and upstreaming effort, stayed small; and to host a newer renderer release that satisfies contract A natively and contract B for everything the guest can reach. The adapter stayed documented as the fallback.
| Option | Why not chosen |
|---|---|
| Older renderer with an adapter in the VMM | Feasible, but puts a second renderer interface into the VMM for as long as older guests ship. Kept as the fallback. |
| Move the guest to the newer AOSP release | Its parameter ids disagree with the guest kernel, so zero copy would be off without a kernel change; transports are forced and modules are missing; it moves the whole guest to a new Android release, with the BSP bring-up work that brings; and it has the same program-binary gap. |
| The newest renderer releases | Extra capability fields, a pipe type treated as fatal, and renumbered handle types. |
The decision record also named what would re-open it: a reachable protocol defect that cannot be fixed in the renderer, Vulkan first light failing on protocol, or the customer moving to a newer AOSP release. A decision with no re-open condition tends to outlive the facts it rested on.
7. Keeping the boundaries honest
A compatibility decision is only as good as the discipline that follows it. Five practices held the boundaries in place for the rest of the work.
- Generated code is regenerated, never hand-edited. The guest’s checked-in generated encoder files had been assembled by porting individual upstream changes, not produced by one generator run. Before adding entries, the generator was pinned to an upstream revision verified to reproduce the affected file byte for byte; its output was then carried into the guest tree, with the few lines where that tree differs documented.
- Shared structs grow the owner’s way. A new feature field is appended after the last field, as upstream grows the struct, not dropped into padding.
- Libraries that share a struct ship together. Appending a field to a guest feature struct changed a second guest library’s copy of it. The two libraries are one ABI unit and are delivered as a pair.
- Mixed-version cells. The change that spanned guest and host was tested as new guest with old host and old guest with new host, as well as both new (Section 18).
- Not every failure is a mismatch. A crash under the application benchmark looked like a protocol problem. It was a handle table that re-issued live indices once very many handles were alive, and the fix was a backport of an upstream change. The same discipline applied later to a file-descriptor leak on the gfxstream path only: the guest’s GLES layer keeps each present fence until its status changes, and the older guest’s fence-status path never reports the change. A one-property switch confirmed the cause, and a backport of the upstream change fixed it.
New interfaces, such as the display copy entry point in Section 12, were written in upstream shape from the start, so that they stay submittable upstream rather than becoming a fork.
8. Let the guest own the memory
If the host cannot map its memory into the guest, the direction has to reverse: the guest allocates, and the host GPU reads and writes the guest’s pages in place.
The mechanism exists in the ecosystem as a guest-handle blob: a virtio-gpu blob resource backed by guest memory, offered by the VMM behind a feature bit. When the guest creates one, the VMM builds a udmabuf over the guest’s pages and passes the file descriptor to the renderer, which imports it into the host Vulkan driver as external memory. The host GPU then renders directly into guest memory.
One platform detail mattered: the Linux udmabuf driver’s default limits on the number of pages in a list and on buffer size are too small for GPU allocations, so they are raised before the VMM starts.
9. One context for memory and commands
The original plan put Vulkan on gfxstream’s pipe transport. A source read before that step was built showed that it cannot work with guest-handle memory.
On the pipe transport, the guest-handle blob is created on the virtio-gpu device’s context, while vkAllocateMemory that uses it is decoded on the pipe’s context. The renderer keys blobs by context and id, so the lookup fails. Creating blobs per thread instead does not work either: there is one host pipe per context, so streams cannot be combined, and a blob on a thread’s descriptor lives as long as the thread, while Vulkan memory belongs to the whole process.
gfxstream’s own design for Vulkan is address-space graphics (ASG): a single command ring per process. Its ring normally lives in host memory, which is the mapping this platform lacks, so the ring moved into guest memory as a guest-handle blob of its own. Two consequences followed:
- the per-process ring always uses the Vulkan capability set, because a process that starts with GLES would otherwise keep empty Vulkan capabilities for later use;
- for this step, the guest encoders and the renderer changed; the VMM and the virtio-gpu device did not.
On the board, the guest-handle allocation and its Vulkan stream then shared one context. Data was correct on both coherent and non-coherent memory types, a Vulkan conformance subset ran without failures, and the guest never created a host-allocated blob or asked to map one.
10. Coherency on a non-coherent GPU
When stage 2 maps guest RAM as normal write-back memory, the guest’s own page tables decide cacheability. The guest maps its blobs cacheable; the GPU is not coherent with the CPU caches. Without maintenance, data was wrong in both directions in every iteration of a coherency test.
The rule chosen is that maintenance stays with the owner of the cache lines, which is the guest:
- Before the GPU reads, the guest cleans the range (
DC CVAC) invkFlushMappedMemoryRanges. After the GPU writes, it cleans and invalidates (DC CIVAC) invkInvalidateMappedMemoryRanges. Both directions then had no mismatches. On this platform they run from guest user space without trapping, so no system call is needed. - At creation, the guest kernel zeroes a blob’s pages through its cacheable linear map. Those dirty zero lines could later be written back over GPU output, so the kernel cleans the pages once before the blob reaches the host.
- Coherent memory types are mapped write-combined in the guest. The request travels as a blob flag that only the guest kernel sees: it accepts the flag for guest-memory blobs only and clears it before sending the command, so the host contract is unchanged.
- The renderer skips host flush and invalidate on memory it never mapped. Forwarding them would break a Vulkan valid-usage rule, and the guest owns those lines anyway.
The alternative, cache maintenance through dma-buf sync calls on the host, gave correct results in a test. It was rejected because it crosses ownership, and its invalidate can discard guest lines that the guest has not yet cleaned.
11. GLES through ANGLE, and driver quirks
With Vulkan working, GLES moved onto it through ANGLE in the guest. A renderer feature for guests that use ANGLE makes colour buffers Vulkan-only, and the renderer then runs no host GL emulation at all. The first attempt without it showed a blank panel: SurfaceFlinger composed into hardware-buffer images whose Vulkan writes never reached the colour buffer’s readers. With the feature on, Android’s UI, SurfaceFlinger and apps render through ANGLE, vulkan.ranchu and gfxstream, and ANGLE provides GLES 3.2.
Where the host driver behaved differently from what the renderer expected, the adjustment went into the renderer, not the driver. Where upstream gfxstream had already fixed the same problem, the fix was a backport with upstream authorship kept, rather than a new patch.
Guest-handle Vulkan zero copy also works on the open-source host driver, so the design does not depend on one vendor’s driver.
12. Display: from three copies to one
Under virgl, a scanout resource has a buffer behind it that the host compositor can show. Under gfxstream it has none, only a read-back. The first working display read each frame back with the CPU, which cost one GPU copy and two CPU copies.
The delivered path gives each scanout a ring of three dma-buf buffers created through udmabuf, imported once by the host compositor, and fills the next slot with one GPU copy, through a new renderer entry point that copies a colour buffer into a dma-buf. The pitch matches the renderer’s fixed stride and was checked against the display controller’s constraints, so no third copy is needed. Both CPU copies are gone.
The one GPU copy is kept by design: the guest’s own Vulkan image, not the colour buffer, decides the layout, and the guest asks for the GPU’s optimal tiling rather than a layout the display controller can scan directly. The copy converts that tiled image into the linear dma-buf the display scans out, in a single pass on the GPU.
13. What a loss figure measures
The contract set a target for virtualisation loss but defined no method for measuring it, so a measurement contract was written to define one. The method rests on these rules:
- State the claim type first. An overhead ratio, an absolute score, an optimisation verdict and a mechanism each need a different statistic and sample unit.
- Same code on both sides. A loss is meaningful only from one benchmark source built for the host and for the guest, with the same scenes and settings. A guest-only application has no denominator.
- Offscreen. Presenting runs are capped by vsync and mix display cost into the ratio.
- Declare the aggregate. The loss from the benchmark’s composite score and the mean of per-scene losses differ noticeably, because the composite weights high-frame-rate scenes more. Report per scene, and state the aggregate next to the figure.
- Score runs carry no instruments. Profilers and probes are used in separate attribution runs, whose own cost is measured.
14. Idle cost first
One cost had nothing to do with rendering, and showed even at idle. Two busy loops in the VMM’s GPU device kept host cores busy at all times, whichever renderer was in use: an event that was never cleared, and a timed wait whose deadline had always already passed. The fixes changed a dozen lines. They also improved the existing virgl stack, which is a reminder that a before-and-after comparison is only fair when both arms carry the same fixes.
15. The per-frame floor
The first contract-shaped measurement showed the pattern clearly. Scenes bound by GPU work lost little, so the render path itself, zero-copy memory included, added little. The loss sat in very light scenes running at thousands of frames per second.
The model is simple: guest frame time is roughly the larger of the host frame time and a fixed per-frame floor. Fitted across scenes and resolutions, the floor explained the light-scene losses. More usefully, it predicts: once a change lowers the floor, the model projects the aggregate loss before the next contract measurement, and that projection is written down before the measurement is taken. Its first projection matched the measurement. A model that predicts is still not a mechanism, so the floor was then decomposed per frame (Section 16).
16. Overheads that were not the work
For the lightest scene, which only clears the screen, the frame time was decomposed into components measured per frame, with a residual row. The ledger closed. It showed the frame bound by the guest’s render thread, and two guest-side costs that were not part of rendering took over half of it:
- A watchdog’s bookkeeping on the render thread, in the guest’s graphics support library. It is switched off in the guest build, a change confined to that one library, after checking that every call site tolerates its absence.
- A doorbell after every ring write. The guest notified the host after each write, whether or not the host was waiting.
The doorbell could not simply be dropped on the guest side. The host renderer goes to sleep without re-checking the ring, so a guest that skips a notification at the wrong moment strands a command, and the guest hangs. The safe change has two halves: the host re-checks the ring before it sleeps, and the guest rings only when the host says it is about to sleep. Because it changes behaviour on both sides, it is negotiated as a new host feature (Section 18).
Each change named, in advance, a counter that had to move if its theory was right. For the doorbell, guest notifications had to fall to about the number of host sleep transitions. They fell by orders of magnitude and equalled the host’s wake-ups, counted at both ends. Before the change the two counts had never matched, because the host’s wake channel was bounded and dropped notifications silently when full. Counting only the producer would have hidden that.
17. Work repeated every frame
The application benchmark’s GPU section lagged the previous stack. The obvious reading was that the new transport was slower. That reading changes too many variables at once: the previous stack reaches the GPU through a native GLES driver, while the new one goes through ANGLE on Vulkan.
The ledger for the slow sub-test showed the guest render thread spending much of each frame on shader modules and pipelines that ANGLE created and destroyed every frame, each through a synchronous round trip to the host, where the driver compiled them again. Tracing showed the cause: the application’s separable program pipeline was re-linked inside ANGLE at draw time, destroying the modules and pipelines that the next draw then recreated. Every module re-created each frame repeated one of a small number of distinct SPIR-V blobs, and no layer between ANGLE and the driver de-duplicated them.
The fix sits in the guest encoder: shader modules are de-duplicated by SPIR-V content, so a repeated create is served from a guest-side cache that shares one host module. De-duplicating below ANGLE removes the repeated work whatever triggers the re-link, which made it the more robust choice than changing ANGLE itself.
18. Changing a protocol safely
The doorbell change alters behaviour on both sides of contract B, so it was treated as a protocol change:
- Negotiated. The guest suppresses notifications only if the host advertises the new feature, under a name of its own, so that it cannot collide with a future upstream feature. The guest records it in a field appended to its feature struct (Section 7).
- Arms. The kept build as control, the candidate, both mixed-version cells, and a rollback to the known set.
- Targeted stress. Lost wake-ups are rare per iteration and almost never appear in benchmark runs, so a purpose-built hang hunt aimed at the transition window ran in every cell: several independent producers and consumers, seeded random idle gaps, transfers large enough to reach every changed path, a timeout on every wait and a wall-clock limit on the run. Its first version used invalid API input and was rejected before use. The final one used valid input only, checked by a validator with a negative control, and the host’s count of data found on the re-check showed that the window was reached.
- Decision rule first. A shift counts only if it exceeds the wider run-to-run range of either arm, with at least three runs per arm. The doorbell change was kept with every candidate run above every control run.
19. How the risks were managed
The design choices above were also risk decisions. Each major risk had a named mitigation before work depended on it:
| Risk | How it was managed |
|---|---|
| No renderer serves both the production guest and the VMM | Contracts A and B compared from source before anything was built; a compatibility adapter designed and held as the fallback; re-open conditions written into the decision (Sections 3–6). |
| The platform cannot share host memory with the guest | Direction reversed to guest-owned memory; runtime sharing recorded as the long-term fix (Sections 1, 8). |
| The biggest unknowns surface last | Feasibility questions such as coherency, decode coverage and the memory budget were run early, alongside the critical path; the whole functional chain was proven before any optimisation, one variable per step. |
| A protocol change hangs the guest in the field | Feature negotiation, mixed-version cells and a targeted hang hunt before the change was kept (Section 18). |
| An optimisation that only appears to help | A predicted counter per theory, decision rules before the data, and score runs without instruments (Sections 13–17). |
Items that did not help close with a verdict too. Pinning the GPU worker threads to the big cores made results worse, so it was not kept and the scheduler’s default placement stays. A file-descriptor leak was localised by a differential arm before any tracing: zero growth per frame on virgl, steady growth on gfxstream, under the same applications, which pointed straight at the gfxstream path (Section 7).
The general lessons are the ones that cost the most to learn. Compatibility across release lines is a question to answer from source, boundary by boundary, before anything is built. When the platform withholds a mechanism, reverse the direction rather than emulate it. And an overhead is found only by a ledger that sums to the frame, a counter predicted in advance, and a rule written before the data.
20. References
- AOSP, gfxstream (guest encoders and host renderer, later releases) and goldfish-opengl (guest encoders, earlier releases).
- crosvm, Rutabaga virtual graphics interface and the rutabaga_gfx gfxstream module; QEMU, virtio-gpu (its rutabaga device).
- OASIS, Virtual I/O Device (VIRTIO) Version 1.3, section 5.7 GPU device.
- Linux kernel, drivers/dma-buf/udmabuf.c and include/uapi/drm/virtgpu_drm.h.
- Khronos, VkMappedMemoryRange (VUID-VkMappedMemoryRange-memory-00684: memory must be currently host mapped).
- Google, ANGLE.
- Arm, Arm Architecture Reference Manual for A-profile (data cache maintenance by virtual address).
- vkmark, the Vulkan benchmark used on both sides.
- Mesa, Virtio-GPU Venus (host-visible memory mapped into the guest).
- crosvm, virtio device constants (virtio-gpu feature bits, including the guest-handle bit).
- AOSP minigbm, external/virtgpu_drm.h (guest-handle parameter and blob flag); dri-devel, drm/uapi: add explicit virtgpu context debug name (upstream parameter 8).
- AOSP, vulkan-cereal, virtio-gpu-gfxstream-renderer.cpp (the renderer's earlier repository and its
pipe_virgl_renderer_*interface). - Skia, GrGLCaps.cpp (program-binary support cleared when no formats are reported).
- dri-devel, udmabuf: configurable list_limit and size_limit_mb.
- Android, Cuttlefish: GPU graphics acceleration (gfxstream and virgl modes).
About this article
Written by Lei Zhou. Text and diagrams © 2026 Lei Zhou, licensed CC BY 4.0.
Upstream behaviour described here reflects the sources linked above as of November 2026; the design, method and results are the author’s own.
Corrections, questions or counter-examples?
If a claim here is wrong or out of date, I would like to hear about it. Comments are hosted in GitHub Discussions and need a GitHub account. Nothing loads from a third party until you click.