ARM CCA · RME · KVM · Confidential Computing

ARM CCA + KVM:
How Multiple Guest OSes Work with RME

Lei ZhouPrincipal Systems Architect & Technical Leader

The answer requires understanding one architectural decision ARM made that is genuinely elegant: KVM and RMM are two different hypervisors, running in two different security worlds, with a deliberate and principled split of authority between them.

ROOT WORLD (EL3) TF-A Monitor: GPT management · world switching · RMI dispatch GPT (Granule Protection Table) one bit per 4KB page: NS | Realm | Secure | Root NORMAL WORLD KVM / Linux (Normal EL2) Resource policy: allocates memory, CPU, devices Creates + schedules both regular VMs and Realm VMs QEMU Guest A Normal EL1 (guest kernel) Normal EL0 (apps) KVM manages directly no isolation from KVM QEMU Guest B Normal EL1 (guest kernel) Normal EL0 (apps) standard VM isolation KVM = hypervisor virtio I/O shared buffer (GPI_NONSECURE) Realm VM explicitly shares pages via RSI for device I/O KVM cannot pull pages out — Realm controls the sharing Normal world physical memory (NS PAS) GPT: GPI_NS — accessible from Normal world KVM manages page allocation to Regular VMs KVM delegates pages to Realm world via RMI_GRANULE_DELEGATE Once delegated → GPT updated → KVM cannot access GPC fault on any Normal world access attempt REALM WORLD RMM — Realm Management Monitor (R-EL2) Protection mechanism: manages RTTs (Realm Translation Tables) Measured + attested. RMI handler. RSI provider. TCB. Realm VM 1 R-EL1: guest kernel R-EL0: guest apps RMM manages stage-2 KVM cannot read state RIM+REM: attested Realm VM 2 R-EL1: guest kernel R-EL0: guest apps Isolated from Realm VM 1 and from KVM Realm world physical memory (Realm PAS) GPT: GPI_REALM — accessible only from Realm world Pages delegated here via RMI_GRANULE_DELEGATE RMM manages RTTs: IPA → PA mapping per Realm VM Any Normal world access → GPC fault (hardware enforced) Any Secure world access → GPC fault GPC check fires on every MMU + SMMU translation All initiators blocked equally (CPU and DMA) RMI via SMC EL3 dispatches RSI schedules VCPUs Secure world (TrustZone): unchanged

The "Aha" Moments

1. Two Hypervisors, Two Jobs — A Principled Split of Authority

ARM's architecture specification explicitly documents the design principle (verified from official ARM documentation):

ARM CCA Design Principle — Verified from ARM Den0125 Policy: decisions about which resources to allocate to a Realm — CPU cycles, memory, devices. This is the responsibility of KVM in Normal EL2.

Mechanism: enacting policy decisions that change architectural state, when those changes cannot be trusted to the hypervisor. This is the responsibility of RMM in Realm EL2.

KVM remains the resource scheduler — it decides which physical pages a Realm gets, which pCPUs run which vCPUs, which devices are assigned. KVM retains full control plane authority. What KVM loses is data plane authority — once pages are delegated to the Realm, KVM cannot observe or modify what is in them.

Stated simply: KVM is the landlord who allocates rooms. RMM is the locksmith who changes the locks once a tenant moves in. The landlord cannot enter after the lock is changed — not by policy, but because the hardware key no longer works for them.

2. Memory Delegation Is One-Way — The Same Lock Pattern, Applied Per-Page

When KVM decides to give physical pages to a Realm VM, it calls RMI_GRANULE_DELEGATE via SMC. The CPU traps to EL3 (Root Monitor). The Root Monitor updates the GPT for those pages: GPI_NS → GPI_REALM. The Root Monitor then switches to RMM at Realm EL2 to complete the operation.

KVM (Normal EL2)
  │  SMC: RMI_GRANULE_DELEGATE(page_PA)
  ▼
Root Monitor (EL3)
  │  Updates GPT: page_PA → GPI_REALM
  │  Switches to Realm EL2
  ▼
RMM (Realm EL2)
  │  Receives delegation, assigns to Realm VM
  │  Updates RTTs (Realm Translation Tables)
  ▼
  Page is now Realm PAS
  Any Normal world access → GPC fault (hardware, immediate)
  KVM cannot read, write, or remap this page
The Lock Pattern — Recognised from the framework This is the same architectural pattern as SSC (Cortex-M55 SoC), HDP (STM32), and SMMU_s_CONF lock — a trusted entity at the highest privilege level updates a protection table, after which the untrusted entity (here: KVM) cannot access the protected resource. The difference: SSC/HDP lock is boot-time and per-region. GPT delegation is runtime and per-page. The principle is identical.

One important nuance verified from the Linux kernel patch series: KVM can destroy a Realm VM at any time, which forces undelgation of all its pages back to NS PAS. KVM retains lifecycle authority (create and destroy) — it loses memory-access authority during the lifetime of a running Realm.

3. The Realm VM Controls Its Own I/O Boundary

Because KVM cannot access Realm memory, standard virtio I/O (which relies on the hypervisor reading and writing guest buffers) cannot work directly. The solution inverts the traditional authority model in an elegant way:

The Realm VM itself decides which pages to make available for I/O. Using the RSI (Realm Services Interface) — a call from Realm EL1/EL0 up to RMM at Realm EL2 — the guest can transition specific pages from Realm PAS to NS PAS (GPI_NS), making them accessible to the hypervisor for device I/O.

Realm VM (R-EL1)
  │  RSI_IPA_STATE_SET(buffer_page, NS)
  ▼
RMM (R-EL2)
  │  SMC to Root Monitor
  ▼
Root Monitor (EL3)
  │  Updates GPT: buffer_page → GPI_NONSECURE
  ▼
  KVM can now DMA device ↔ buffer_page
  Realm VM retains control — it chose which page to share
  KVM cannot pull pages out; Realm pushes them out
Authority inversion In traditional virtualisation, the hypervisor decides what memory the guest can access. In CCA, the Realm VM decides what memory the hypervisor can access. The guest controls its own I/O boundary, not the hypervisor.

4. GPC: The Same Target-Side Pattern at 4KB Granularity

The Granule Protection Check (GPC) fires at the output of every MMU and SMMU translation — both stage 1 and stage 2, for both CPU and DMA. When any translation produces a physical address, the GPC looks up that PA in the GPT. If the accessing security world does not match the GPT entry, a Granule Protection Fault occurs.

This maps directly into the framework:

MechanismIdentityResourceSideGranularity
TZASC / TZC-400HNONSEC (binary S/NS)DRAM rangeTargetConfigurable ranges
MPC (STM32)HNONSEC (binary S/NS)SRAM blockTarget256–512 B per block
GPT/GPC (ARM CCA)Security world (NS/Realm/Secure/Root)Physical granuleTarget4 KB per granule
AMD SEV-SNP RMPASID (per-VM)Physical pageTarget4 KB per page
The "Aha" — Confidential Computing = Fine-Grained MPC GPT/GPC is not a new security pattern. It is the same target-side access control pattern as TZASC and MPC — applied at 4KB per-page granularity with four-world identity instead of binary Secure/NonSecure. Confidential computing extended the granularity, not the principle.

5. What KVM Can and Cannot Do — The Precise Boundary

OperationKVM can?Mechanism that enforces
Create a Realm VM✅ YesKVM calls RMI_REALM_CREATE via SMC
Schedule Realm VM vCPU✅ YesKVM calls RMI_REC_ENTER
Assign physical pages to Realm✅ Yes (delegate)RMI_GRANULE_DELEGATE → GPT update by Root Monitor
Provide virtio I/O devices✅ Yes (via shared NS pages)Realm VM must explicitly RSI-share pages first
Destroy Realm VM✅ Yes (any time)KVM calls RMI_REALM_DESTROY → forces undelegation
Read Realm VM memory❌ NoGPT = GPI_REALM → GPC fault on any NS access
Modify Realm VM page tables❌ NoRTTs managed exclusively by RMM at Realm EL2
Observe Realm VM register state❌ NoCPU register state protected during world switch
Forge Realm attestation token❌ NoSigned by Root Monitor with platform private key
Remap a Realm page to another VM❌ NoGPT = GPI_REALM → NS access faults; only Root Monitor updates GPT

The Attestation Flow

The Realm VM's attestation is managed entirely within the trusted components — RMM and Root Monitor. KVM and the host OS are not in the attestation path at all.

Realm VM (R-EL1) requests attestation:
  RSI_ATTESTATION_TOKEN_INIT (nonce from relying party)
  RSI_ATTESTATION_TOKEN_CONTINUE
        │
        ▼
RMM (R-EL2) collects measurements:
  RIM  — Realm Initial Measurement (initial image digest)
  REM  — Realm Extensible Measurement (runtime measurements)
  RMM measurement itself (measured at boot by Root Monitor)
        │
        ▼
Root Monitor (EL3) signs the token:
  Platform private key (device certificate chain)
  Produces: CCA token (CBOR, per IETF RATS RFC 9334)
        │
        ▼
Token returned to Realm VM → forwarded to relying party

TCB components visible in attestation:
  • CPU hardware (RME extension)
  • Root Monitor (TF-A, measured at boot)
  • RMM (measured, in attestation token)
  • Realm VM initial image (RIM)

NOT in TCB:
  • KVM  ← removed from trust path entirely
  • Host Linux
  • Other Realm VMs
  • Other guest VMs in Normal world

Comparison: ARM CCA vs AMD SEV-SNP vs Intel SGX

PropertyARM CCA (Realm)AMD SEV-SNPIntel SGX
Isolation granularityVM-level (Realm VM)VM-levelProcess-level (enclave)
Memory protection mechanismGPT/GPC at 4KBRMP at 4KBEPCM at 4KB
Mechanism sideTarget (GPC at translation output)Target (RMP at walk end)Target (EPCM on EPC access)
Hypervisor in TCB?No (KVM excluded)No (hypervisor excluded)N/A (process-level)
Trusted componentRMM (Realm EL2) + Root MonitorAMD Secure ProcessorCPU enclave mode
Memory encryptionImplementation-defined (not mandated)Mandatory (per-VM AES key)Mandatory (MEE)
Intra-VM privilege levelsR-EL0/EL1/EL2VMPL 0–3N/A
Device DMA to private memoryVia SMMU GPC + TDISP (in development)Via AMD IOMMU + RMPNot supported (EPC is CPU-internal)
Entry point enforcementWorld switch (hardware)VMGEXIT (hardware)ECALL via EENTER (SG-equivalent)
The SGX ↔ NSC/SG parallel — from your 2022 article Your observation about SGX's ECALL gateway being structurally equivalent to ARM Cortex-M33's NSC/SG mechanism is precise. Both define hardware-enforced controlled entry points into a protected domain. Both cause a fault on arbitrary branches. The difference: SGX enforces at the CPU instruction level (EENTER must target a valid TCS); ARM NSC enforces at the branch target (must land on SG instruction). Same pattern, different implementation depth.

A Note on Memory Encryption

ARM CCA's hardware architecture specification states that isolation is enforced "by faulting exceptions and by physical memory encryption." However — this is an important nuance — memory encryption is implementation-defined, not architecturally mandated by the CCA specification. A compliant CCA implementation can exist without inline DRAM encryption, relying solely on GPT/GPC for isolation.

This contrasts with AMD SEV-SNP, where per-VM AES-128 encryption is a core architectural feature, not optional. For deployments where physical DRAM probing is in the threat model, the presence or absence of inline memory encryption is an important platform selection criterion, not something guaranteed by the CCA logo.

References

  1. ARM Den0125 — "Arm CCA Software Architecture" (April 2025) — verified from developer.arm.com
  2. ARM Den0125 — "Arm CCA Hardware Architecture" — GPT, GPC, Realm EL2 structure
  3. ARM developer.arm.com — "Arm Confidential Compute Architecture" overview
  4. Linux kernel Documentation/arch/arm64/arm-cca.html — KVM + RMM integration
  5. Linux kernel mailing list — "[PATCH v3 00/43] arm64: Support for Arm CCA in KVM" (June 2024)
  6. ACAI: "Protecting Accelerator Execution with Arm CCA" (arXiv:2305.15986) — RMM + GPT verified
  7. virtCCA paper (arXiv:2306.11011) — RMI call sequence, SMC dispatch through EL3 verified
  8. "Sharing is caring" paper (arXiv:2603.03403) — GPT world tagging, RTT management
  9. Intel TDX Demystified (arXiv:2303.15540) — CCA vs TDX comparison

Licensed under CC BY 4.0.