← RESEARCH
LAB / 2025.11 CLOUD RESEARCH

Cloud-Native KVM Platform

Research and design around a VM cloud built on KVM/QEMU and the CNCF ecosystem — Kubernetes, KubeVirt, Cilium/OVN, Rook-Ceph, GitOps and observability.

KVM/QEMUKubernetesKubeVirtCiliumOVNRook-Ceph

Why

Public IaaS from AWS, Azure, and GCP delivers robust infrastructure, but it carries high operational cost, vendor lock-in, constrained customization, and data-sovereignty limits. Organizations that need a private cloud reach for OpenStack and inherit its own tax: a monolithic Python-based architecture, steep operational overhead, and difficult upgrade paths.

Meanwhile the CNCF ecosystem has matured. Kubernetes is the industry-standard control plane; KubeVirt runs KVM/QEMU virtual machines as native Kubernetes resources; Cilium brings eBPF networking; Rook-Ceph delivers distributed storage through the operator pattern. These are battle-tested, graduated projects with declarative APIs and GitOps-friendly lifecycles.

This research exists to answer one question: how far can Kubernetes-native building blocks be pushed as the control plane for virtual machines? The goal is to investigate a production-grade VM cloud that keeps the robustness of hardware-assisted virtualization while gaining the agility and operational simplicity of cloud-native tooling.

Scope

The research investigates the design and prototype of a KVM-based cloud with cloud-provider feature parity, bounded to a single datacenter for the first phase.

  • Compute — VM lifecycle (create, start, stop, delete, live-migrate) via KubeVirt and CDI image management.
  • VPC networking — tenant isolation, subnets, virtual routers, NAT gateways, security groups, and floating IPs.
  • Storage — distributed block/object storage with snapshots and cloning on Rook-Ceph.
  • Multi-tenancy & IAM — namespace-level isolation, resource quotas, and policy-based access control.
  • Operations — GitOps delivery, monitoring, logging, and backup.

Out of scope for phase one: bare-metal provisioning, CaaS/FaaS/DBaaS, managed Kubernetes, edge, and multi-region topology.

Problems & Solutions

Running VMs under a container control plane

KVM/QEMU gives near-native performance but is not Kubernetes-native. KubeVirt closes the gap: a VirtualMachine becomes a custom resource, virt-controller manages lifecycle, and virt-launcher wraps each QEMU process in a pod. The Containerized Data Importer handles image import and format conversion (qcow2, raw, vmdk), so disks provision through standard PVCs.

AWS-style VPC isolation without a monolithic SDN

Tenants need overlapping CIDRs, private subnets, and security groups. Cilium enforces identity-based L3–L7 policy over eBPF, Multus attaches tenant, storage, and provider networks per VM, and MetalLB publishes floating IPs via BGP. Security groups compile down to CiliumNetworkPolicy; VPC, Subnet, and Router custom resources model the topology declaratively.

Reliable storage with snapshots and migration

VM disks demand high IOPS and live-migration support. Rook-Ceph provides RBD block volumes at 3x replication through the Ceph CSI driver, with native snapshots and clones, self-healing, and horizontal OSD scaling.

Operational simplicity

ArgoCD/Flux reconciles the platform and tenant state from Git. Prometheus, Grafana, Loki, and Jaeger supply metrics, logs, and traces; Keycloak and OPA handle identity and admission policy.

Architecture

G cluster_platform Platform services ui User interfaces Web console · CLI · REST/gRPC gateway API gateway & auth Envoy · Keycloak · OPA ui->gateway controlplane Kubernetes control plane + GitOps (ArgoCD) gateway->controlplane compute Compute KubeVirt · CDI · VM operator controlplane->compute network Network Cilium · Multus · MetalLB · VPC op controlplane->network storage Storage Rook-Ceph · CSI · snapshots controlplane->storage workers Worker nodes kubelet · KVM/QEMU · Cilium · Ceph OSD compute->workers network->workers storage->workers vms VMs workers->vms observability Observability Prometheus · Loki · Jaeger workers->observability

A VM request authenticates at the gateway, creates a VirtualMachine CR, and KubeVirt materializes a VirtualMachineInstance. Storage attaches a Ceph-backed PVC, networking assigns the VPC interface and IP, the scheduler places the workload, and virt-handler launches the QEMU process on the selected node.