Testing KubeVirt controllers in seconds, not clusters

KubeVirt’s new integration test framework runs a real kube-apiserver and etcd locally, with real controllers in-process. Tests exercise VM → VMI → Pod → Scheduled in seconds, without deploying a cluster or booting a VM.

The base framework, builder update and non-voting presubmit are merged. VEP #371 targets alpha in v1.10.0; further coverage and webhook support remain under review.

The problem

The instancetype ControllerRevision upgrade tests were my starting point: pure controller logic, no guest required, yet running them meant paying the cost of the functional suite.

In controller unit tests built around GoMock, Controller A’s writes don’t become API-server state that Controller B can observe. Functional tests exercise those interactions but require a deployed cluster. This tier gives controllers a real API server, catching reconciliation bugs and status-update races with a shorter feedback loop. The VEP has more concrete examples.

The idea: shamelessly repeated

I spent years in OpenStack Nova before KubeVirt, and borrowed this pattern from its functional test framework: real services, fake external dependencies. Nova runs its API, conductor and scheduler against a real database, with a fake hypervisor driver and fixtures for Neutron, Cinder and Glance. envtest makes the equivalent approach possible for KubeVirt.

How it works

• CRDs come from virt-operator’s Go definitions. Real clients and API-server semantics provide OpenAPI validation, status subresource enforcement and resourceVersion conflicts.
• The framework calls production controller constructors directly, catching signature changes at compile time. Its own wiring uses dummy CDI informers, an instancetype stub, and no-op firmware and network synchronisation.
• envtest has no kubelet, scheduler or kube-controller-manager. The pod simulator binds pods to a fake node and marks them Running and Ready, allowing VMIs to reach Scheduled. Explicit resource cleanup compensates for the missing namespace and garbage-collection controllers.

The merged base has no KubeVirt admission webhooks. #18729 adds the real VM admission handlers, enabled by default with an opt-out for schema-only tests.

What a test looks like

Adapted from the merged lifecycle test, with imports omitted. Start registers suite shutdown; CreateNamespace registers per-test resource cleanup.

var f *framework.Framework

var _ = BeforeSuite(func() {
	f = framework.New()
	f.Start(context.Background())
})

var _ = Describe("VM Lifecycle", func() {
	It("creates a Scheduled VMI from a VM", func(ctx SpecContext) {
		ns := f.CreateNamespace(ctx)

		vm := libvmi.NewVirtualMachine(
			libvmi.New(libvmi.WithMemoryRequest("128Mi")),
			libvmi.WithRunStrategy(virtv1.RunStrategyAlways),
		)

		vm, err := f.VirtClient().VirtualMachine(ns).Create(ctx, vm, metav1.CreateOptions{})
		Expect(err).NotTo(HaveOccurred())

		Eventually(ctx, matcher.ThisVMIWith(ns, vm.Name),
			10*time.Second, 100*time.Millisecond,
		).Should(matcher.BeInPhase(virtv1.Scheduled))
	})
})

This single-test lifecycle suite runs in roughly 16–20 seconds locally, including API-server startup and cleanup, as reported in the base PR. Tests reuse the functional suite’s client interface and matchers, so migrations can retain assertions while replacing cluster setup with framework startup and isolated fixtures.

In a separate PoC benchmark, eight instancetype ControllerRevision upgrade tests ran 13.4x faster: 152.6s in the functional sig-compute lane versus 11.4s with shared envtest. The benchmark doesn’t specify whether startup and cleanup are included.

What’s next

The VEP’s graduation criteria describe the remaining work:

• Alpha (v1.10.0) — remaining coverage for instancetypes, generation tracking, webhooks, schema/CEL validation and pod failures (#18727–#18731); non-voting CI.
• Beta — fake libvirt gRPC and virt-handler simulation so VMIs reach Running, workload-update controller support, and further test migrations.
• GA — wider operator, migration and storage coverage, contributor guidance, and required integration tests for controller changes. CI becomes voting after zero flakes across multiple release cycles with representative coverage.

What this doesn’t replace

There’s no hypervisor, networking dataplane or guest OS here. Infrastructure tests stay in the functional suite, and overlapping cluster-based coverage remains necessary to catch simulation drift. Tests needing components the framework currently stubs also need further support before migrating.

Try make test-integration from a KubeVirt checkout. It uses the standard builder container, with envtest binaries included. The lifecycle test provides a starting point for your own controller coverage.