Feature Request

When a VM is slow, we want to know where its vCPUs spent their time: running guest code, exiting to KVM, halted, or halt-polling. KVM counts this per vCPU and, since Linux 5.14, exposes it through binary stats (KVM_GET_STATS_FD), including histograms such as halt_wait_hist. Firecracker's metrics only cover the exits that reach Firecracker itself (vcpu.exit_mmio_*, vcpu.exit_io_*).

Describe the desired solution

An opt-in metrics config field, next to emit_id and properties:

{"metrics_path": "metrics.fifo", "emit_kvm_stats": true}

Firecracker opens the stats descriptors for the VM and each vCPU when the vCPUs are created, re-reads them before each metrics write, and emits them as kvm_stats:

{"kvm_stats": {
  "vm": {"instant": {"pages_4k": 3072, "pages_2m": 60}},
  "vcpus": [{"index": 0,
             "counters": {"exits": 18211, "halt_exits": 412, "io_exits": 9024},
             "histograms": {"halt_wait_hist": [0, 0, 0, 0, 0, 0, 0, 0, 0, 2, 18, 40]}}]}}

• Off by default, so existing output is unchanged.
• Before Linux 5.14: logs a warning once and leaves the field out.
• The descriptors are opened before the VMM thread's seccomp filter is installed; reading them needs pread64 in that filter.
• Firecracker already owns the VM and vCPU fds, so this needs no extra privilege.

We have a working implementation and are happy to open a PR if this approach sounds good.

Describe possible alternatives

Today, the only routes are from outside the process: debugfs (needs root, sums over vCPUs, no full histograms), or opening the descriptors through ptrace-attach access to the Firecracker process.

Alternatively, we can have a new request/response endpoint instead of the metrics, like QEMU's query-stats or Firecracker's GET /balloon/statistics. That keeps the metrics schema fixed, but callers would have to query this separately.

Additional context

This follows QEMU, which exposes the same stats through QMP since 7.1 (query-stats, targets vm and vcpu, provider kvm; qapi/stats.json): the full set KVM reports, as running totals, with names defined by the kernel rather than a fixed list. libvirt exposes them as virsh domstats --vm and per vCPU (vcpu.<n>.<name>) since 8.9.0.

Checks

• Have you searched the Firecracker Issues database for similar requests?
• Have you read all the existing relevant Firecracker documentation?
• Have you read and understood Firecracker's core tenets?