{{ message }}
Repository navigation
AArch64 support #1125
Description
Hi,
Henry(@MrXinWang) and I are developers of ARM. We are working to enable Cloud-hypervisor on AArch64. Now an initial version is ready, although all functionalities are not supported.
As a minimal implementation, it supports a small set of devices: including serial device, RTC and Virtio devices with MMIO transport option. We are about to submit the code as a starting point, all other devices and functionalities will be added in steps.
Before coding, we made an analysis of the gap to fully support AArch64, which could be taken as a TODO list: (updated)
- Build on AArch64: AArch64 initial build #1168
- Fix build warnings and errors
- Add cross-build workflows in PR Check
- FDT and architecture specific devices: AArch64: Adding AArch64 arch-related files #1223
- Device Manager:
- Hide Port-IO
- Add legacy devices to MMIO bus.
- Allocators:
- GsiAllocator: Update IRQ and GSI allocation algorithm.
- Ioapic: Refactor IOAPIC #1232
- Now Ioapic is directly referenced in device manager, but it is architecutre specific. A way to improve could be refactoring: introduce a architecture agnostic Interrupt Controller trait. Different architecture provides its own implementation: for X86 it's Ioapic; for Aarch64 it would be GIC.
- Booting:
- A fundamental requirement for Aarch64 is: VCPU's have to be created before GIC. But in current Cloud-Hypervisor the VCPU's are created in late stage of booting. We need to refactor the steps in booting a guest to work with both architecture at first: vmm: Change booting process to cover AArch64 requirements #1238
- And populate Arm64 specific booting source code: Enable AArch64 #1126
- Virtio:
- MMIO transport: Add device info to FDT
- PCI transport: Add PCI bus info to FDT
- Migration:
- Enable dependencies for AArch64: kvm-bindings, kvm-ioctls.
- Vhost-user-*:
- To be tested and determined.
- VFIO:
- To be tested and determined.
- CI:
- Update Dockerfile.
- Get guest disk images for integration tests.
- Add and setup CI machines.
- Enable Aarch64 CI.
Activity
[-]AAarch64 Support[/-][+]AArch64 Support[/+][-]AArch64 Support[/-][+]AArch64 support[/+]michael2012z commentedon Jun 12, 2020
Having PR #1126 merged, now Cloud-hypervisor is enabled on AArch64 with a minimal functionality set. Follow this guide if you want to taste.
Next we will focus on:
rbradford commentedon Jul 21, 2021
Hey @michael2012z @MrXinWang Just checking in to see if the todo list above is still correct. I know you folks have been making lots of excellent progress on this recently.
MrXinWang commentedon Jul 21, 2021
Hi @rbradford , recently me and @michael2012z had a discussion about the gap between arm64 and x86_64, it seems that arm64 are only missing these parts:
Compared to other three, CPU Topology is rather small (but still on arm64 we need to support both ACPI and device tree). Also it may be good to support NUMA in device tree.
MrXinWang commentedon Jan 18, 2022
Hi @rbradford @michael2012z
With #3513, we have finished all planned work in this issue. One thing we need to note that is currently the vfio framework is only verified by the
test_vfio_usertest case in our CI. Although there is an SRIOV NIC in our CI machine that can be used for verifying the vfio, we are not sure how can we deliver it with our test framework - as users may run the test framework locally and found the SRIOV NIC missing.For now the only feature gap between x86_64 and AArch64 is the vCPU hotplug.
rbradford commentedon Jan 19, 2022
@MrXinWang Cool. Is CPU hotplug possible with aarch64 in general? If not then this issue can be closed.
MrXinWang commentedon Jan 19, 2022
It is (in a way), there are some custom implementations in kata-containers (kata-containers/kata-containers#3292) but never merged to mainline qemu, and also vCPU hotplug needs some kernel patches to support (I think also not merged...).
Maybe we can first investigate it.
rbradford commentedon May 23, 2022
I think this issue can be closed now as aarch64 support is complete!
grass-lu commentedon Nov 4, 2023
@michael2012z @rbradford hi， for aach64 ,hotplug pci device don't work , cloud you help me , thanks. the discussion about it： #5910