Scribd
Upload
86 views15 pages

Msi in Guest Arm64

Msi in arm64

Uploaded by

Mj Embd
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
86 views15 pages

Msi in Guest Arm64

Msi in arm64

Uploaded by

Mj Embd
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 15

MSIs in guest OSs on arm64 and

direct injection

Will Deacon <[email protected]>


Marc Zyngier <[email protected]>
Linux Plumbers Conference

Santa Fe
November 4, 2016
© ARM 2016
The problem

Devices can generate interrupts by writing a payload to a memory-mapped doorbell register


If a device is passed through to a guest OS, its DMA accesses are translated by the SMMU
The SMMU translates IPAs allocated by userspace (VFIO) into physical addresses in memory

2 © ARM 2016
The problem

Devices can generate interrupts by writing a payload to a memory-mapped doorbell register


If a device is passed through to a guest OS, its DMA accesses are translated by the SMMU
The SMMU translates IPAs allocated by userspace (VFIO) into physical addresses in memory
Some IPA ranges may be reserved or untranslated by the SMMU (what about x86?)

2 © ARM 2016
The problem

Devices can generate interrupts by writing a payload to a memory-mapped doorbell register


If a device is passed through to a guest OS, its DMA accesses are translated by the SMMU
The SMMU translates IPAs allocated by userspace (VFIO) into physical addresses in memory
Some IPA ranges may be reserved or untranslated by the SMMU (what about x86?)
MSIs may or may not be translated by the SMMU page tables

2 © ARM 2016
The problem

Devices can generate interrupts by writing a payload to a memory-mapped doorbell register


If a device is passed through to a guest OS, its DMA accesses are translated by the SMMU
The SMMU translates IPAs allocated by userspace (VFIO) into physical addresses in memory
Some IPA ranges may be reserved or untranslated by the SMMU (what about x86?)
MSIs may or may not be translated by the SMMU page tables
Userspace needs to be aware of these limitations

2 © ARM 2016
Basic system overview

3 © ARM 2016
The x86 solution

x86 does not translate MSIs in the same way as DMA writes, and instead handles MSIs specially at
the IOMMU. The doorbell is a fixed 1MB region at 0xfee00000, so consequently:
The MSI region doesn’t require translation entries in the IOMMU page tables
Userspace can actively avoid the MSI region
Userspace can utilise a single memory map for all guest OSs

4 © ARM 2016
arm64 constraints

arm64 typically translates MSIs in the same way as DMA writes. The MSI region is not at an
architecturally fixed address, nor is it of a standard size. Some reserved addresses may abort before
reaching the SMMU. Consequently:
The MSI region must be mapped in the SMMU page tables
Userspace needs to discover the MSI region
Userspace must adapt the memory map to avoid reserved regions
Reserved regions and MSI regions can be handled in the same way!
The regions are specific to the hardware at the point of adding the device

5 © ARM 2016
VFIO ABI implications

v14 of patches from Eric Auger:


http://lists.infradead.org/pipermail/linux-arm-kernel/2016-October/
461086.html
VFIO_IOMMU_INFO_CAPS Retrieve IOMMU information
VFIO_IOMMU_TYPE1_INFO_CAP_MSI_RESV Obtain consolidated MSI region info
VFIO_DMA_MAP_FLAG_RESERVED_MSI_IOVA Map an IPA region to the MSI region
No interface for reserved regions.
No support for device hotplug following migration.
Focusses on remappable doorbells.

6 © ARM 2016
Open questions:

How should the reserved regions be discovered?


How can devices be hotplugged on after migration?
Does the doorbell need to placed by userspace?
…

7 © ARM 2016
Direct injection

8 © ARM 2016
The ITS

EP (DID) ITS

DID
0 (addr,data0) IRQ−a

data
1 (addr,data1) IRQ−b

2 (addr,data2) IRQ−c

MSI−X table Translation table


map(DID,data,irq)

9 © ARM 2016
Maintaining mappings between ITS and vITS
EP (DID) ITS

Ho
DID

st
0 (addr,data0) IRQ−a

data
1 (addr,data1) IRQ−b

2 (addr,data2) IRQ−c

MSI−X table Translation table


map(DID,data,irq)

EP (vDID) vITS

Gu
vIRQ−x

es
(vaddr,vdata0)

t
(vaddr,vdata1) vIRQ−y

(vaddr,vdata2) vIRQ−z
10 © ARM 2016
map(vDID,vdata,virq)
Mappings

map(vDID,vdata,virq) -> map(DID,data,virq)


vdata -> data
Piggy-back on VFIO_DEVICE_SET_IRQS?
vDID -> DID
Really problematic, as only platform emulation knows that mapping
New VFIO API providing the mapping?
Or KVM-specific attribute for the vITS?
KVM already has a devid field in kvm_irq_routing_msi to deal with injection of virtual MSIs

11 © ARM 2016
The trademarks featured in this presentation are registered and/or unregistered trademarks of ARM limited
(or its subsidiaries) in the EU and/or elsewhere. All rights reserved. All other marks featured may be
trademarks of their respective owners.
Copyright © 2016 ARM Limited

© ARM 2016

You might also like