|
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index] Re: [Xen-devel] [Query] Flow of PCI device dedicated to a domU
Could you please use plain text in emails? HTML makes it harder for me to read, as you can see below for yourself. On Thu, 24 Jul 2014, Xenbegn developer wrote: > On Tue, Jul 22, 2014 at 7:41 PM, Stefano Stabellini > <stefano.stabellini@xxxxxxxxxxxxx> wrote: > Please don't use HTML emails and please don't top post. > > On Tue, 22 Jul 2014, Xenbegn developer wrote: > > Ian Can you please help in this. I am trying to understand how things > are done in x86 xen. Based on > that will > > implement in ARM PCI passthrough > > The basic mechanism is that Xen assigns the PCI root complex/host > bridge to Dom0. Dom0 walks the bus and finds out the devices available. > > When the user assigns a PCI device to a VM, the toolstack (xl/libxl) > issues hypercalls to remap interrupts and MMIO regions of that device > into the guest VM. > The device is added to the guest virtual PCI bus that is created by the > xen-pcifront driver at boot. PCI config space reads/writes go through > xen-pcifront that talks to xen-pciback in Dom0. > > > [manish] In ARM can we do things a bit simpler way, like a hypervisor trap > when guest accesses configuration space > rather than the pci-back pci-front communication. Eliminating pci-front for > config space access. Actually you'll find that pcifront/pciback is the easiest possible solution: no emulation is involved, no need to write hypervisor code. Also both drivers already exist and are known to work well. If you look at the x86 emulation code for HVM guests, you'll that is way more complex. I would avoid something like that for ARM if we can. > PCI bus enumeration is done by Dom0 at boot time. Dom0 issues hypercalls > to Xen to tell the hypervisor what devices are available. It would be > conceivable to do the enumeration directly in Xen if Xen had the > appropriate driver for the host bridge. It might be even better. > > The main interaction between Xen and the config space of PCI devices is > about MSI programming. Xen programs MSIs directly because it is Xen that > is going to receive them and then inject them into guests. > > [manish] Same logic as above > > If I were you I would base my work on Julien SMMU series and I would > ignore MSIs for now. I would try to get pcifront and pciback running and > then assign a PCI device to a guest, only using legacy interrupts. > > After that you could try to add support for doing PCI config space reads > and writes in Xen. Then introduce MSI support but, as Ian wrote, you > would need to rebase your work on the GICv3 series. I don't think you > actually need full ITS support to be able to get MSI running though. > You might already have everything you need in terms of patches, but they > are still not applied to xen-unstable yet. > > > > > > On Tue, Jul 22, 2014 at 4:21 PM, Xenbegn developer > <xen.begn.dev@xxxxxxxxx> wrote: > > > > > > > >    On Tue, Jul 22, 2014 at 4:03 PM, Andrew Cooper > <andrew.cooper3@xxxxxxxxxx> wrote: > >       On 22/07/14 11:13, Xenbegn developer wrote: > > > > > > > >    On Tue, Jul 22, 2014 at 3:26 PM, Andrew Cooper > <andrew.cooper3@xxxxxxxxxx> wrote: > >       On 22/07/14 10:50, Xenbegn developer wrote: > > > > > > > >    On Wed, Jul 16, 2014 at 3:20 PM, Andrew Cooper > <andrew.cooper3@xxxxxxxxxx> wrote: > >       On 16/07/14 07:12, Xenbegn developer wrote: > >    Hi, > > I am trying to understand the flow of how a PCI device driver in domU > works > > after a PCI device is is assigned to a domU. > > > > a) If a PCI device is assigned to a domU, this device has to be on a > PCI > > bus. So as per my view xen would have to somehow provide a PCI > Controller > > on which this device is attached. > > => Is my assumption correct ? If yes how it is done, No then also How > > enumeration of this device happens in domU kernel > > > > > > No. > > > > PV guests have no PCI root ports/bridges; they use devices as single > entities > > knowing that Xen/dom0 takes care of the other bits. HVM guests have > their PCI > > devices attached to the virtual southbridge which is all emulated by > Qemu. > > > > > > b) Is the Configuration space of the PCI device directly accessible > to the > > domU (assuming the kernel accesses it using memory map) ? If not then > it is > > trapped by xen > > > > > > All configuration space is trap+emulate in Xen, although almost all > operations > > permitted. > > > > > > c) Who assigns the MSI (addr + value) in the PCI device. If Xen then > how > > Xen does a translation from Physical MSI to guest MSI (where in code) > > > > > > Xen controls all interrupts on the system, which is why it needs to > trap all > > config accesses to notice when a domain is attempting to change the > interrupt > > information. In that case, Xen fixes up its delivery of interrupts > to the guest, > > but leaves the underlying interrupt information intact. > > > > ~Andrew > > > > > > Linux kernel pci code has a function called __write_msi_msg. This > method writes to > > config space for MSI/MSIX > > pci_write_config_dword(dev, pos + PCI_MSI_ADDRESS_LO, > >     Â msg->address_lo); > > > > Now as per (b) and (c) it has to be trapped in xen. But in > x86/traps.c (guest_io_write) > > I dont see any MSI handling. > > Can you please explain (point to the code) where domain is attempting > to change the > > interrupt information. > >  > > > > > > PV guests are expected to use PHYSOP hypercalls for interrupt > management. Xen does not > > provide transparent emulation support for them. > > > > HVM guests do get transparent emulation support, as from the point of > view of an HVM guest, > > they are talking to a real PCI device. > > > > Can you please point to a device driver code which is calling PHYSOP > calls for setting an MSI Addr > > + MSI number > > > > > > No. Partly because I don't know exactly, and mainly because this is > something you should trivially be > > able to search for. > > > > It is not that trivial. Can anoyone from Citrix reply ? > >    ~Andrew > > > > > > > > > > > > > > _______________________________________________ Xen-devel mailing list Xen-devel@xxxxxxxxxxxxx http://lists.xen.org/xen-devel
|
 |
Lists.xenproject.org is hosted with RackSpace, monitoring our |