RE: [Xen-devel] [PATCH] Improve the current FLR logic

["Cui, Dexuan" <dexuan.cui@xxxxxxxxx>]

Yes, when a domain is destroyed, the place where I do FLR is late. Thanks for 
pointing out this.
Looks the situation here is a little tricky. I'm trying to find a better place.

When a device is not assigned to other domain, since Dom0 might dynamically 
rebind the device to another driver rather than pciback, "completely deassign 
the devices from the IOMMU" may not be a good idea?

For the old xend running on a new hypervisor, FLR is actually skipped.
I'm not sure if this is unacceptable. I'd like to hear more comments.

Thanks,
-- Dexuan


-----Original Message-----
From: Espen Skoglund [mailto:espen.skoglund@xxxxxxxxxxxxx] 
Sent: 2008年7月15日 1:52
To: Cui, Dexuan
Cc: Keir Fraser; xen-devel@xxxxxxxxxxxxxxxxxxx
Subject: Re: [Xen-devel] [PATCH] Improve the current FLR logic

Maybe I've got this wrong, but it looks like you're doing the FLR
after the device has been deassigned (i.e., given back to dom0).
Shouldn't you do the FLR before you actually deassign the device
instead?  If the device is currently set up for doing DMA transactions
to host memory, the pending transactions will end up in dom0's memory
space.  This is fine if GMFN == MFN since the guest memory has still
not been released, but for GMFN != MFN you'll end up corrupting
arbitrary dom0 memory.

The right procedure for deassigning devices should be:

  - Revoke device resources.
  - FLR.
  - Reassign device to dom0.

Likewise, for assigning devices we should do:

  - FLR.
  - Assign device to guest.
  - Grant device resources to guest.

Another option is to completely deassign the devices from the IOMMU
rather than reassigning them to dom0.  Devices bound to pciback need
not be assigned to dom0's IOMMU tables anyway.

Further, your patch probably breaks things when running old dom0/xend
on a new hypervisor since you'll end up not doing any FLR at all.  I
recently experienced the same thing with some of my own PCI cleanup
patches.

        eSk


[Dexuan Cui]
> Hi, all,
> The attached patches try to improve the current FLR logic.
> The idea is: removing the FLR logic from hypervisor and adding the
> improved logic in Control Panel.

> The current FLR logic in hypervisor has some issues: 1) Dstate
> transition is not guaranteed to properly clear the device state; 2) the
> current code for PCIe FLR is actually buggy: PCI_EXP_DEVSTA_TRPND
> doesn't mean the completion of FLR; according to the PCIe spec, after
> issuing FLR, we should wait at least 100ms.

> To make it easier to improve the FLR logic, and to make the hypervisor
> thin, I think we might as well move the logic to Control Panel for the
> time being. In the long run, the essential logic may be implemented in
> the pciback driver of Dom0 instead.

> [...]

> I made some tests on my hosts. Looks the patches work well.

> I'd like to ask for your comments, and test feedbacks. Thank you
> very much!

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-devel