Xen project Mailing List

What you are describing is quite an old bug. I hope I can recall most things from memory, but you will find lots of information simply by searching in this mailing list for "FLR".

- Most if not all consumer graphic cards are flagged with FLReset- (Both AMD and NVidia). I think there are only a few pro grade cards that allow proper FLR from linux

- Some people reported not having this issue with consumer cards (or lesser versions of this issue, like only the domU crashing and not the dom0), but there were few similarities between them and it was usually attributed to luck

- AMD cards seem to be more prone to crashing the dom0 than NVidia, but the behaviour is common in both cards

- The issue happens both for hotplugging cards into running domUs as well as restarting a domU with an attached card in the config. The usual behaviour is that a card will pass through once, but is then tainted and will not work with furthor pass through attempts until the dom0 is restarted

- issue exist both with qemu and qemu-traditional device model

- As you stated the issue is caused because currently there is no proper kernel level method to reset. All existing methods (FLR, sleep state resets, rebinding to linux graphic driver, etc..) do not work

- This issue only exist in the xl toolstack (but there basically from day 1). Back in the day, xm was able to do a proper reset because it did the pci management themself and not relied on kernel and sysfs features.

This is all I can think of right now off the top of my head and hope this helps you understanding the issue. There were some active threads about the issue like 6-9 month ago with more information. I think now this whole issue died down because vga passthrough was primary driven by enthusiasts (probably wanting a gaming windows VM on a server) and was never the focus of mainline xen development and because of this issue a lot of people switched to kvm/qemu because their vfio can do proper FLR and currently is a more stable experience for vga passthrough. This was what I did as well and while I miss Xen (the CPU and power management things feel much smoother and management is easier), not having your dom0 crash because a windows domU decided on a random windows update to reboot is a nice feature.

2016-02-08 20:09 GMT+01:00 Charlie Mei <cmei@xxxxxxxxxxxxxx>:

Update on this: the original issue was caused by failing to disable
the device manager in Windows 7 before running pci-detach on the Dom0.

The current issue we are dealing with is that the device fails to
reinitialize appropriately upon pci-reattach and being re-enabled in
Device Manager.

I'd ran "lspci -vv <device>" on both of our test cards, and with both
came up with "FLReset-". Is this an indication these cards do now
allow for hotplugging?

On 19 October 2015 at 04:50, George Dunlap <George.Dunlap@xxxxxxxxxxxxx> wrote:
> On Thu, Oct 15, 2015 at 10:29 PM, Charlie Mei <cmei@xxxxxxxxxxxxxx> wrote:
>> Hi,
>>
>> What is the current state for PCI hotplugging in Windows 7 HVM? In
>> particular, I am wondering if anyone else attempted hotplugging
>> graphics cards using PCI passthrough in Xen. I can successfully boot
>> Windows 7 HVM on top of Xen, and have PCI passthrough working for
>> graphics cards. `pci-detach` of a graphics card not acting as a
>> frame-buffer works fine, but re-attaching that card crashes not just
>> the DomU Windows 7, but also Xen Dom0.
>
> Crashing dom0 is certainly a bug. If there's any chance you could get
> a serial log of the dom0 crash, I'm sure we'd appreciate it.
>
> In your description you only talk about re-attaching a PCI device, but
> you don't say what happens if you hot-plug it without having first
> attached it to the guest: that is, if you boot the host, boot the
> guest, then do "xl pci-attach".
>
> It's significant because if it works before the card has ever been
> attached, but not after, then there may be some issue with the FLR
> (function-level reset), which is supposed to set the card back into
> the state it started in at boot
>
> -George

_______________________________________________
Xen-users mailing list
Xen-users@xxxxxxxxxxxxx
http://lists.xen.org/xen-users

Re: [Xen-users] Windows 7 PCI hotplugging