Re: [Xen-devel] [PATCH] xen: Avoid calling device suspend/resume callbacks

[Jan Beulich <JBeulich@xxxxxxxx>]

On 29.07.2019 19:06, Andrew Cooper wrote:
> On 29/07/2019 16:57, Jan Beulich wrote:
>> On 29.07.2019 17:41, Ross Lagerwall wrote:
>>> When suspending/resuming or migrating under Xen, there isn't much need
>>> for suspending and resuming all the attached devices since the Xen/QEMU
>>> should correctly maintain the hardware state. Drop these calls and
>>> replace with more specific calls to ensure Xen frontend devices are
>>> properly reconnected.
>> Is this true for the general pass-through case as well? While migration
>> may not be (fully) compatible with pass-through, iirc save/restore is.
> 
> What gives you this impression?
> 
> Migration and save/restore are *literally* the same thing, except that
> in one case you're piping the data to/from disk, and in the other you're
> piping it to the destination and restoring it immediately.
> 
> If you look at the toolstack code, it is all in terms of reading/writing
> an fd (including libxl's API) which is either a network socket or a
> file, as chosen by xl.

Sure. The main difference is where the restore happens (by default):
For live migration I expect this to be on a different host, whereas
for a non-live restore I'd expect this to be the same host. And it
is only the "same host" case where one can assume the same physical
piece of hardware to be available again for passing through to this
guest. In the "different host" case restore _may_ be possible, using
identical hardware. (And yes, in the "same host" case restore may
also be impossible, if the hardware meanwhile has been assigned to
another guest. But as said, I'm talking about the default case here.)

>> Would qemu restore state of physical PCI devices?
> 
> What state would Qemu be in a position to know about, which isn't
> already present in Qemu's datablob?

That's a valid (rhetorical) question, but not helping to answer mine.

> What we do with graphics cards is to merge Xens logdirty bitmap, with a
> dirty list provided by the card itself.  This needs a device-specific
> knowledge.  In addition, there is an opaque blob of data produced by the
> source card, which is handed to the destination card.  That also lives
> in the stream.
> 
> Intel's Scalable IOV spec is attempting to rationalise this by having a
> standard ways of getting logdirty and "internal state" information out
> of a device, but for the moment, it requires custom device-driver
> specific code to do anything migration related with real hardware.

Which isn't very nice, since it doesn't scale well as a model.

> As for why its safe to do like this, the best argument is that this is
> how all other vendors do migration, including KVM.  Xen is the
> odd-one-out using the full S3 path.

So how do "all other vendors" deal with device specific state? So
far I was under the impression that to deal with this is precisely
why we use the S3 logic in the kernel.

Jan
_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxxx
https://lists.xenproject.org/mailman/listinfo/xen-devel