Re: [Xen-devel] Multiple platform PCI device ID registries?

[Paul Durrant <Paul.Durrant@xxxxxxxxxx>]

> -----Original Message-----
> From: Ian Campbell
> Sent: 13 November 2013 12:39
> To: Paul Durrant
> Cc: xen-devel; Ian Jackson; Stefano Stabellini
> Subject: Re: Multiple platform PCI device ID registries?
> 
> On Wed, 2013-11-13 at 12:30 +0000, Paul Durrant wrote:
> > > -----Original Message-----
> > > From: Ian Campbell
> > > Sent: 13 November 2013 12:09
> > > To: Paul Durrant
> > > Cc: xen-devel; Ian Jackson; Stefano Stabellini
> > > Subject: Re: Multiple platform PCI device ID registries?
> > >
> > > On Wed, 2013-11-13 at 11:58 +0000, Paul Durrant wrote:
> > >
> > > > > I'm concerned because that comment refers to XenServer 6.1 but it
> now
> > > > > appears to be being reused as the default device ID for the "Xen
> > > > > pvdevice".
> > > > >
> > > > > Maybe it is safe to reuse this in this way, but the docs should be
> > > > > updated I think.
> > > > >
> > > >
> > > > I believe it is safe.
> > >
> > > How would you describe it in that document? "Xen PV device (extended
> > > platform device). Previously used in XenServer 6.1" ?
> > >
> > > Is there some spec we can link to regarding how this device should be
> > > used?
> 
> ???

Sorry, missed that.

No, it wasn't previously used in 6.1 because it's not really the same (as the 
pv device doesn't implement the fixed IO ports). I also don't want to confuse 
it with the platform device, for that reason. It is a new distinct and I agree 
that the fact it defaults to device ID is confusing although it's safe - I will 
therefore submit a patch to QEMU to modify it as I suggested before, so that 
the id *must* be specified by the toolstack.

> 
> > > > > It also seems to be causing a moderate amount of fallout in the
> > > > > "[Xen-devel] [BUG] Xen vm kernel crash in get_free_entries." thread.
> > > > >
> > > >
> > > > As I understood it, that problem is slightly distinct. It's a case of
> > > > someone bringing up HVM linux in a XenServer VM and then getting a
> > > > crash because PVonHVM doesn't set up properly due to the change in
> > > > device_id for the *platform* device.
> > >
> > > How and when did this change? Was it just a XenServer quirk in a
> > > particular version?
> > >
> >
> > Yes, we changed it rather than adding the additional device as we
> > believed that we may end up shipping drivers to Windows Update and did
> > not want them to install on older XenServers. With hindsight that was
> > the wrong thing to do; using and additional device is a much better
> > solution.
> 
> This sounds like a tricky one to resolve compatibly in Linux -- we can't
> just bind to device id 0x2 since that is now the Citrix ID for the PV
> device.

Yes, probably best not to go there, even when I do fix the PV device.

> 
> Best bet is to just document "Don't Do That Then". Is there some
> XenServer way we can tell people to fix this (by changing the ID back?)
> 

Well part of the problem is that we don't support use of PVonHVM linux in 
XenServer at all! The best thing is probably to tackle this on the XenServer 
forums if and when someone posts the problem there. Adding the extra 
blacklisting code to XenServer's QEMU and then getting that into a hotfix 
should hopefully avoid future problems too - although we may get 'why are my PV 
frontends not working?'-type questions.

> >
> > > >  Using device ID to for the PV device should not cause such fallout as
> > > > the platform device is left unmolested.
> > > >
> > > > > I'm busy wondering if maybe Qemu should blacklist non-xenserver
> > > product
> > > > > ids when it has been configured with the Xen PV device using the
> Citrix
> > > > > devid?
> > > > >
> > > >
> > > > That should not be necessary as the point of registering the devid is
> > > > that on-one else creates driver that bind to that devid. Certainly
> > > > no-one has to date.
> > >
> > > The problem is that unplug time we don't necessarily have PCI yet to
> > > know if we are going to be able to bind to the device which eventually
> > > shows up.
> > >
> >
> > The issue is entirely XenServer's problem though. No change should be
> > needed upstream.
> 
> > The background is that in 6.1, we used product nr 3 as it was the next
> > unregistered number at the time.
> 
> Product number, not device ID?
> 
> >  We subsequently found out that Linux PVonHVM was using this without
> > having registered it. I thus registered it and moved the XenServer
> > drivers onto product nr 4. Unfortunately we still have to allow people
> > to use 6.1 drivers so we still have to allow drivers with product nr 3
> > to be used. I believe we can spot Linux PVonHVM though as the build
> > number is always small so it would actually be feasible to blacklist
> > Linux PVonHVM if the platform device_id is 2; hopefully that would
> > stop the crashes. I'll add a patch to XenServer's QEMU patch queue to
> > do that.
> >
> > > Remind me: When the PV device is in use the platform device must
> always
> > > be present? Right? That's the sort of thing which should be mentioned in
> > > the spec I asked about above.
> > >
> >
> > Yes it does, otherwise you can't unplug emulated devices.
> >
> > Given that you can't actually turn off the platform device in upstream
> > QEMU with xl at the moment, I wonder whether the correct solution is
> > simply to remove the option for doing so and have it always on in trad
> > too.
> 
> I think "no platform device and no pv device" is a valid and useful
> configuration, meaning "use emulated devices". "no platform device, yes
> pv device" is the one to avoid.
> 

Hmm. How about splitting out the fixed IO ports then? That way the platform 
device could be safely turned off if the PV device was present. It seems like a 
cleaner and safer thing to do.

  Paul
_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxx
http://lists.xen.org/xen-devel