Re: [PATCH v8] xen/pt: reserve PCI slot 2 for Intel igd-passthru

[Igor Mammedov <imammedo@xxxxxxxxxx>]

On Mon, 16 Jan 2023 13:00:53 -0500
Chuck Zmudzinski <brchuckz@xxxxxxxxxxxx> wrote:

> On 1/16/23 10:33, Igor Mammedov wrote:
> > On Fri, 13 Jan 2023 16:31:26 -0500
> > Chuck Zmudzinski <brchuckz@xxxxxxx> wrote:
> >   
> >> On 1/13/23 4:33 AM, Igor Mammedov wrote:  
> >> > On Thu, 12 Jan 2023 23:14:26 -0500
> >> > Chuck Zmudzinski <brchuckz@xxxxxxx> wrote:
> >> >     
> >> >> On 1/12/23 6:03 PM, Michael S. Tsirkin wrote:    
> >> >> > On Thu, Jan 12, 2023 at 10:55:25PM +0000, Bernhard Beschow wrote:     
> >> >> >  
> >> >> >> I think the change Michael suggests is very minimalistic: Move the if
> >> >> >> condition around xen_igd_reserve_slot() into the function itself and
> >> >> >> always call it there unconditionally -- basically turning three lines
> >> >> >> into one. Since xen_igd_reserve_slot() seems very problem specific,
> >> >> >> Michael further suggests to rename it to something more general. All
> >> >> >> in all no big changes required.      
> >> >> > 
> >> >> > yes, exactly.
> >> >> >       
> >> >> 
> >> >> OK, got it. I can do that along with the other suggestions.    
> >> > 
> >> > have you considered instead of reservation, putting a slot check in 
> >> > device model
> >> > and if it's intel igd being passed through, fail at realize time  if it 
> >> > can't take
> >> > required slot (with a error directing user to fix command line)?    
> >> 
> >> Yes, but the core pci code currently already fails at realize time
> >> with a useful error message if the user tries to use slot 2 for the
> >> igd, because of the xen platform device which has slot 2. The user
> >> can fix this without patching qemu, but having the user fix it on
> >> the command line is not the best way to solve the problem, primarily
> >> because the user would need to hotplug the xen platform device via a
> >> command line option instead of having the xen platform device added by
> >> pc_xen_hvm_init functions almost immediately after creating the pci
> >> bus, and that delay in adding the xen platform device degrades
> >> startup performance of the guest.
> >>   
> >> > That could be less complicated than dealing with slot reservations at 
> >> > the cost of
> >> > being less convenient.    
> >> 
> >> And also a cost of reduced startup performance  
> > 
> > Could you clarify how it affects performance (and how much).
> > (as I see, setup done at board_init time is roughly the same
> > as with '-device foo' CLI options, modulo time needed to parse
> > options which should be negligible. and both ways are done before
> > guest runs)  
> 
> I preface my answer by saying there is a v9, but you don't
> need to look at that. I will answer all your questions here.
> 
> I am going by what I observe on the main HDMI display with the
> different approaches. With the approach of not patching Qemu
> to fix this, which requires adding the Xen platform device a
> little later, the length of time it takes to fully load the
> guest is increased. I also noticed with Linux guests that use
> the grub bootoader, the grub vga driver cannot display the
> grub boot menu at the native resolution of the display, which
> in the tested case is 1920x1080, when the Xen platform device
> is added via a command line option instead of by the
> pc_xen_hvm_init_pci fucntion in pc_piix.c, but with this patch
> to Qemu, the grub menu is displayed at the full, 1920x1080
> native resolution of the display. Once the guest fully loads,
> there is no noticeable difference in performance. It is mainly
> a degradation in startup performance, not performance once
> the guest OS is fully loaded.

Looking at igd-assign.txt, it recommends to add IGD using '-device' CLI
option, and actually drop at least graphics defaults explicitly.
So it is expected to work fine even when IGD is constructed with
'-device'.

Could you provide full CLI current xen starts QEMU with and then
a CLI you used (with explicit -device for IGD) that leads
to reduced performance?

CCing vfio folks who might have an idea what could be wrong based
on vfio experience.