[Xen-users] How to debug dom0 sluggishness?

[Dario Bertini <berdario@xxxxxxxxx>]

Some months ago I ran an helloworld MirageOS app on Xen on my machine
without any problem, now I wanted to get back into using Xen, but I
have some issues.

Plenty of things changed: The previous machine (for which I don't have
access anymore) on which I run it was a Dell laptop with both
integrated (intel) and discrete (amd) gpus, while now I'm running on a
Thinkpad with only intel. I was running Ubuntu 15.04, while now I'm
running 15.10 (shouldn't change much, but maybe the different kernel
version is at fault). Now I'm also running Btrfs for my / filesystem
(mainly for the checksumming).

So, after booting dom0, on my natively installed Linux (Ubuntu),
everything seems fine, but after a split second, the mouse pointer
disappears from both lightdm and my unity/gnome session after login.

Also, I see that the cpu usage from my activity-monitor widget is
apparently almost to 100%, but while looking to top, I only see things
like Xorg using ~16%.

Thinking that memory might have been an issue, I try to create a
zero-ed 2GB file to be used as swap (I know: I cannot use a swapfile
that simply with btrfs), and I realized that it's incredibly slow:
20MB/s to write it to disk. After rebooting into my normal kernel, I
try to create the same kind of file, and I get a bandwidth around
300MB/s (as expected with my SSD).

The whole gui, on the xen kernel, seems sluggish... alt-tabbing and
switching desktops has noticeable delays.... I was thinking that maybe
the intel graphic drivers would be the issue, but the disk slowness,
and increased apparent cpu usage, lead me to think that it might not
be that simple.

I tried to do a few things: install/uninstall Virtualbox (so as not to
load vboxdrv), enable/disable VT-x in the EFI... but I don't see any
improvement.

This is the dmesg output when running with the Xen kernel.
https://gist.github.com/berdario/c91959d90960f03a0c7f

This is after adding `pci=biosirq` to the boot options (I added it to
both the `multiboot` and the `module` line, since I don't know where
it belongs, thus why I got `PCI: Unknown option `biosirq'`, I think)
https://gist.github.com/berdario/70ea71d8e7d7000afd4d

 The few that seem relevant are:

    pci 0000:00:14.0: can't find IRQ for PCI INT A; please try using pci=biosirq

    ACPI Exception: AE_BAD_PARAMETER, Thread 79174528 could not
acquire Mutex [0x1] (20150619/utmutex-285)

and

    vgaarb: this pci device is not a vga device

and this is the output of `xl dmesg`

https://gist.github.com/berdario/575a87ad6818b9f65fcd

For the 2n error, I tried to set `acpi=off` in the boot options (but
Xen it's not using the usual `linux` command, but a `multiboot` one...
so I might have put the option in the wrong place)

In both cases I tried to google for possible known causes of this
problem, and apparently other people had similar issues
https://bbs.archlinux.org/viewtopic.php?id=187467, but none of the
resuls seem to lead to anything. Same thing for the xen wiki
http://wiki.xenproject.org/wiki/Xen_Common_Problems#Host_.2F_Dom0
http://wiki.xenproject.org/wiki/Xen_FAQ_Dom0

I tried to have a look at http://wiki.xenproject.org/wiki/Debugging_Xen
I haven't been able to run xen-bugtool, since the needed xen.util
module doesn't seem to be available on the system (nor on PYPI, so I'm
not sure where to install it from)

What would the next step be? How to pin down the exact issue?

Thank you

PS: after writing all this, I tried to google again one of the IRQ
errors, and I found this thread

http://lists.xenproject.org/archives/html/xen-devel/2013-11/msg04000.html

Since Ubuntu ships with Xen4.5, and the EFI boot is apparently untested on it

http://wiki.xenproject.org/wiki/Xen_EFI
http://xenbits.xenproject.org/docs/unstable/misc/efi.html

> For x86, building xen.efi requires gcc 4.5.x or above (4.6.x or newer 
> recommended, as 4.5.x was probably never really tested for this purpose)

Apparently I'm out of luck.

I tried to setup the chainload nonetheless, but it kicked me straight
back into the main grub menu.

And I could try to manually configure xen to boot directly from the
EFI... but Ubuntu doesn't drop the .efi file in the EFI partition, so
if I do it manually it'd probably break with every kernel update

Enabling the "legacy boot" option in the EFI still wouldn't work,
without reinstalling the bootloader (grub EFI cannot load linux/xen in
"bios" mode, from my understanding).

So, I updated the question that I opened on stackexchange:
http://unix.stackexchange.com/questions/245400/how-to-debug-xen-dom0-sluggishness/246260#246260

and concluded: "I'll now resort to test xen on another (virtual?) machine."

But, while discussing this on irc, dwfreed persuaded me that the issue
might not be the Grub/EFI booting after all, and so I'm now sending
this email.

If you have any suggestion, or maybe another solution to this EFI
issue (maybe there's a prebuilt version of Xen 4.6 somewhere?) it'd be
welcome.


Thank you

_______________________________________________
Xen-users mailing list
Xen-users@xxxxxxxxxxxxx
http://lists.xen.org/xen-users