Re: [Xen-users] issues with pvh mode for dom0 and domU on xen 4.5.0 / kernel 3.18.9

[Atom2 <ariel.atom2@xxxxxxxxxx>]

Am 15.06.15 um 09:33 schrieb Roger Pau Monné:
> Hello,
Roger, thanks for your reply.
> El 15/06/15 a les 0.46, Atom2 ha escrit:
> > Hi guys,
> > I recently switched from xen 4.4.2 to 4.5.0 after it became stable on
> > gentoo (kernel version is 3.18.9) and thought I'd give pvh for both dom0
> > and domUs a spin - unfortunately with not too much success:
> >
> >
> > DOM0:
> > For dom0 I simply added dom0pvh=1 to the xen command line. The system
> > booted up and I was also able to confirm that dom0 is running in the
> > correct mode by checking with xen-detect.
> >
> > What I however found out is that xen creates a bunch of error/warning
> > messages some of which make it to xl's dmesg file (the majority seems to
> > get dropped due to rate limits). None of these messages are there when
> > started without dom0pvh=1. Please see the two attached files for a
> > comparision between the xl dmesg for PVH ("dmesg.xl.pvh") and the xl
> > dmesg in non-PVH mode ("dmesg.xl").
> >
> > There are a few (most likely irrelevant) differences at line 109 to 111
> > relating to messages about the "Start info", the "Page tables", and the
> > "Boot stack". The main difference is in the additional lines in the file
> > "dmesg.xl.pvh" on line numbers 121-122 and 124-156 including 8 lines
> > about suppressed messages totalling in excess of 236.000 (ignored)
> > messages.  It's probably worth noteing that no further messages make it
> > to xl's dmesg and also /var/log/messages does not have anything strange
> > once the dom0 is up and running.
> Those are errors from the IOMMU, your BIOS is probably missing some RMRR
> regions. Do you know which devices are at 0000:00:1a.0 and 0000:00:02.0?
Sure:
0000:00:1a.0 is one of the two USB controllers as part of the chipset;
lspci output for this device is as follows:
00:1a.0 USB controller: Intel Corporation 6 Series/C200 Series Chipset 
Family USB Enhanced Host Controller #2 (rev 05)
0000:00:02.0 is the integrated graphics chip (IGD) in the XEON processor;
lspci output for this device is as follows:
00:02.0 VGA compatible controller: Intel Corporation Xeon E3-1200 
Processor Family Integrated Graphics Controller (rev 09)
For reference: The system is a C206 chipset motherboard with a XEON 
E3-1260L processor
The system has 32GB ECC memory installed

> (XEN) [VT-D]DMAR:[DMA Read] Request device [0000:00:1a.0] fault addr
> dc086000, iommu reg = ffff82c000203000
> (XEN) [VT-D]DMAR: reason 06 - PTE Read access is not set
>
> The above error looks reasonable, this memory is between 3 and 4 GiB
> which matches the MMIO hole.
>
> (XEN) [VT-D]DMAR:[DMA Write] Request device [0000:00:02.0] fault addr
> 72c481000, iommu reg = ffff82c000201000
> (XEN) [VT-D]DMAR: reason 05 - PTE Write access is not set
>
> This however is very high address, and is within a region marked as
> usable in the memory map:
>
> (XEN)  0000000100000000 - 000000081e600000 (usable)
>
> There are some patches on the list to add additional RMRR regions on the
> command line, however they are not committed yet, so there's not much
> you can try right now (apart from trying it on a different box).
That probably means I'll have to wait and see for further developments, 
most likely in 4.6
I guess there's also nothing for you guys to be gained from this report ...
> > It, however, appears that the pvh dom0 compared to the standard dom0
> > consumes _significantly_ more CPU time as shown by "xl info" from within
> > dom0 - which to me seems counter-intuitive given my (limited)
> > understanding of what pvh tries to achieve.
Any idea about that ...
> > DOMU:
> > For a test domU I just added pvh=1 to it's (otherwise unchanged)
> > configuration file and tried to start the domU by issuing
> > xl -v -v -v /path/to/config/file -c
> >
> > The domU did not come up at all (but works flawlessly when commenting
> > out the pvh=1 configuration line); details of the xl command output for
> > the failed attempt can be found in the attached file xl.domU. I honestly
> > can't make much sense out of the error message which in essence seems to
> > complain about an unsupported feature and a missing file or directory
> > before giving up.
> AFAICT from the log provided you seem to be trying to launch a MiniOS
> based guest with pvh=1, which is not supported. MiniOS doesn't support
> the PVH mode yet.
Well, I am using pvgrub to fire up the domUs and provide the grub 
configuration via a ramdisk:
=== relevant part of my config file ===
kernel                  = '/usr/libexec/xen/boot/pv-grub-x86_64.gz'
ramdisk                 = '/etc/xen/guests/grub.d/mysql.grub'
=== end relevant part of config file ===

I suppose pvgrub is then what you refer to with the term MiniOS.
Because other than this it's just a standard linux kernel w/ an 
initramfs that get's booted within the domU.
In other words: pvh currently does not work with pvgrub at all?
Any plans to change this - pvgrub IMHO is the most flexible tool to 
start a domU.
Once you get used to that, there's no way back ...
> Roger.
Thanks again Atom2

_______________________________________________
Xen-users mailing list
Xen-users@xxxxxxxxxxxxx
http://lists.xen.org/xen-users