Re: [Xen-users] pvgrub2(-like?) booting methods for PVHv2 guests

[Juergen Gross <jgross@xxxxxxxx>]

On 25/01/18 12:39, Hans van Kranenburg wrote:
> On 01/25/2018 07:01 AM, Juergen Gross wrote:
>> On 24/01/18 17:37, Hans van Kranenburg wrote:
>>> On 01/23/2018 06:41 AM, Juergen Gross wrote:
>>>> On 23/01/18 05:46, Andy Smith wrote:
>>>>> Hi,
>>>>>
>>>>> What are the current options for booting PVHv2 guests in Xen 4.10.0?
>>>>>
>>>>> As far as I can see there is only direct kernel booting, i.e.
>>>>> supplying "kernel = /path/to/vmlinux".
>>>>>
>>>>> In PV land there is pvgrub which is very convenient for being able
>>>>> to use kernel and initramfs, but I am not seeing support for that in
>>>>> upstream grub2. I see that Juergen Gross post a patch series that
>>>>> claims to add this feature:
>>>>>
>>>>>     <https://lists.gnu.org/archive/html/grub-devel/2017-11/msg00054.html>
>>>>>
>>>>> …but this doesn't seem to have been applied yet. Is it an ongoing
>>>>> idea or has that method been abandoned?
>>>>
>>>> Still waiting for review.
>>>
>>> Since I wanted to try to test this, I scraped the patches off the
>>> grub-devel list and did the following:
>>>
>>> 1. clone https://git.savannah.gnu.org/git/grub.git
>>> 2. apply patches on top of master, plus an extra one with a puzzle piece
>>> to make it actually compile (see attachment)
>>>
>>> 3. build:
>>> ./autogen.sh
>>> ./configure --target=i386 --with-platform=xenpvh
>>> make
>>>
>>> 4. use mkstandalone to create a grub image to boot
>>> ./grub-mkstandalone -d grub-core --locale-directory=/ -o
>>> ~/grub-i386-xenpvh-fire-ze-missile -O i386-xenpvh
>>> ~/build/pvgrub2/pvgrub2/xvda-fire-ze-missile.cfg
>>>
>>> contents of the config file that I want to have inserted into it (same
>>> file I use for booting PV now):
>>>
>>> root='(xen/xvda)'
>>> insmod xzio
>>> insmod gzio
>>> insmod btrfs
>>> insmod ext2
>>> echo 'Loading Linux ...'
>>> linux /vmlinuz root=/dev/xvda ro elevator=noop
>>> echo 'Loading initial ramdisk ...'
>>> initrd /initrd.img
>>> boot
>>>
>>> 5. Try to use it already, to see what happens. (Xen 4.10.0)
>>>
>>> -# cat yolo-pvgrub2
>>>
>>> type="pvh"
>>> kernel = "/yolo/grub-i386-xenpvh-fire-ze-missile"
>>> name = "yolo-pvgrub2"
>>> memory = 1024
>>> vcpus = 10
>>> vif = [
>>>     "mac=02:00:0a:8c:d9:0d,bridge=ovs0.217",
>>> ]
>>> disk = [
>>>     '/dev/test/yolo,raw,xvda,rw',
>>> ]
>>>
>>> -# xen create -c yolo-pvgrub2
>>> Parsing config from yolo-pvgrub2
>>> libxl: info: libxl_create.c:109:libxl__domain_build_info_setdefault:
>>> qemu-xen is unavailable, using qemu-xen-traditional instead: No such
>>> file or directory
>>> S3 disabled
>>> S4 disabled
>>> CONV disabled
>>>
>>>                      GNU GRUB  version 2.03
>>>
>>>    Minimal BASH-like line editing is supported. For the first word, TAB
>>>    lists possible command completions. Anywhere else TAB lists possible
>>>    device or file completions.
>>>
>>>
>>> grub>
>>>
>>> Ok, so for some reason it seems to ignore the config? But I can type
>>> things manually:
>>>
>>> grub> insmod xzio
>>> grub> insmod gzio
>>> grub> insmod btrfs
>>> grub> insmod ext2
>>> grub> search.file /vmlinuz
>>>  xen/xvda
>>>
>>> Ok, so it can see the disk and fine the symlink to the kernel image. Now
>>> let's try the rest of it:
>>>
>>> grub> root='(xen/xvda)'
>>> grub> linux /vmlinuz root=/dev/xvda ro elevator=noop
>>> grub> initrd /initrd.img
>>> grub> boot
>>>
>>> And now it just hangs, no output any more.
>>>
>>> 6. Apply linux patches to the domU kernel and Xen
>>>
>>> domU Linux kernel:
>>> - Branch off from linux 4.14.14
>>> - add https://git.kernel.org/pub/scm/linux/kernel/git/tip/tip.git as
>>> remote and cherry-pick the 4 patches mentioned in
>>>   https://lists.gnu.org/archive/html/grub-devel/2017-12/msg00017.html
>>>
>>> Xen: pick this one on top of 4.10.0:
>>>  https://lists.xen.org/archives/html/xen-devel/2017-11/msg01807.html
>>>  4a5733771e6f33918eba07b584cccce564a67ac1
>>>
>>> 7. Test again after dom0 reboot into the new xen and put the new domU
>>> kernel on it.
>>>  -> same result: grub command prompt, and no activity after 'boot'.
>>>
>>> Extra things:
>>>
>>> 8. Start domU after copying kernel/initrd to the dom0:
>>>
>>> type="pvh"
>>> kernel = "/yolo/vmlinuz-4.14.14-yolo"
>>> ramdisk = "/yolo/initrd.img-4.14.14-yolo"
>>> root = "/dev/xvda ro elevator=noop console=hvc0"
>>>
>>> This boots, dmesg: http://paste.debian.net/plainh/7120cef2
>>>
>>> 9. Just for fun, start domU with previous kernel/initrd without the patches:
>>>
>>> type="pvh"
>>> kernel = "/yolo/vmlinuz-4.14.0-3-amd64"
>>> ramdisk = "/yolo/initrd.img-4.14.0-3-amd64"
>>> root = "/dev/xvda ro elevator=noop console=hvc0"
>>>
>>> This boots and has 1 vcpu and no ACPI lines in dmesg, which is expected
>>> breakage, I believe, reading xen-devel.
>>>
>>> So, what am I missing?
>>
>> I used the attached script to build it (derived from the pvgrub build
>> script used in SUSE's grub-xen RPM).
> 
> Thanks. This gets me a step further.
> 
> Using the "graft point syntax" thingie
> ("/boot/grub/grub.cfg=./grub.cfg") helped getting the config actually
> executed instead of getting the prompt.
> 
> Next I got:
> 
> Welcome to GRUB!
> 
> Loading Linux ...
> Loading initial ramdisk ...
> 
> ...and then it stops.
> 
> So I modified my grub config a bit from the one shown above, adding
> explicit console=hvc0:
> 
> root='(xen/xvda)'
> insmod xzio
> insmod gzio
> insmod btrfs
> insmod ext2
> echo 'Loading Linux ...'
> linux /vmlinuz root=/dev/xvda ro elevator=noop console=hvc0
> echo 'Loading initial ramdisk ...'
> initrd /initrd.img
> echo 'There we go! ...'
> boot
> 
> And now I get console output and things happen. Only it can't find the disk.
> 
> [...]
> Begin: Running /scripts/local-block ... done.
> Begin: Running /scripts/local-block ... done.
> done.
> Gave up waiting for root file system device.  Common problems:
>  - Boot args (cat /proc/cmdline)
>    - Check rootdelay= (did the system wait long enough?)
>  - Missing modules (cat /proc/modules; ls /dev)
> ALERT!  /dev/xvda does not exist.  Dropping to a shell!
> (initramfs)
> 
> Full log at http://paste.debian.net/plainh/426bed60
> 
> Now I have to find out why's that. But first time for lunch. ;)

Missing xen-blkfront driver in initrd?


Juergen


_______________________________________________
Xen-users mailing list
Xen-users@xxxxxxxxxxxxxxxxxxxx
https://lists.xenproject.org/mailman/listinfo/xen-users