Re: [Xen-users] pvgrub2(-like?) booting methods for PVHv2 guests

[Hans van Kranenburg <hans@xxxxxxxxxxx>]

On 25/01/2018 13:01, Juergen Gross wrote:
> On 25/01/18 12:39, Hans van Kranenburg wrote:
>> On 01/25/2018 07:01 AM, Juergen Gross wrote:
>>> On 24/01/18 17:37, Hans van Kranenburg wrote:
>>>> On 01/23/2018 06:41 AM, Juergen Gross wrote:
>>>>> On 23/01/18 05:46, Andy Smith wrote:
>>>>>> Hi,
>>>>>>
>>>>>> What are the current options for booting PVHv2 guests in Xen 4.10.0?
>>>>>>
>>>>>> As far as I can see there is only direct kernel booting, i.e.
>>>>>> supplying "kernel = /path/to/vmlinux".
>>>>>>
>>>>>> In PV land there is pvgrub which is very convenient for being able
>>>>>> to use kernel and initramfs, but I am not seeing support for that in
>>>>>> upstream grub2. I see that Juergen Gross post a patch series that
>>>>>> claims to add this feature:
>>>>>>
>>>>>>     <https://lists.gnu.org/archive/html/grub-devel/2017-11/msg00054.html>
>>>>>>
>>>>>> …but this doesn't seem to have been applied yet. Is it an ongoing
>>>>>> idea or has that method been abandoned?
>>>>>
>>>>> Still waiting for review.
>>>>
>>>> Since I wanted to try to test this, I scraped the patches off the
>>>> grub-devel list and did the following:
>>>>
>>>> 1. clone https://git.savannah.gnu.org/git/grub.git
>>>> 2. apply patches on top of master, plus an extra one with a puzzle piece
>>>> to make it actually compile (see attachment)
>>>>
>>>> 3. build:
>>>> ./autogen.sh
>>>> ./configure --target=i386 --with-platform=xenpvh
>>>> make
>>>>
>>>> 4. use mkstandalone to create a grub image to boot
>>>> ./grub-mkstandalone -d grub-core --locale-directory=/ -o
>>>> ~/grub-i386-xenpvh-fire-ze-missile -O i386-xenpvh
>>>> ~/build/pvgrub2/pvgrub2/xvda-fire-ze-missile.cfg
>>>>
>>>> contents of the config file that I want to have inserted into it (same
>>>> file I use for booting PV now):
>>>>
>>>> root='(xen/xvda)'
>>>> insmod xzio
>>>> insmod gzio
>>>> insmod btrfs
>>>> insmod ext2
>>>> echo 'Loading Linux ...'
>>>> linux /vmlinuz root=/dev/xvda ro elevator=noop
>>>> echo 'Loading initial ramdisk ...'
>>>> initrd /initrd.img
>>>> boot
>>>>
>>>> 5. Try to use it already, to see what happens. (Xen 4.10.0)
>>>>
>>>> -# cat yolo-pvgrub2
>>>>
>>>> type="pvh"
>>>> kernel = "/yolo/grub-i386-xenpvh-fire-ze-missile"
>>>> name = "yolo-pvgrub2"
>>>> memory = 1024
>>>> vcpus = 10
>>>> vif = [
>>>>     "mac=02:00:0a:8c:d9:0d,bridge=ovs0.217",
>>>> ]
>>>> disk = [
>>>>     '/dev/test/yolo,raw,xvda,rw',
>>>> ]
>>>>
>>>> -# xen create -c yolo-pvgrub2
>>>> Parsing config from yolo-pvgrub2
>>>> libxl: info: libxl_create.c:109:libxl__domain_build_info_setdefault:
>>>> qemu-xen is unavailable, using qemu-xen-traditional instead: No such
>>>> file or directory
>>>> S3 disabled
>>>> S4 disabled
>>>> CONV disabled
>>>>
>>>>                      GNU GRUB  version 2.03
>>>>
>>>>    Minimal BASH-like line editing is supported. For the first word, TAB
>>>>    lists possible command completions. Anywhere else TAB lists possible
>>>>    device or file completions.
>>>>
>>>>
>>>> grub>
>>>>
>>>> Ok, so for some reason it seems to ignore the config? But I can type
>>>> things manually:
>>>>
>>>> grub> insmod xzio
>>>> grub> insmod gzio
>>>> grub> insmod btrfs
>>>> grub> insmod ext2
>>>> grub> search.file /vmlinuz
>>>>  xen/xvda
>>>>
>>>> Ok, so it can see the disk and fine the symlink to the kernel image. Now
>>>> let's try the rest of it:
>>>>
>>>> grub> root='(xen/xvda)'
>>>> grub> linux /vmlinuz root=/dev/xvda ro elevator=noop
>>>> grub> initrd /initrd.img
>>>> grub> boot
>>>>
>>>> And now it just hangs, no output any more.
>>>>
>>>> 6. Apply linux patches to the domU kernel and Xen
>>>>
>>>> domU Linux kernel:
>>>> - Branch off from linux 4.14.14
>>>> - add https://git.kernel.org/pub/scm/linux/kernel/git/tip/tip.git as
>>>> remote and cherry-pick the 4 patches mentioned in
>>>>   https://lists.gnu.org/archive/html/grub-devel/2017-12/msg00017.html
>>>>
>>>> Xen: pick this one on top of 4.10.0:
>>>>  https://lists.xen.org/archives/html/xen-devel/2017-11/msg01807.html
>>>>  4a5733771e6f33918eba07b584cccce564a67ac1
>>>>
>>>> 7. Test again after dom0 reboot into the new xen and put the new domU
>>>> kernel on it.
>>>>  -> same result: grub command prompt, and no activity after 'boot'.
>>>>
>>>> Extra things:
>>>>
>>>> 8. Start domU after copying kernel/initrd to the dom0:
>>>>
>>>> type="pvh"
>>>> kernel = "/yolo/vmlinuz-4.14.14-yolo"
>>>> ramdisk = "/yolo/initrd.img-4.14.14-yolo"
>>>> root = "/dev/xvda ro elevator=noop console=hvc0"
>>>>
>>>> This boots, dmesg: http://paste.debian.net/plainh/7120cef2
>>>>
>>>> 9. Just for fun, start domU with previous kernel/initrd without the 
>>>> patches:
>>>>
>>>> type="pvh"
>>>> kernel = "/yolo/vmlinuz-4.14.0-3-amd64"
>>>> ramdisk = "/yolo/initrd.img-4.14.0-3-amd64"
>>>> root = "/dev/xvda ro elevator=noop console=hvc0"
>>>>
>>>> This boots and has 1 vcpu and no ACPI lines in dmesg, which is expected
>>>> breakage, I believe, reading xen-devel.
>>>>
>>>> So, what am I missing?
>>>
>>> I used the attached script to build it (derived from the pvgrub build
>>> script used in SUSE's grub-xen RPM).
>>
>> Thanks. This gets me a step further.
>>
>> Using the "graft point syntax" thingie
>> ("/boot/grub/grub.cfg=./grub.cfg") helped getting the config actually
>> executed instead of getting the prompt.
>>
>> Next I got:
>>
>> Welcome to GRUB!
>>
>> Loading Linux ...
>> Loading initial ramdisk ...
>>
>> ...and then it stops.
>>
>> So I modified my grub config a bit from the one shown above, adding
>> explicit console=hvc0:
>>
>> root='(xen/xvda)'
>> insmod xzio
>> insmod gzio
>> insmod btrfs
>> insmod ext2
>> echo 'Loading Linux ...'
>> linux /vmlinuz root=/dev/xvda ro elevator=noop console=hvc0
>> echo 'Loading initial ramdisk ...'
>> initrd /initrd.img
>> echo 'There we go! ...'
>> boot
>>
>> And now I get console output and things happen. Only it can't find the disk.
>>
>> [...]
>> Begin: Running /scripts/local-block ... done.
>> Begin: Running /scripts/local-block ... done.
>> done.
>> Gave up waiting for root file system device.  Common problems:
>>  - Boot args (cat /proc/cmdline)
>>    - Check rootdelay= (did the system wait long enough?)
>>  - Missing modules (cat /proc/modules; ls /dev)
>> ALERT!  /dev/xvda does not exist.  Dropping to a shell!
>> (initramfs)
>>
>> Full log at http://paste.debian.net/plainh/426bed60
>>
>> Now I have to find out why's that. But first time for lunch. ;)
> 
> Missing xen-blkfront driver in initrd?

Exactly the same kernel/initrd boots successfully when directly loading
them from dom0 without grub in between, see boot output I pastebinned
earlier.

K

_______________________________________________
Xen-users mailing list
Xen-users@xxxxxxxxxxxxxxxxxxxx
https://lists.xenproject.org/mailman/listinfo/xen-users