Re: [Xen-users] pvgrub2(-like?) booting methods for PVHv2 guests

[Hans van Kranenburg <hans@xxxxxxxxxxx>]

On 02/06/2018 07:34 PM, Hans van Kranenburg wrote:
> On 02/06/2018 06:02 PM, Juergen Gross wrote:
>> On 06/02/18 17:53, Hans van Kranenburg wrote:
>>> On 02/06/2018 09:35 AM, Juergen Gross wrote:
>>>> On 05/02/18 20:49, Hans van Kranenburg wrote:
>>>>> On 01/25/2018 03:31 PM, Juergen Gross wrote:
>>>>>> On 25/01/18 15:12, Hans van Kranenburg wrote:
>>>>>>> On 01/25/2018 02:46 PM, Hans van Kranenburg wrote:
>>>>>>>> On 25/01/2018 13:29, Juergen Gross wrote:
>>>>>>>>> On 25/01/18 13:19, Andy Smith wrote:
>>>>>>>>>> Hi Hans,
>>>>>>>>>>
>>>>>>>>>> On Thu, Jan 25, 2018 at 12:39:56PM +0100, Hans van Kranenburg wrote:
>>>>>>>>>>> And now I get console output and things happen. Only it can't find 
>>>>>>>>>>> the disk.
>>>>>>>>>>
>>>>>>>>>> I was trying similar thing (4.10 and PVH) and also ended up with a
>>>>>>>>>> guest with no block devices. I reported this on grub-devel:
>>>>>>>>>>
>>>>>>>>>>     
>>>>>>>>>> <http://lists.gnu.org/archive/html/grub-devel/2018-01/msg00018.html>
>>>>>>>>>>
>>>>>>>>>> as I was thinking this was not a Xen problem since same thing boots
>>>>>>>>>> okay outside grub with direct kernel boot.
>>>>>>>>>>
>>>>>>>>>> Juergen did reply and said I needed this kernel patch in the guest:
>>>>>>>>>>
>>>>>>>>>>     
>>>>>>>>>> <https://lists.xen.org/archives/html/xen-devel/2017-11/msg01681.html>
>>>>>>>>>>
>>>>>>>>>> But I think you have this don't you?
>>>>>>>>
>>>>>>>> Yes, see my earlier mail with all the steps that I did, step 6.
>>>>>>>>
>>>>>>>>> As the ACPI tables are found, I'd say yes. :-)
>>>>>>>>
>>>>>>>> dmesg output is pretty different when I boot directly with the kernel
>>>>>>>> and initrd copied on the dom0.
>>>>>>>>
>>>>>>>> Remember, it's the same kernel/initrd, and without grub in between it
>>>>>>>> boots with all vcpus network and disk.
>>>>>>>>
>>>>>>>> With grub in between, this at least does look suspicious:
>>>>>>>>
>>>>>>>> [    0.032110] PCI: System does not support PCI
>>>>>>>
>>>>>>> Eh, that's in both, stay awake Hans.
>>>>>>>
>>>>>>>> And yes, there are also no successful netfront lines.
>>>>>>>>
>>>>>>>> Without grub: http://paste.debian.net/plainh/7120cef2
>>>>>>>> With grub: http://paste.debian.net/plainh/426bed60
>>>>>>>
>>>>>>> Or the diff between them, which shows what changes when inserting grub
>>>>>>> in between:
>>>>>>>
>>>>>>> http://paste.debian.net/plainh/52b2d618
>>>>>>>
>>>>>>> I must admit I don't know too much (yet) about all those changed lines,
>>>>>>> but this next is also a very interesting change?...
>>>>>>>
>>>>>>> -Booting paravirtualized kernel on Xen PVH
>>>>>>> +Booting paravirtualized kernel on Xen HVM
>>>>>>
>>>>>> Aah, yes, this should be the reason for the problems.
>>>>>>
>>>>>> I addressed the ACPI problem first. What is missing now is to set PVH
>>>>>> mode when booting via grub.
>>>>>>
>>>>>> So please stay tuned...
>>>>>
>>>>> For my info... Is this something like flipping a bit somewhere, or will
>>>>> it involve a whole lot more complicated wizardry?
>>>>>
>>>>> Real question is of course if there's a PoC solution to apply that I can
>>>>> use now? ;-] I'm throwing custom build kernels at my test environment,
>>>>> and if I could save the time (and mistakes) of manually editing the
>>>>> guest files and instead hit the pvgrub2+PVHv2 path many times to test...
>>>>
>>>> So I looked into this briefly and discovered that my memory really needs
>>>> the backup I have on my hard disk: you want commit
>>>> 418492ba40b2c2bbdaf1a169aac5b1673bde8189 which was for 4.15.
>>>
>>> Aha. Well, I can as well better jump to 4.15 now, since picking that
>>> patch on 4.14.17 shows I need more other patches it depends on, related
>>> to struct x86_legacy_features and x86_hyper_init reorganization.
>>>
>>> So, I just built a 4.15.1 kernel (using gcc 7.2.0), which definitely has
>>> this one in it:
>>>   418492ba40b2 x86/virt/xen: Use guest_late_init to detect Xen PVH guest
>>>
>>> I can boot via pvgrub2 (the Xen that's running now is current
>>> stable-4.10 plus the RSDP for PVH guest near 4GB patch).
>>>
>>> But, it still tells me...
>>>   Booting paravirtualized kernel on Xen HVM
>>>
>>> ...and then hangs somewhere halfway.
>>>
>>> Full boot log here: http://paste.debian.net/plainh/ae3ea14b
>>
>> You didn't add the RSDP detection patch, right? In the log I see:
>>
>> ACPI BIOS Error (bug): A valid RSDP was not found (20170831/tbxfroot-244)
> 
> Yes, I didn't have those. Argh...
> 
> However, after putting the 'xen: re-enable booting as Xen PVH guest' v2
> patches on top, same happens.
> 
> ACPI BIOS Error (bug): A valid RSDP was not found ...
> 
> When taking pvgrub2 out and directly booting a copy of the
> kernel/initrd, it does actually boot:
> 
> http://paste.debian.net/plainh/896936d4
> 
> So, the pvgrub2 path is now borken again?
> 
> My pvgrub2 binary is still the same one as before.

To be sure I'm not fooling myself, I reverted the two 're-enable booting
as Xen PVH guest' patches and now 4.15 also fails to boot directly
without pvgrub, so that's good.

Hans

_______________________________________________
Xen-users mailing list
Xen-users@xxxxxxxxxxxxxxxxxxxx
https://lists.xenproject.org/mailman/listinfo/xen-users