Re: [Xen-users] pvgrub2(-like?) booting methods for PVHv2 guests

[Hans van Kranenburg <hans@xxxxxxxxxxx>]

On 02/07/2018 09:37 AM, Juergen Gross wrote:
> On 06/02/18 19:34, Hans van Kranenburg wrote:
>> On 02/06/2018 06:02 PM, Juergen Gross wrote:
>>> On 06/02/18 17:53, Hans van Kranenburg wrote:
>>>> On 02/06/2018 09:35 AM, Juergen Gross wrote:
>>>>> On 05/02/18 20:49, Hans van Kranenburg wrote:
>>>>>> On 01/25/2018 03:31 PM, Juergen Gross wrote:
>>>>>>> On 25/01/18 15:12, Hans van Kranenburg wrote:
>>>>>>>> On 01/25/2018 02:46 PM, Hans van Kranenburg wrote:
>>>>>>>>> On 25/01/2018 13:29, Juergen Gross wrote:
>>>>>>>>>> On 25/01/18 13:19, Andy Smith wrote:
>>>>>>>>>>> Hi Hans,
>>>>>>>>>>>
>>>>>>>>>>> On Thu, Jan 25, 2018 at 12:39:56PM +0100, Hans van Kranenburg wrote:
>>>>>>>>>>>> And now I get console output and things happen. Only it can't find 
>>>>>>>>>>>> the disk.
>>>>>>>>>>>
>>>>>>>>>>> I was trying similar thing (4.10 and PVH) and also ended up with a
>>>>>>>>>>> guest with no block devices. I reported this on grub-devel:
>>>>>>>>>>>
>>>>>>>>>>>     
>>>>>>>>>>> <http://lists.gnu.org/archive/html/grub-devel/2018-01/msg00018.html>
>>>>>>>>>>>
>>>>>>>>>>> as I was thinking this was not a Xen problem since same thing boots
>>>>>>>>>>> okay outside grub with direct kernel boot.
>>>>>>>>>>>
>>>>>>>>>>> Juergen did reply and said I needed this kernel patch in the guest:
>>>>>>>>>>>
>>>>>>>>>>>     
>>>>>>>>>>> <https://lists.xen.org/archives/html/xen-devel/2017-11/msg01681.html>
>>>>>>>>>>>
>>>>>>>>>>> But I think you have this don't you?
>>>>>>>>>
>>>>>>>>> Yes, see my earlier mail with all the steps that I did, step 6.
>>>>>>>>>
>>>>>>>>>> As the ACPI tables are found, I'd say yes. :-)
>>>>>>>>>
>>>>>>>>> dmesg output is pretty different when I boot directly with the kernel
>>>>>>>>> and initrd copied on the dom0.
>>>>>>>>>
>>>>>>>>> Remember, it's the same kernel/initrd, and without grub in between it
>>>>>>>>> boots with all vcpus network and disk.
>>>>>>>>>
>>>>>>>>> With grub in between, this at least does look suspicious:
>>>>>>>>>
>>>>>>>>> [    0.032110] PCI: System does not support PCI
>>>>>>>>
>>>>>>>> Eh, that's in both, stay awake Hans.
>>>>>>>>
>>>>>>>>> And yes, there are also no successful netfront lines.
>>>>>>>>>
>>>>>>>>> Without grub: http://paste.debian.net/plainh/7120cef2
>>>>>>>>> With grub: http://paste.debian.net/plainh/426bed60
>>>>>>>>
>>>>>>>> Or the diff between them, which shows what changes when inserting grub
>>>>>>>> in between:
>>>>>>>>
>>>>>>>> http://paste.debian.net/plainh/52b2d618
>>>>>>>>
>>>>>>>> I must admit I don't know too much (yet) about all those changed lines,
>>>>>>>> but this next is also a very interesting change?...
>>>>>>>>
>>>>>>>> -Booting paravirtualized kernel on Xen PVH
>>>>>>>> +Booting paravirtualized kernel on Xen HVM
>>>>>>>
>>>>>>> Aah, yes, this should be the reason for the problems.
>>>>>>>
>>>>>>> I addressed the ACPI problem first. What is missing now is to set PVH
>>>>>>> mode when booting via grub.
>>>>>>>
>>>>>>> So please stay tuned...
>>>>>>
>>>>>> For my info... Is this something like flipping a bit somewhere, or will
>>>>>> it involve a whole lot more complicated wizardry?
>>>>>>
>>>>>> Real question is of course if there's a PoC solution to apply that I can
>>>>>> use now? ;-] I'm throwing custom build kernels at my test environment,
>>>>>> and if I could save the time (and mistakes) of manually editing the
>>>>>> guest files and instead hit the pvgrub2+PVHv2 path many times to test...
>>>>>
>>>>> So I looked into this briefly and discovered that my memory really needs
>>>>> the backup I have on my hard disk: you want commit
>>>>> 418492ba40b2c2bbdaf1a169aac5b1673bde8189 which was for 4.15.
>>>>
>>>> Aha. Well, I can as well better jump to 4.15 now, since picking that
>>>> patch on 4.14.17 shows I need more other patches it depends on, related
>>>> to struct x86_legacy_features and x86_hyper_init reorganization.
>>>>
>>>> So, I just built a 4.15.1 kernel (using gcc 7.2.0), which definitely has
>>>> this one in it:
>>>>   418492ba40b2 x86/virt/xen: Use guest_late_init to detect Xen PVH guest
>>>>
>>>> I can boot via pvgrub2 (the Xen that's running now is current
>>>> stable-4.10 plus the RSDP for PVH guest near 4GB patch).
>>>>
>>>> But, it still tells me...
>>>>   Booting paravirtualized kernel on Xen HVM
>>>>
>>>> ...and then hangs somewhere halfway.
>>>>
>>>> Full boot log here: http://paste.debian.net/plainh/ae3ea14b
>>>
>>> You didn't add the RSDP detection patch, right? In the log I see:
>>>
>>> ACPI BIOS Error (bug): A valid RSDP was not found (20170831/tbxfroot-244)
>>
>> Yes, I didn't have those. Argh...
>>
>> However, after putting the 'xen: re-enable booting as Xen PVH guest' v2
>> patches on top, same happens.
> 
> Sure. Those don't take RSDP from boot parameters set by grub2.
> 
> You need to apply the 4 patch series from:
> 
> git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip.git x86/boot
> 
> Commit-Ids are:
> 2f74cbf947f45fa082dda8eac1a1f1299a372f49
> 0c89cf36424f7c1177de8a5712514d7cc2eb369f
> 88750a6c33f813b815516990f01fb5ee488c477e
> 930ba49b2ce7b09a5eddc21385fd944ba6b4e829

Ah, D'OH. The patch sets currently bite each other. And, for some reason
I messed up and was wrongly thinking I already had those in 4.15, since
which is entirely not the case. No idea why I thought that.

So, since I'm testing pvgrub2, I threw the 'xen: re-enable booting as
Xen PVH guest' out again, and applied those 4 again on 4.15.1.

Now I at least have this...
[    0.000000] Booting paravirtualized kernel on Xen PVH
...and no more 'A valid RSDP was not found'...

...but quite some complaining going on about ACPI, like 'BIOS bug: APIC
version mismatch' and 'ACPI BIOS Warning (bug): Incorrect checksum in
table' however, and per cpu there's a 10 second boot delay (didn't see
that one before yet), and I have only 1 vcpu in the end etc...

Full output:
http://paste.debian.net/plainh/262cfbcf

Hans

_______________________________________________
Xen-users mailing list
Xen-users@xxxxxxxxxxxxxxxxxxxx
https://lists.xenproject.org/mailman/listinfo/xen-users