Re: Xen PVH domU start-of-day VCPU state

[Roger Pau Monné <roger.pau@xxxxxxxxxx>]

On Tue, May 26, 2020 at 05:42:24PM +0200, Martin Lucina wrote:
> Oh! I think I've found a solution, even though I don't entirely understand the
> problem/root cause:
> 
> On Tuesday, 26.05.2020 at 12:12, Martin Lucina wrote:
> > > On Tue, May 26, 2020 at 11:34:21AM +0200, Roger Pau Monné wrote:
> > > Forgot to ask, but can you also add the output of readelf -lW
> > > <kernel>?
> > 
> >     Elf file type is EXEC (Executable file)
> >     Entry point 0x1001e0
> >     There are 7 program headers, starting at offset 64
> > 
> >     Program Headers:
> >       Type           Offset   VirtAddr           PhysAddr           FileSiz 
> >  MemSiz   Flg Align
> >       INTERP         0x001000 0x0000000000100000 0x0000000000100000 
> > 0x000018 0x000018 R   0x8
> >           [Requesting program interpreter: /nonexistent/solo5/]
> >       LOAD           0x001000 0x0000000000100000 0x0000000000100000 
> > 0x00626c 0x00626c R E 0x1000
> >       LOAD           0x008000 0x0000000000107000 0x0000000000107000 
> > 0x007120 0x00ed48 RW  0x1000
> >       NOTE           0x0080ac 0x00000000001070ac 0x00000000001070ac 
> > 0x000018 0x000018 R   0x4
> >       NOTE           0x00f120 0x00000000001070c4 0x00000000001070c4 
> > 0x000014 0x000000 R   0x4
>                                                                               
>  ^^^^^^^^
> 
> I should have picked up on the above, but thought it wasn't relevant.
> 
> >       NOTE           0x008088 0x0000000000107088 0x0000000000107088 
> > 0x000024 0x000024 R   0x4
> >       NOTE           0x008000 0x0000000000107000 0x0000000000107000 
> > 0x000088 0x000088 R   0x4
> > 
> >      Section to Segment mapping:
> >       Segment Sections...
> >        00     .interp
> >        01     .interp .text .rodata .eh_frame
> >        02     .note.solo5.manifest .note.solo5.abi .note.solo5.not-openbsd 
> > .data .bss
> 
> And also the missing .note.solo5.xen above.
> 
> >        03     .note.solo5.not-openbsd
> >        04     .note.solo5.xen
> >        05     .note.solo5.abi
> >        06     .note.solo5.manifest
> 
> Turns out that the .note.solo5.xen section as defined in boot.S was not
> marked allocatable, and that was doing <something> that was confusing our
> linker script[1] (?).

Hm, I would have said there was no need to load notes into memory, and
hence using a MemSize of 0 would be fine.

Maybe libelf loader was somehow getting confused and not loading the
image properly?

Can you paste the output of `xl -vvv create ...` when using the broken
image?

> 
> If I make this simple change:
> 
> --- a/bindings/xen/boot.S
> +++ b/bindings/xen/boot.S
> @@ -32,7 +32,7 @@
>  #define ENTRY(x) .text; .globl x; .type x,%function; x:
>  #define END(x)   .size x, . - x
> 
> -.section .note.solo5.xen
> +.section .note.solo5.xen, "a", @note
> 
>         .align  4
>         .long   4
> 
> then I get the expected output from readelf -lW, and I can get as far as
> the C _start() with no issues!
> 
> FWIW, here's the diff of readelf -lW before/after:
> 
> --- before    2020-05-26 17:36:46.117885855 +0200
> +++ after     2020-05-26 17:38:07.090508322 +0200
> @@ -8,9 +8,9 @@
>    INTERP         0x001000 0x0000000000100000 0x0000000000100000 0x000018 
> 0x000018 R   0x8
>        [Requesting program interpreter: /nonexistent/solo5/]
>    LOAD           0x001000 0x0000000000100000 0x0000000000100000 0x00615c 
> 0x00615c R E 0x1000
> -  LOAD           0x008000 0x0000000000107000 0x0000000000107000 0x007120 
> 0x00ed28 RW  0x1000
> +  LOAD           0x008000 0x0000000000107000 0x0000000000107000 0x006120 
> 0x00dd28 RW  0x1000

This seems suspicious, there's a change of the size of the LOAD
section, but your change to the note type should not affect the LOAD
section?

Hm, maybe it does because the .note.solo5.xen was considered writable
by default?

Roger.