Re: Xen PVH domU start-of-day VCPU state

[Roger Pau Monné <roger@xxxxxxx>]

On Wed, May 27, 2020 at 04:57:02PM +0200, Martin Lucina wrote:
> On Wednesday, 27.05.2020 at 16:41, Roger Pau Monné wrote:
> > > > > If I make this simple change:
> > > > > 
> > > > > --- a/bindings/xen/boot.S
> > > > > +++ b/bindings/xen/boot.S
> > > > > @@ -32,7 +32,7 @@
> > > > >  #define ENTRY(x) .text; .globl x; .type x,%function; x:
> > > > >  #define END(x)   .size x, . - x
> > > > > 
> > > > > -.section .note.solo5.xen
> > > > > +.section .note.solo5.xen, "a", @note
> > > > > 
> > > > >         .align  4
> > > > >         .long   4
> > > > > 
> > > > > then I get the expected output from readelf -lW, and I can get as far 
> > > > > as
> > > > > the C _start() with no issues!
> > > > > 
> > > > > FWIW, here's the diff of readelf -lW before/after:
> > > > > 
> > > > > --- before    2020-05-26 17:36:46.117885855 +0200
> > > > > +++ after     2020-05-26 17:38:07.090508322 +0200
> > > > > @@ -8,9 +8,9 @@
> > > > >    INTERP         0x001000 0x0000000000100000 0x0000000000100000 
> > > > > 0x000018 0x000018 R   0x8
> > > > >        [Requesting program interpreter: /nonexistent/solo5/]
> > > > >    LOAD           0x001000 0x0000000000100000 0x0000000000100000 
> > > > > 0x00615c 0x00615c R E 0x1000
> > > > > -  LOAD           0x008000 0x0000000000107000 0x0000000000107000 
> > > > > 0x007120 0x00ed28 RW  0x1000
> > > > > +  LOAD           0x008000 0x0000000000107000 0x0000000000107000 
> > > > > 0x006120 0x00dd28 RW  0x1000
> > > > 
> > > > This seems suspicious, there's a change of the size of the LOAD
> > > > section, but your change to the note type should not affect the LOAD
> > > > section?
> > > 
> > > Indeed.
> > 
> > You could try to disassemble the text sections with objdump -d (or -D
> > for all sections) and see if there's a difference between both
> > versions, but having solved the issue maybe you just want to move
> > on.
> 
> I have moved on, making good progress:
> 
>     domainbuilder: detail: xc_dom_release: called
>     Hello, world!
>     Solo5: Xen hvm_start_info @0x0000000000119000
>     Solo5: magic=0x336ec578 version=1
>     Solo5: cmdline_paddr=0x0
>     Solo5: memmap_paddr=0x119878 entries=5
>     Solo5: memmap[0] = { 0x0, 0x10000400, 1 }
>     Solo5: mem_size=0x10000000
>                 |      ___|
>       __|  _ \  |  _ \ __ \
>     \__ \ (   | | (   |  ) |
>     ____/\___/ _|\___/____/
>     Solo5: Bindings version v0.6.5-4-g57724f8-dirty
>     Solo5: Memory map: 256 MB addressable:
>     Solo5:   reserved @ (0x0 - 0xfffff)
>     Solo5:       text @ (0x100000 - 0x104fff)
>     Solo5:     rodata @ (0x105000 - 0x106fff)
>     Solo5:       data @ (0x107000 - 0x118fff)
>     Solo5:       heap >= 0x119000 < stack < 0x10000000
>     Solo5: Clock source: KVM paravirtualized clock
>     Solo5: trap: type=#GP ec=0x0 rip=0x103a96 rsp=0xfffff90 rflags=0x2
>     Solo5: ABORT: cpu_x86_64.c:181: Fatal trap
>     Solo5: Halted
> 
> (The #GP is due to the timekeeping code not yet ported to Xen).
> 
> Random question: With memory="256" in the xl config, why is the size of the
> first XEN_HVM_MEMMAP_TYPE_RAM memmap entry not a multiple of PAGE_SIZE? I
> had to align it down, since we put the stack at the top of RAM. 0x10000400
> seems... odd.

IIRC some memory is stolen by the domain builder to place stuff (like
the ACPI tables), and the end is not aligned to a page size boundary
because it's not mandatory.

Roger.