[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [Minios-devel] [Xen-devel] Building PVH mini-os with libc support



On 28/12/2017 23:49, Bruno Alvisio wrote:

Hello Andrew,

 

Thanks. Yup, you were right. I did an objdump of the kernel and found the offending instruction 0x5cfcb to be callq *%r15 in the console_print function (see below):


Answering out of order...

I am not familiar with the XTF relocalability code. Any pointer or suggestion at this point would be again appreciated.


XTF is the Xen Test Framework, a microkernel project for testing purposes.

http://xenbits.xen.org/gitweb/?p=xtf.git;a=summary
http://xenbits.xen.org/docs/xtf/

Amongst other things, it runs as a set of regression tests for all new code introduced into upstream Xen.

All I meant by that statement was "I recall bugs like this", which is why I made the blind guess at your offending opcode being `jmp *%r15`.

 

0000000000056ee9 <console_print>:

   56ee9:   55                      push   %rbp

   56eea:   48 89 e5                mov    %rsp,%rbp

   56eed:   41 57                   push   %r15

   56eef:   41 56                   push   %r14

   56ef1:   41 55                   push   %r13

   56ef3:   41 54                   push   %r12

   56ef5:   53                      push   %rbx

   56ef6:   48 83 ec 18             sub    $0x18,%rsp

   56efa:   49 89 fe                mov    %rdi,%r14

   56efd:   41 89 d4                mov    %edx,%r12d

   56f00:   48 89 65 c0             mov    %rsp,-0x40(%rbp)

   56f04:   8d 42 01                lea    0x1(%rdx),%eax

   56f07:   48 98                   cltq   

   56f09:   48 83 c0 0f             add    $0xf,%rax

   56f0d:   48 83 e0 f0             and    $0xfffffffffffffff0,%rax

   56f11:   48 29 c4                sub    %rax,%rsp

   56f14:   48 89 e3                mov    %rsp,%rbx

   56f17:   83 3d 00 00 00 00 00    cmpl   $0x0,0x0(%rip)        # 56f1e <console_print+0x35>

   56f1e:   74 09                   je     56f29 <console_print+0x40>

   56f20:   4c 8b 3d 00 00 00 00    mov    0x0(%rip),%r15        # 56f27 <console_print+0x3e>

   56f27:   eb 07                   jmp    56f30 <console_print+0x47>

   56f29:   4c 8b 3d 00 00 00 00    mov    0x0(%rip),%r15        # 56f30 <console_print+0x47>

   56f30:   4d 85 f6                test   %r14,%r14

   56f33:   74 19                   je     56f4e <console_print+0x65>

   56f35:   41 80 7e 30 00          cmpb   $0x0,0x30(%r14)

   56f3a:   74 12                   je     56f4e <console_print+0x65>

   56f3c:   44 89 e2                mov    %r12d,%edx

   56f3f:   4c 89 f7                mov    %r14,%rdi

   56f42:   41 ff d7                callq  *%r15

   56f45:   48 8b 65 c0             mov    -0x40(%rbp),%rsp

   56f49:   e9 84 00 00 00          jmpq   56fd2 <console_print+0xe9>

   56f4e:   4d 63 ec                movslq %r12d,%r13

   56f51:   4c 89 ea                mov    %r13,%rdx

   56f54:   48 89 df                mov    %rbx,%rdi

   56f57:   e8 00 00 00 00          callq  56f5c <console_print+0x73>

   56f5c:   4a 8d 44 2b ff          lea    -0x1(%rbx,%r13,1),%rax

   56f61:   48 39 c3                cmp    %rax,%rbx

   56f64:   73 4b                   jae    56fb1 <console_print+0xc8>

   56f66:   48 89 de                mov    %rbx,%rsi

   56f69:   80 3b 0a                cmpb   $0xa,(%rbx)

   56f6c:   75 30                   jne    56f9e <console_print+0xb5>

   56f6e:   c6 03 0d                movb   $0xd,(%rbx)

   56f71:   0f b6 43 01             movzbl 0x1(%rbx),%eax

   56f75:   88 45 cf                mov    %al,-0x31(%rbp)

   56f78:   c6 43 01 0a             movb   $0xa,0x1(%rbx)

   56f7c:   49 89 dd                mov    %rbx,%r13

   56f7f:   49 29 f5                sub    %rsi,%r13

   56f82:   41 8d 55 02             lea    0x2(%r13),%edx

   56f86:   4c 89 f7                mov    %r14,%rdi

   56f89:   41 ff d7                callq  *%r15

   56f8c:   0f b6 45 cf             movzbl -0x31(%rbp),%eax

   56f90:   88 43 01                mov    %al,0x1(%rbx)

   56f93:   48 8d 73 01             lea    0x1(%rbx),%rsi

   56f97:   41 83 c5 01             add    $0x1,%r13d

   56f9b:   45 29 ec                sub    %r13d,%r12d

   56f9e:   48 83 c3 01             add    $0x1,%rbx

   56fa2:   4d 63 ec                movslq %r12d,%r13

   56fa5:   4a 8d 44 2e ff          lea    -0x1(%rsi,%r13,1),%rax

   56faa:   48 39 d8                cmp    %rbx,%rax

   56fad:   77 ba                   ja     56f69 <console_print+0x80>

   56faf:   eb 03                   jmp    56fb4 <console_print+0xcb>

   56fb1:   48 89 de                mov    %rbx,%rsi

   56fb4:   80 38 0a                cmpb   $0xa,(%rax)

   56fb7:   75 0c                   jne    56fc5 <console_print+0xdc>

   56fb9:   c6 00 0d                movb   $0xd,(%rax)

   56fbc:   42 c6 04 2e 0a          movb   $0xa,(%rsi,%r13,1)

   56fc1:   41 83 c4 01             add    $0x1,%r12d

   56fc5:   44 89 e2                mov    %r12d,%edx

   56fc8:   4c 89 f7                mov    %r14,%rdi

   56fcb:   41 ff d7                callq  *%r15

   56fce:   48 8b 65 c0             mov    -0x40(%rbp),%rsp

   56fd2:   48 8d 65 d8             lea    -0x28(%rbp),%rsp

   56fd6:   5b                      pop    %rbx

   56fd7:   41 5c                   pop    %r12

   56fd9:   41 5d                pop    %r13

   56fdb:   41 5e                   pop    %r14

   56fdd:   41 5f                   pop    %r15

   56fdf:   5d                      pop    %rbp

   56fe0:   c3                      retq

 

 

I added the options –fno-pic to TARGET_CFLAGS and TARGET_CPPFLAGS and –no-pie to TARGET_LDFLAGS and recompile the kernel. Now, the kernel carshed at instruction 0x673bb in the run_idle_thread function:

 

00000000000673ac <run_idle_thread>:

   673ac:   55                      push   %rbp

   673ad:   48 89 e5                mov    %rsp,%rbp

   673b0:   48 8b 05 00 00 00 00    mov    0x0(%rip),%rax        # 673b7 <run_idle_thread+0xb>

   673b7:   48 8b 60 10             mov    0x10(%rax),%rsp

   673bb:   ff 70 18                pushq  0x18(%rax)

   673be:   c3                      retq

   673bf:   5d                      pop    %rbp

   673c0:   c3                      retq

 

 

Finally, just to try I commented out the run_idle_thread function and the kernel crashed at the very beginning at 0x63. The kern dump in this case points to the stack:

 

      5e:   e8 00 00 00 00          callq  63 <stack_start>


Both the above mov instruction at 0x673b0 and this call instruction with a 4-byte displacement of 0 look suspiciously like they are waiting for relocation, as displacements of 0 are exceedingly rare (there are more efficient ways to encode such operands).

Therefore, I don't think you've succeeded in preventing your binary from being relocatable.

~Andrew

 

0000000000000063 <stack_start>:

    ...

      6b:   90                      nop

      6c:   90                      nop

      6d:   90                      nop

 

 

Thanks,

Bruno


On Thu, Dec 28, 2017 at 7:18 PM, Andrew Cooper <andrew.cooper3@xxxxxxxxxx> wrote:
On 28/12/17 18:33, Bruno Alvisio wrote:
(d360) Bootstrapping...

(XEN) Dom360 callback via changed to Direct Vector 0x20

(d360) Xen Minimal OS (hvm)!

(XEN) d360v0 Triple fault - invoking HVM shutdown action 1

(XEN) *** Dumping Dom360 vcpu#0 state: ***

(XEN) ----[ Xen-4.10.0-rc  x86_64  debug=y   Not tainted ]----

(XEN) CPU:    7

(XEN) RIP:    0008:[<0000000000056fc8>]

(XEN) RFLAGS: 0000000000010006   CONTEXT: hvm guest (d360v0)

(XEN) rax: 00000000000bfe75   rbx: 00000000000bfe75   rcx: 0000000000000000

(XEN) rdx: 0000000000000017   rsi: 00000000000bfe60   rdi: 0000000000000000

(XEN) rbp: 00000000000bfec0   rsp: 00000000000bfe60   r8:  0000000000000000

(XEN) r9:  0000000000089982   r10: 0000000000000016   r11: 0000000000000000

(XEN) r12: 0000000000000017   r13: 0000000000000016   r14: 0000000000000000

(XEN) r15: 0d8b4c1575ff8548   cr0: 0000000080000011   cr4: 0000000000000220

(XEN) cr3: 0000000000099000   cr2: 0000000000000000

(XEN) fsb: 0000000000000000   gsb: 0000000000000000   gss: 0000000000000000

(XEN) ds: 0033   es: 0033   fs: 0033   gs: 0033   ss: 0000   cs: 0008

 

 

Any help on this would be greatly appreciated.


You will need to disassemble your minios kernel and see which instruction is at 0x56fc8.  (Chances are, it will be `jmp %r15`).

The content of %r15 looks like x86 opcode, which is reminiscent of the XTF relocatability bugs.  Make doubly sure you are compiling with -fno-pic and link with -no-pie.

~Andrew


_______________________________________________
Minios-devel mailing list
Minios-devel@xxxxxxxxxxxxxxxxxxxx
https://lists.xenproject.org/mailman/listinfo/minios-devel

 


Rackspace

Lists.xenproject.org is hosted with RackSpace, monitoring our
servers 24x7x365 and backed by RackSpace's Fanatical Support®.