[Xen-devel] FreeBSD boot loader on VT-x based Xen system

[Randy Thelen <rthelen@xxxxxxxxxx>]

Folks --

I'm trying to run FreeBSD 6.0 on a VT-x based Xen system.  I'm  
experiencing a problem where the boot loader is crashing before the  
FreeBSD kernel (proper) is loaded.
The sequence of operations is that the VMX domain is constructed, the  
BIOS is executed, GRUB loads, I type the instructions to load FreeBSD  
(a boot loader) and that code begins to execute and then crashes.
Here's a console log that we captured:

> GNU GRUB  version 0.95  (638K lower / 261112K upper memory)
> [ Minimal BASH-like line editing is supported.  For the first word,  
> TAB
> lists possible command completions.  Anywhere else TAB lists the  
> possible
> completions of a device/filename.]
>
> grub> root (hd0,1,a)
>     Filesystem type is ufs2, partition type 0xa5
>
> grub> kernel /boot/loader
> [FreeBSD-a.out, loadaddr=0x200000, text=0x1000, data=0x32000,  
> bss=0x0, entry =0x200000]
> grub> boot
> BTX loader 1.00  Starting in protected mode (base mem=9fc00)
> Arguments passed (esp=67e88):
> <howto=80000000 bootdev=a0300000 junk=0 0 0 bootinfo=67ec4>
> Relocated bootinfo (size=48) to 9fbb8
> Relocated arguments (size=18) to 9fba0
> BTX version is 1.01
> Relocated kernel (size=780) to 9000
> Client base address is a000
> Client format is ELF
> text segment: offset=1000 vaddr=0 filesz=2a3b0 memsz=2a3b0
> data segment: offset=2c000 vaddr=2c000 filesz=53cc memsz=b4c0
> Loading complete
>
> int=0000000d  err=00000000  efl=00010006  eip=000090ac
> eax=00094c00  ebx=00202820  ecx=00000033  edx=0000a000
> esi=00209701  edi=00001f98  ebp=0000a000  esp=000017f0
> cs=0008  ds=0000  es=0000    fs=0020  gs=0020  ss=0010
> cs:eip=ff 35 0c 90 00 00 51 51-51 51 52 b1 07 6a 00 e2
>        fc 61 07 1f 0f a1 0f a9-cf fa bc 00 18 00 00 0f
> ss:esp=2b 00 00 00 02 02 00 00-00 4c 09 00 33 00 00 00
>        0a 69 6e 74 3d 30 30 30-30 30 30 30 64 20 20 65
> BTX halted
The instructions that were being executed can be found, in a FreeBSD  
source tree, at:
/usr/src/sys/boot/i386/btx/btx/btx.S

Here they are for your perusal:

                .code32
init.8:         xorl %ecx,%ecx                  # Zero
                movb $SEL_SDATA,%cl             # To 32-bit
                movw %cx,%ss                    #  stack
/*
 * Launch user task.
 */
                movb $SEL_TSS,%cl               # Set task
                ltr %cx                         #  register
                movl $MEM_USR,%edx              # User base address
                movzwl %ss:BDA_MEM,%eax         # Get free memory
                shll $0xa,%eax                  # To bytes
                subl $0x1000,%eax               # Less arg space
                subl %edx,%eax                  # Less base
                movb $SEL_UDATA,%cl             # User data selector
                pushl %ecx                      # Set SS
                pushl %eax                      # Set ESP
                push $0x202                     # Set flags (IF set)
                push $SEL_UCODE                 # Set CS
                pushl btx_hdr+0xc               # Set EIP

The error occurs with the last instruction shown above: pushl btx_hdr 
+0xc.
Here's a disassembly of those instructions:

0000907f (02) 31c9                     XOR ECX, ECX
00009081 (02) b1 10                    MOV CL, 0x10
00009083 (02) 8ed1                     MOV SS, CX
00009085 (02) b1 38                    MOV CL, 0x38
00009087 (03) 0f00d9                   LTR CX
0000908a (05) ba 00a00000              MOV EDX, 0xa000
0000908f (08) 36 0fb705 13040000       MOVZX EAX, SS:[0x413]
00009097 (03) c1e0 0a                  SHL EAX, 0xa
0000909a (05) 2d 00100000              SUB EAX, 0x1000
0000909f (02) 29d0                     SUB EAX, EDX
000090a1 (02) b1 33                    MOV CL, 0x33
000090a3 (01) 51                       PUSH ECX
000090a4 (01) 50                       PUSH EAX
000090a5 (05) 68 02020000              PUSH 0x202
000090aa (02) 6a 2b                    PUSH 0x2b
000090ac (06) ff35 0c000000            PUSH DWORD [0xc]

And, we can see that the instruction stream at 90ac matches the  
instruction stream in the eip dump in the exception display.  (Note  
that I'm disassembling the btx.o file, and so the linker has not  
fixed up the reference to btx_hdr+0xc to be 900c, instead we're  
seeing just 0xc.  Note too, that this explains why I believe I can  
replace the reference to btx_hdr+0xc with an explicit reference to  
the code segment register; see below.)
My guess is that there's a bug in VMXAssist.  I'd like some  
corroboration of that point, if possible.  In fact, because the code  
had long jumped to the above code sequence to enter 32bit mode, are  
we still in VMXAssist?  Or have we entered into pure execution mode?
It appears that the problem is based on an incorrect data segment  
register value.  The code segment register contains 8, which points  
to a valid code segment descriptor.  But, the ds register is 0, and  
that's never good.  How or why this code works on real hardware is  
undoubtedly a side effect of ... I dunno a bug, legacy, or a design  
that's not clear to me.  At any rate, If I replace the push  
instruction with:
                pushl %cs:btx_hdr+0xc

referencing the code segment selector, then we get past the exception  
noted above and into our next problem.  I'm just curious about this  
particular bug.  Can somebody lend some insight or direction in  
tracking this down?  I'm more than happy to do lots of leg work on  
tracking this down, but a little direction would go a long way to help.
-- Randy

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-devel