Xen project Mailing List

Re: [Xen-users] Trouble booting FreeBSD i386 PV DomU

To: tech mailinglists <mailinglists.tech@xxxxxxxxx>

From: Roger Pau Monné <roger.pau@xxxxxxxxxx>

Date: Tue, 26 Mar 2013 10:31:50 +0100

Cc: xen-users <xen-users@xxxxxxxxxxxxx>, "freebsd-xen@xxxxxxxxxxx" <freebsd-xen@xxxxxxxxxxx>

Delivery-date: Tue, 26 Mar 2013 09:33:02 +0000

List-id: Xen user discussion <xen-users.lists.xen.org>

On 21/02/13 10:18, tech mailinglists wrote: > Hello all, > > I have created a FreeBSD PV DomU image formatted with ZFS. I compiled > FreeBSD with KERNCONF=XEN for the kernel and the normal world and > distribution target. Then I transfered it to a Debian Dom0 with Xen 4.2.1. > > I tried to boot the image befor I migrate it to LVM and I now get teh > following output: > > Parsing config from freebsd-test.cfg > libxl: error: libxl_dm.c:1212:device_model_spawn_outcome: domain 12 > device model: spawn failed (rc=-3) > libxl: error: libxl_qmp.c:641:libxl__qmp_initialize: Connection error: > No such file or directory > Daemon running with PID 3451 > WARNING: loader(8) metadata is missing! > GDB: no debug ports present > KDB: debugger backends: ddb > KDB: current backend: ddb > APIC: Using the MPTable enumerator. > SMP: Added CPU 0 (BSP) > Copyright (c) 1992-2012 The FreeBSD Project. > Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994 > The Regents of the University of California. All rights reserved. > FreeBSD is a registered trademark of The FreeBSD Foundation. > FreeBSD 9.1-RELEASE #0: Wed Feb 20 14:16:03 CET 2013 > root@build:/usr/obj/usr/src/sys/XEN i386 > WARNING: WITNESS option enabled, expect reduced performance. > Xen reported: 3341.754 MHz processor. > Timecounter "ixen" frequency 1953125 Hz quality 0 > CPU: Intel(R) Core(TM) i7 CPU 975 @ 3.33GHz (3341.75-MHz > 686-class CPU) > Origin = "GenuineIntel" Id = 0x106a5 Family = 6 Model = 1a > Stepping = 5 > > Features=0xbfe3fbff<FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,DTS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE> > > Features2=0x98e3bd<SSE3,DTES64,MON,DS_CPL,VMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,SSE4.1,SSE4.2,POPCNT> > AMD Features=0x28100000<NX,RDTSCP,LM> > AMD Features2=0x1<LAHF> > > Data TLB: 4 KB pages, 4-way set associative, 64 entries > 1st-level data cache: 32 KB, 8-way set associative, 64 byte line size > L2 cache: 256 kbytes, 8-way associative, 64 bytes/line > real memory = 2147483648 (2048 MB) > Physical memory chunk(s): > 0x0000000000a58000 - 0x000000007d995fff, 2096357376 bytes (511806 pages) > avail memory = 2092105728 (1995 MB) > INTR: Adding local APIC 0 as a target > ULE: setup cpu 0 > [XEN] IPI cpu=0 irq=128 vector=RESCHEDULE_VECTOR (0) > [XEN] IPI cpu=0 irq=129 vector=CALL_FUNCTION_VECTOR (1) > Event-channel device installed. > io: <I/O> > random: <entropy source, Software, Yarrow> > mem: <memory> > Pentium Pro MTRR support enabled > null: <null device, zero device> > nfslock: pseudo-device > [XEN] xen_rtc_probe: probing Hypervisor RTC clock > rtc0: <Xen Hypervisor Clock> on motherboard > [XEN] xen_rtc_attach: attaching Hypervisor RTC clock > rtc0: registered as a time-of-day clock (resolution 1000000us, > adjustment 0.500000000s) > xenstore0: <XenStore> on motherboard > Grant table initialized > xc0: <Xen Console> on motherboard > Device configuration finished. > procfs registered > Event timer "ixen" quality 600 > Timecounters tick every 10.000 msec > lo0: bpf attached > xenbusb_front0: <Xen Frontend Devices> on xenstore0 > xenbusb_add_device: Device device/suspend/event-channel ignored. State 6 > xenbusb_back0: <Xen Backend Devices> on xenstore0 > xctrl0: <Xen Control Device> on xenstore0 > [XEN] hypervisor wallclock nudged; nudging TOD. > run_interrupt_driven_hooks: still waiting after 60 seconds for > xenbus_free_evtchn > run_interrupt_driven_hooks: still waiting after 120 seconds for > xenbus_free_evtchn > run_interrupt_driven_hooks: still waiting after 180 seconds for > xenbus_free_evtchn > run_interrupt_driven_hooks: still waiting after 240 seconds for > xenbus_free_evtchn > run_interrupt_driven_hooks: still waiting after 300 seconds for > xenbus_free_evtchn > panic: run_interrupt_driven_config_hooks: waited too long > cpuid = 0 > KDB: enter: panic > [ thread pid 0 tid 100000 ] > Stopped at kdb_enter+0x3a: movl $0,kdb_why > db> Hello, I've been trying to get a FreeBSD PV guest, I've installed FreeBSD i386 HEAD and then tried to recompile the kernel using KERNCONF=XEN, but Xen refuses to load the resulting kernel: root@loki:~# xl -vvv create -c freebsd32pv.cfg Parsing config from freebsd32pv.cfg libxl: debug: libxl_create.c:1174:do_domain_create: ao 0x22e5a20: create: how=(nil) callback=(nil) poller=0x22e5a80 libxl: debug: libxl_device.c:229:libxl__device_disk_set_backend: Disk vdev=hda spec.backend=unknown libxl: debug: libxl_device.c:265:libxl__device_disk_set_backend: Disk vdev=hda, using backend phy libxl: debug: libxl_create.c:677:initiate_domain_create: running bootloader libxl: debug: libxl_bootloader.c:327:libxl__bootloader_run: no bootloader configured, using user supplied kernel libxl: debug: libxl_event.c:561:libxl__ev_xswatch_deregister: watch w=0x22e60c0: deregister unregistered libxl: debug: libxl_numa.c:435:libxl__get_numa_candidate: New best NUMA placement candidate found: nr_nodes=1, nr_cpus=8, nr_vcpus=15, free_memkb=4353 libxl: detail: libxl_dom.c:192:numa_place_domain: NUMA placement candidate with 1 nodes, 8 cpus and 4353 KB free selected domainbuilder: detail: xc_dom_allocate: cmdline="boot_verbose=1,vfs.root.mountfrom=ufs:/dev/ad0s1a,kern.hz=100", features="(null)" libxl: debug: libxl_dom.c:380:libxl__build_pv: pv kernel mapped 0 path /root/kernel.freebsd domainbuilder: detail: xc_dom_kernel_file: filename="/root/kernel.freebsd" domainbuilder: detail: xc_dom_malloc_filemap : 5235 kB domainbuilder: detail: xc_dom_boot_xen_init: ver 4.2, caps xen-3.0-x86_64 xen-3.0-x86_32p hvm-3.0-x86_32 hvm-3.0-x86_32p hvm-3.0-x86_64 domainbuilder: detail: xc_dom_parse_image: called domainbuilder: detail: xc_dom_find_loader: trying ELF-generic loader ... domainbuilder: detail: loader probe OK xc: detail: elf_parse_binary: phdr: paddr=0xc0000000 memsz=0x439eb8 xc: detail: elf_parse_binary: phdr: paddr=0xc043a000 memsz=0x23a578 xc: detail: elf_parse_binary: memory: 0xc0000000 -> 0xc0674578 xc: detail: elf_xen_parse: __xen_guest: "LOADER=generic,GUEST_OS=freebsd,GUEST_VER=7.0,XEN_VER=xen-3.0,BSD_SYMTAB,VIRT_BASE=0xc0000000" xc: detail: elf_xen_parse_guest_info: LOADER="generic" xc: detail: elf_xen_parse_guest_info: GUEST_OS="freebsd" xc: detail: elf_xen_parse_guest_info: GUEST_VER="7.0" xc: detail: elf_xen_parse_guest_info: XEN_VER="xen-3.0" xc: detail: elf_xen_parse_guest_info: BSD_SYMTAB="" xc: detail: elf_xen_parse_guest_info: VIRT_BASE="0xc0000000" xc: detail: elf_xen_addr_calc_check: ELF_PADDR_OFFSET unset, using 0xc0000000 xc: detail: elf_xen_addr_calc_check: addresses: xc: detail: virt_base = 0xc0000000 xc: detail: elf_paddr_offset = 0xc0000000 xc: detail: virt_offset = 0x0 xc: detail: virt_kstart = 0xc0000000 xc: detail: virt_kend = 0xc07439d0 xc: detail: virt_entry = 0xc0046000 xc: detail: p2m_base = 0xffffffffffffffff domainbuilder: detail: xc_dom_load_elf_symtab/parse: bsd_symtab_start=c07439d0, kernel.end=0xc07439d0 -- symtab=0xc07439d4, maxaddr=0xc0743e40 domainbuilder: detail: xc_dom_load_elf_symtab: h=25 symtab, size=0x4a310, maxaddr=0xc078e150 domainbuilder: detail: xc_dom_load_elf_symtab: h=26 strtab, size=0x69500, maxaddr=0xc07f7650 domainbuilder: detail: xc_dom_parse_elf_kernel: xen-3.0-x86_32: 0xc0000000 -> 0xc07f7650 domainbuilder: detail: xc_dom_mem_init: mem 512 MB, pages 0x20000 pages, 4k each domainbuilder: detail: xc_dom_mem_init: 0x20000 pages domainbuilder: detail: xc_dom_boot_mem_init: called domainbuilder: detail: xc_dom_malloc : 1024 kB domainbuilder: detail: xc_dom_build_image: called domainbuilder: detail: xc_dom_alloc_segment: kernel : 0xc0000000 -> 0xc07f8000 (pfn 0x0 + 0x7f8 pages) domainbuilder: detail: xc_dom_pfn_to_ptr: domU mapping: pfn 0x0+0x7f8 at 0x7f88fc22c000 xc: detail: elf_load_binary: phdr 2 at 0x0x7f88fc22c000 -> 0x0x7f88fc665eb8 xc: detail: elf_load_binary: phdr 3 at 0x0x7f88fc666000 -> 0x0x7f88fc694c9c xc: detail: elf_load_bsdsyms: shdr 4 at 0x0x7f88fca4bd84 -> 0x0x7f88fc8a09e8 xc: detail: elf_load_bsdsyms: shdr 24 at 0x0x7f88fce8cec4 -> 0x0x7f88fc8bc0a4 xc: detail: elf_load_bsdsyms: shdr 25 at 0x0x7f88fce8d418 -> 0x0x7f88fc8bc1c0 xc: detail: elf_load_bsdsyms: shdr 26 at 0x0x7f88fced7728 -> 0x0x7f88fc9064d0 domainbuilder: detail: xc_dom_load_elf_symtab/load: bsd_symtab_start=c07439d0, kernel.end=0xc07f8000 -- symtab=0xc07439d4, maxaddr=0xc0743e40 domainbuilder: detail: xc_dom_load_elf_symtab: h=25 symtab, size=0x4a310, maxaddr=0xc078e150 domainbuilder: detail: xc_dom_load_elf_symtab: h=26 strtab, size=0x69500, maxaddr=0xc07f7650 domainbuilder: detail: xc_dom_alloc_segment: phys2mach : 0xc07f8000 -> 0xc0878000 (pfn 0x7f8 + 0x80 pages) domainbuilder: detail: xc_dom_pfn_to_ptr: domU mapping: pfn 0x7f8+0x80 at 0x7f88fec35000 domainbuilder: detail: xc_dom_alloc_page : start info : 0xc0878000 (pfn 0x878) domainbuilder: detail: xc_dom_alloc_page : xenstore : 0xc0879000 (pfn 0x879) domainbuilder: detail: xc_dom_alloc_page : console : 0xc087a000 (pfn 0x87a) domainbuilder: detail: nr_page_tables: 0x00000000ffffffff/32: 0x0000000000000000 -> 0x00000000ffffffff, 1 table(s) domainbuilder: detail: nr_page_tables: 0x00000000003fffff/22: 0x00000000c0000000 -> 0x00000000c0bfffff, 3 table(s) domainbuilder: detail: xc_dom_alloc_segment: page tables : 0xc087b000 -> 0xc087f000 (pfn 0x87b + 0x4 pages) domainbuilder: detail: xc_dom_pfn_to_ptr: domU mapping: pfn 0x87b+0x4 at 0x7f88fedbe000 domainbuilder: detail: xc_dom_alloc_page : boot stack : 0xc087f000 (pfn 0x87f) domainbuilder: detail: xc_dom_build_image : virt_alloc_end : 0xc0880000 domainbuilder: detail: xc_dom_build_image : virt_pgtab_end : 0xc0c00000 domainbuilder: detail: xc_dom_boot_image: called domainbuilder: detail: arch_setup_bootearly: doing nothing domainbuilder: detail: xc_dom_compat_check: supported guest type: xen-3.0-x86_64 domainbuilder: detail: xc_dom_compat_check: supported guest type: xen-3.0-x86_32p domainbuilder: detail: xc_dom_compat_check: supported guest type: hvm-3.0-x86_32 domainbuilder: detail: xc_dom_compat_check: supported guest type: hvm-3.0-x86_32p domainbuilder: detail: xc_dom_compat_check: supported guest type: hvm-3.0-x86_64 xc: error: panic: xc_dom_boot.c:127: xc_dom_compat_check: guest type xen-3.0-x86_32 not supported by xen kernel, sorry: Invalid kernel It refuses to boot because it detects the kernel doesn't support PAE. I've hacked sys/i386/xen/locore.s in order to add the PAE flag: .ascii "LOADER=generic,GUEST_OS=freebsd,GUEST_VER=7.0,XEN_VER=xen-3.0,BSD_SYMTAB,VIRT_BASE=0xc0000000,PAE=yes[extended-cr3]" I know there's an elfnote that should tell Xen if kernel support PAE or not, but it seems to be ignored: ELFNOTE(Xen, XEN_ELFNOTE_PAE_MODE, .asciz, "yes") Anyway, I've recompiled the kernel with the PAE flag hack, but then when trying to load the resulting kernel it crashes very early: (XEN) d12:v0: unhandled page fault (ec=0002) (XEN) Pagetable walk from 0000000000000100: (XEN) L4[0x000] = 0000000141710027 000000000000087b (XEN) L3[0x000] = 0000000000000000 ffffffffffffffff (XEN) domain_crash_sync called from entry.S (XEN) Domain 12 (vcpu#0) crashed on cpu#7: (XEN) ----[ Xen-4.2.1 x86_64 debug=n Not tainted ]---- (XEN) CPU: 7 (XEN) RIP: e019:[<00000000c00452a0>] (XEN) RFLAGS: 0000000000000246 EM: 1 CONTEXT: pv guest (XEN) rax: 0000000000000100 rbx: 0000000000000000 rcx: 0000000000000000 (XEN) rdx: 00000000c0878000 rsi: 0000000000000000 rdi: 0000000000000000 (XEN) rbp: 00000000c043e12c rsp: 00000000c043e07c r8: 0000000000000000 (XEN) r9: 0000000000000000 r10: 0000000000000000 r11: 0000000000000000 (XEN) r12: 0000000000000000 r13: 0000000000000000 r14: 0000000000000000 (XEN) r15: 0000000000000000 cr0: 000000008005003b cr4: 00000000000026f0 (XEN) cr3: 000000013fb58000 cr2: 0000000000000100 (XEN) ds: e021 es: e021 fs: e021 gs: e021 ss: e021 cs: e019 (XEN) Guest stack trace from esp=c043e07c: (XEN) 00000002 c00452a0 0001e019 00010046 c03adc1f 00000000 00000000 00000000 (XEN) 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 (XEN) 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 (XEN) 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 (XEN) 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 (XEN) 00000000 c0878000 00000000 c0884000 00000000 c0046015 c0878000 c043e138 (XEN) 0000e021 00000040 00000020 c0468d0c 01800000 00000001 c014ffe0 c043e2c4 (XEN) 01800000 0fffffff c01500b0 c043e2c4 01b00000 01000000 c016ef90 c043e2d4 (XEN) 01b00000 01000000 c016f0b0 c0647994 c0647a3c 00000000 ffffffff c0000001 (XEN) c0469810 00000000 c03db795 00000000 c03ecf36 00000000 00000000 c03db81e (XEN) c0469810 00000000 ffffffff 80040006 c0469804 00000000 c03db7f3 c0172990 (XEN) c03db706 00000000 00000000 c03db7fa c0469810 00000000 ffffffff 80040006 (XEN) c043e2c0 00000000 c03db786 c0172990 c03db706 00000000 00000000 c03db7c7 (XEN) c0469810 00000000 ffffffff 80040002 c04697fc 00000000 c03db79d c0172990 (XEN) c03de14d 00000000 00000000 c03db7a8 02380000 0fffffff c004a6b0 00000000 (XEN) c0469810 00000000 ffffffff c0000006 00000000 00000000 c03db643 c004a520 (XEN) c03db706 00000000 00000000 c03db709 c0469810 00000000 ffffffff 80000003 (XEN) 00000000 00000000 c03fb8f6 c004a470 c0433acf 00000000 00000000 c03db6d7 (XEN) 0000c000 00500000 00000000 3454f8e9 c03db791 00000000 c0647994 c03db832 (XEN) 00000000 c0436966 c004baa0 00000000 00000000 00000000 00000000 c0408707 Is Xen i386 PV broken? Or I'm being incredebly stupid and missing something obvious? _______________________________________________ Xen-users mailing list Xen-users@xxxxxxxxxxxxx http://lists.xen.org/xen-users

©2013 Xen Project, A Linux Foundation Collaborative Project. All Rights Reserved.
Linux Foundation is a registered trademark of The Linux Foundation.
Xen Project is a trademark of The Linux Foundation.