[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [Xen-devel] libxl: problem with devices in PV



On Wed, 20 Jul 2011, Roger Pau Monnà wrote:
> Hello,
> 
> I'm trying to run PV machines using the new libxenlight toolstack on
> NetBSD. So far, I've been able to connect to the domu using the
> console (I was unable to do so before). I'm attaching a little patch
> that removes setting the consback to IOEMU when detecting a qdisk
> (that was preventing the domu from even booting). With the
> introduction of libxenlight, Xen doesn't use vbd for disk backend
> anymore, it uses qdisk, which I assume Qemu automatically attaches to
> running guests. It works fine with HVM guests, but it seems to fail
> with PV guests. The config file I'm using is:
> 

Xen uses qdisk only when blktap is not available.
I suggest you install blktap if you can because it is significantly
faster than qdisk at the moment.
Otherwise if you use a physical partition and specify phy: in the config
file you should get the kernel based blkback that is the fastest option
available.


> bootloader = '/usr/xen42/bin/pygrub'
> vcpus       = '1'
> memory      = '30720'
> 
> root        = '/dev/xvda2 ro'
> disk        = [
>                 'file:/home/xen/debian/disk.img,xvda2,w',
>                 'file:/home/xen/debian/swap.img,xvda1,w',
>               ]
> 
> name        = 'debian'
> 
> on_poweroff = 'destroy'
> on_reboot   = 'restart'
> on_crash    = 'restart'
> 
> And the output from booting:
> 
> [    0.000000] Initializing cgroup subsys cpuset
> [    0.000000] Initializing cgroup subsys cpu
> [    0.000000] Linux version 2.6.32-5-xen-amd64 (Debian
> 2.6.32-34squeeze1) (dannf@xxxxxxxxxx) (gcc version 4.3.5 (Debian
> 4.3.5-4) ) #1 SMP Thu May 19 01:16:47 UTC 2011
> [    0.000000] Command line: root=/dev/xvda2 ro root=/dev/xvda2 ro
> [    0.000000] KERNEL supported cpus:
> [    0.000000]   Intel GenuineIntel
> [    0.000000]   AMD AuthenticAMD
> [    0.000000]   Centaur CentaurHauls
> [    0.000000] ACPI in unprivileged domain disabled
> [    0.000000] released 0 pages of unused memory
> [    0.000000] BIOS-provided physical RAM map:
> [    0.000000]  Xen: 0000000000000000 - 00000000000a0000 (usable)
> [    0.000000]  Xen: 00000000000a0000 - 0000000000100000 (reserved)
> [    0.000000]  Xen: 0000000000100000 - 0000000780000000 (usable)
> [    0.000000] DMI not present or invalid.
> [    0.000000] last_pfn = 0x780000 max_arch_pfn = 0x400000000
> [    0.000000] last_pfn = 0x100000 max_arch_pfn = 0x400000000
> [    0.000000] init_memory_mapping: 0000000000000000-0000000100000000
> [    0.000000] init_memory_mapping: 0000000100000000-0000000780000000
> [    0.000000] RAMDISK: 016ba000 - 02feb000
> [    0.000000] No NUMA configuration found
> [    0.000000] Faking a node at 0000000000000000-0000000780000000
> [    0.000000] Bootmem setup node 0 0000000000000000-0000000780000000
> [    0.000000]   NODE_DATA [0000000000008000 - 000000000000ffff]
> [    0.000000]   bootmap [00000000008c7000 -  00000000009b6fff] pages f0
> [    0.000000] (8 early reservations) ==> bootmem [0000000000 - 0780000000]
> [    0.000000]   #0 [0000000000 - 0000001000]   BIOS data page ==>
> [0000000000 - 0000001000]
> [    0.000000]   #1 [0006bee000 - 0006c29000]   XEN PAGETABLES ==>
> [0006bee000 - 0006c29000]
> [    0.000000]   #2 [0000006000 - 0000008000]       TRAMPOLINE ==>
> [0000006000 - 0000008000]
> [    0.000000]   #3 [0001000000 - 00016999d4]    TEXT DATA BSS ==>
> [0001000000 - 00016999d4]
> [    0.000000]   #4 [00016ba000 - 0002feb000]          RAMDISK ==>
> [00016ba000 - 0002feb000]
> [    0.000000]   #5 [0002feb000 - 0006bee000]   XEN START INFO ==>
> [0002feb000 - 0006bee000]
> [    0.000000]   #6 [0000100000 - 00008c7000]          PGTABLE ==>
> [0000100000 - 00008c7000]
> [    0.000000]   #7 [0006c29000 - 000a043000]          PGTABLE ==>
> [0006c29000 - 000a043000]
> [    0.000000] Zone PFN ranges:
> [    0.000000]   DMA      0x00000000 -> 0x00001000
> [    0.000000]   DMA32    0x00001000 -> 0x00100000
> [    0.000000]   Normal   0x00100000 -> 0x00780000
> [    0.000000] Movable zone start PFN for each node
> [    0.000000] early_node_map[2] active PFN ranges
> [    0.000000]     0: 0x00000000 -> 0x000000a0
> [    0.000000]     0: 0x00000100 -> 0x00780000
> [    0.000000] SFI: Simple Firmware Interface v0.7 http://simplefirmware.org
> [    0.000000] SMP: Allowing 1 CPUs, 0 hotplug CPUs
> [    0.000000] No local APIC present
> [    0.000000] APIC: disable apic facility
> [    0.000000] PM: Registered nosave memory: 00000000000a0000 - 
> 0000000000100000
> [    0.000000] PCI: Warning: Cannot find a gap in the 32bit address range
> [    0.000000] PCI: Unassigned devices with 32bit resource registers may 
> break!
> [    0.000000] Allocating PCI resources starting at 780100000 (gap:
> 780100000:400000)
> [    0.000000] Booting paravirtualized kernel on Xen
> [    0.000000] Xen version: 4.2-unstable (preserve-AD)
> [    0.000000] NR_CPUS:512 nr_cpumask_bits:512 nr_cpu_ids:1 nr_node_ids:1
> [    0.000000] PERCPU: Embedded 30 pages/cpu @ffff88002804f000 s90328
> r8192 d24360 u122880
> [    0.000000] pcpu-alloc: s90328 r8192 d24360 u122880 alloc=30*4096
> [    0.000000] pcpu-alloc: [0] 0
> [    0.000000] Xen: using vcpu_info placement
> [    0.000000] Built 1 zonelists in Zone order, mobility grouping on.
> Total pages: 7754710
> [    0.000000] Policy zone: Normal
> [    0.000000] Kernel command line: root=/dev/xvda2 ro root=/dev/xvda2 ro
> [    0.000000] PID hash table entries: 4096 (order: 3, 32768 bytes)
> [    0.000000] Initializing CPU#0
> [    0.000000] Checking aperture...
> [    0.000000] No AGP bridge found
> [    0.000000] Memory: 30803904k/31457280k available (3149k kernel
> code, 384k absent, 652992k reserved, 1907k data, 604k init)
> [    0.000000] SLUB: Genslabs=14, HWalign=64, Order=0-3, MinObjects=0,
> CPUs=1, Nodes=1
> [    0.000000] Hierarchical RCU implementation.
> [    0.000000] NR_IRQS:4352 nr_irqs:512
> [    0.000000] Console: colour dummy device 80x25
> [    0.000000] console [tty0] enabled
> [    0.000000] console [hvc0] enabled
> [    0.000000] installing Xen timer for CPU 0
> [    0.000000] Detected 2394.046 MHz processor.
> [    0.004000] Calibrating delay loop (skipped), value calculated
> using timer frequency.. 4788.09 BogoMIPS (lpj=9576184)
> [    0.004000] Security Framework initialized
> [    0.004000] SELinux:  Disabled at boot.
> [    0.005680] Dentry cache hash table entries: 4194304 (order: 13,
> 33554432 bytes)
> [    0.014625] Inode-cache hash table entries: 2097152 (order: 12,
> 16777216 bytes)
> [    0.017597] Mount-cache hash table entries: 256
> [    0.017761] Initializing cgroup subsys ns
> [    0.017769] Initializing cgroup subsys cpuacct
> [    0.017776] Initializing cgroup subsys devices
> [    0.017782] Initializing cgroup subsys freezer
> [    0.017786] Initializing cgroup subsys net_cls
> [    0.017827] CPU: L1 I cache: 32K, L1 D cache: 32K
> [    0.017835] CPU: L2 cache: 256K
> [    0.017838] CPU: L3 cache: 12288K
> [    0.017844] CPU 0/0x20 -> Node 0
> [    0.017848] CPU: Unsupported number of siblings 32
> [    0.017854] Performance Events: unsupported p6 CPU model 44 no PMU
> driver, software events only.
> [    0.017874] SMP alternatives: switching to UP code
> [    0.020000] Freeing SMP alternatives: 28k freed
> [    0.020192] Brought up 1 CPUs
> [    0.020309] devtmpfs: initialized
> [    0.024342] Grant table initialized
> [    0.024351] regulator: core version 0.5
> [    0.024403] NET: Registered protocol family 16
> [    0.025064] PCI: setting up Xen PCI frontend stub
> [    0.026729] bio: create slab <bio-0> at 0
> [    0.026798] ACPI: Interpreter disabled.
> [    0.026833] xen_balloon: Initialising balloon driver with page order 0.
> [    0.026878] vgaarb: loaded
> [    0.026939] PCI: System does not support PCI
> [    0.026944] PCI: System does not support PCI
> [    0.027019] Switching to clocksource xen
> [    0.028001] pnp: PnP ACPI: disabled
> [    0.028001] NET: Registered protocol family 2
> [    0.028001] IP route cache hash table entries: 524288 (order: 10,
> 4194304 bytes)
> [    0.028460] TCP established hash table entries: 524288 (order: 11,
> 8388608 bytes)
> [    0.030137] TCP bind hash table entries: 65536 (order: 8, 1048576 bytes)
> [    0.030331] TCP: Hash tables configured (established 524288 bind 65536)
> [    0.030337] TCP reno registered
> [    0.030405] NET: Registered protocol family 1
> [    0.030463] Unpacking initramfs...
> [    0.051340] Freeing initrd memory: 25796k freed
> [    0.058949] PCI-DMA: Using software bounce buffering for IO (SWIOTLB)
> [    0.058964] DMA: Placing 64MB software IO TLB between
> ffff880020000000 - ffff880024000000
> [    0.058970] DMA: software IO TLB at phys 0x20000000 - 0x24000000
> [    0.059058] platform rtc_cmos: registered platform RTC device (no
> PNP device found)
> [    0.059272] audit: initializing netlink socket (disabled)
> [    0.059288] type=2000 audit(1311165823.857:1): initialized
> [    0.064506] HugeTLB registered 2 MB page size, pre-allocated 0 pages
> [    0.065744] VFS: Disk quotas dquot_6.5.2
> [    0.065796] Dquot-cache hash table entries: 512 (order 0, 4096 bytes)
> [    0.065877] msgmni has been set to 32768
> [    0.066057] alg: No test for stdrng (krng)
> [    0.066112] Block layer SCSI generic (bsg) driver version 0.4
> loaded (major 253)
> [    0.066119] io scheduler noop registered
> [    0.066122] io scheduler anticipatory registered
> [    0.066127] io scheduler deadline registered
> [    0.066157] io scheduler cfq registered (default)
> [    0.072642] registering netback
> [    0.074001] Linux agpgart interface v0.103
> [    0.074032] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled
> [    0.173825] input: Macintosh mouse button emulation as
> /devices/virtual/input/input0
> [    0.173872] PNP: No PS/2 controller found. Probing ports directly.
> [    0.174689] i8042.c: No controller found.
> [    0.174750] mice: PS/2 mouse device common for all mice
> [    0.174829] rtc_cmos rtc_cmos: rtc core: registered rtc_cmos as rtc0
> [    0.174883] cpuidle: using governor ladder
> [    0.174890] cpuidle: using governor menu
> [    0.174897] No iBFT detected.
> [    0.175118] TCP cubic registered
> [    0.175221] NET: Registered protocol family 10
> [    0.175788] Mobile IPv6
> [    0.175797] NET: Registered protocol family 17
> [    0.175889] registered taskstats version 1
> [    5.272069] XENBUS: Waiting for devices to initialise:
> 295s...290s...285s...280s...275s...270s...265s...260s...255s...250s...245s...240s...235s...230s...225s...220s...215s...210s...205s...200s...195s...190s...185s...180s...175s...170s...165s...160s...155s...150s...145s...140s...135s...130s...125s....
> udevd[46]: worker [53] unexpectedly returned with status 0x0100
> udevd[46]: worker [53] failed while handling '/devices/vbd-51713'
> udevd[46]: worker [54] unexpectedly returned with status 0x0100
> udevd[46]: worker [54] failed while handling '/devices/vbd-51714'
> Begin: Loading essential drivers ... done.
> Begin: Running /scripts/init-premount ... done.
> Begin: Mounting root file system ... Begin: Running /scripts/local-top ... 
> done.
> Begin: Waiting for root file system ...
> 120s...115s...110s...105s...100s...95s...90s...done.
> Gave up waiting for root device.  Common problems:
>  - Boot args (cat /proc/cmdline)
>    - Check rootdelay= (did the system wait long enough?)
>    - Check root= (did the system wait for the right device?)
>  - Missing modules (cat /proc/modules; ls /dev)
> ALERT!  /dev/xvda2 does not exist.  Dropping to a shell!
> 
> 
> BusyBox v1.17.1 (Debian 1:1.17.1-8) built-in shell (ash)
> Enter 'help' for a list of built-in commands.
> 
> /bin/sh: can't access tty; job control turned off
> (initramfs)
> [  600.488236] XENBUS: Timeout connecting to device: device/vbd/51714
> (local state 3, remote state 1)
> [  600.488328] XENBUS: Timeout connecting to device: device/vbd/51713
> (local state 3, remote state 1)
> 
> >From what I see, the guest is unable to find the disk, which is
> strange, because HVM guests work fine, and the disk is attached
> correctly. Does this also happen when using file:/ backend (qdisk)
> under Linux also? Is it a problem with Qemu or is it related to libxl?

It doesn't happen under Linux; it is probably a problem with Qemu.  Qemu
uses the gntdev device (/dev/xen/gntdev) to open the grant table
(necessary to implement xen backends in userspace).
There doesn't seem anything equivalent on NetBSD, hence it fails...
_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-devel

 


Rackspace

Lists.xenproject.org is hosted with RackSpace, monitoring our
servers 24x7x365 and backed by RackSpace's Fanatical Support®.