|
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index] [Xen-devel] Unable to handle kernel paging request
Uh oh.... On Sat, Aug 05, 2006 at 08:38:03AM +0100, Ian Pratt wrote: > > So I built -unstable changeset 10868, and ran an even heavier workload > > (the above, plus 'bonnie' in the guests) on dom0 and two guests > > overnight, and they experienced no soft lockups; running -unstable, > > changeset 10868, credit scheduler. This same workload would have > > caused soft lockups within seconds in -testing changeset 9732 using > > the sedf scheduler; I may not have been able to get it started at all. > > Response time remained subsecond under -unstable; -testing would have > > been on its knees. > > That's good to hear. 3.0.3 is going to be a big leap forward in many > ways. Same environment as we've discussed earlier in this thread, now on unstable changeset 10868, Xen on top of DRBD, this one seems to be a reliable crash when under combined disk and network load, still isolating but would welcome suggestions... Steve root (hd0,0) Filesystem type is ext2fs, partition type 0x83 kernel /boot/xen-3.0.gz dom0_mem=65536 com1=9600,8n1 [Multiboot-elf, <0x100000:0x7a008:0x55ff8>, shtab=0x1d0078, entry=0x100000] module /boot/vmlinuz-2.6.16.13-xen root=801 ro console=ttyS0 ramdisk_size=32768 [Multiboot-module @ 0x1d1000, 0x494ed0 bytes] module /boot/initrd.img-2.6.16.13-xen [Multiboot-module @ 0x666000, 0x9a6400 bytes] __ __ _____ ___ _ _ _ \ \/ /___ _ __ |___ / / _ \ _ _ _ __ ___| |_ __ _| |__ | | ___ \ // _ \ '_ \ |_ \| | | |__| | | | '_ \/ __| __/ _` | '_ \| |/ _ \ / \ __/ | | | ___) | |_| |__| |_| | | | \__ \ || (_| | |_) | | __/ /_/\_\___|_| |_| |____(_)___/ \__,_|_| |_|___/\__\__,_|_.__/|_|\___| http://www.cl.cam.ac.uk/netos/xen University of Cambridge Computer Laboratory Xen version 3.0-unstable (root@xxxxxxxxxxxxxxxxx) (gcc version 3.3.5 (Debian 1:3.3.5-13)) Fri Aug 4 10:54:50 PDT 2006 Latest ChangeSet: Sat Jul 29 06:05:59 2006 +0100 10868:d2bf1a7cc131 (XEN) Command line: /boot/xen-3.0.gz dom0_mem=65536 com1=9600,8n1 (XEN) Physical RAM map: (XEN) 0000000000000000 - 000000000009c000 (usable) (XEN) 000000000009c000 - 00000000000a0000 (reserved) (XEN) 00000000000e0000 - 0000000000100000 (reserved) (XEN) 0000000000100000 - 000000003ffec340 (usable) (XEN) 000000003ffec340 - 000000003fff0000 (ACPI data) (XEN) 000000003fff0000 - 0000000040000000 (reserved) (XEN) 00000000fec00000 - 0000000100000000 (reserved) (XEN) System RAM: 1023MB (1048096kB) (XEN) Xen heap: 10MB (10396kB) (XEN) Using scheduler: SMP Credit Scheduler (credit) (XEN) PAE disabled. (XEN) found SMP MP-table at 0009c1d0 (XEN) DMI 2.3 present. (XEN) Using APIC driver default (XEN) ACPI: RSDP (v000 IBM ) @ 0x000fdfd0 (XEN) ACPI: RSDT (v001 IBM SEREMRLD 0x00001000 IBM 0x45444f43) @ 0x3ffeff80 (XEN) ACPI: FADT (v001 IBM SEREMRLD 0x00001000 IBM 0x45444f43) @ 0x3ffeff00 (XEN) ACPI: MADT (v001 IBM SEREMRLD 0x00001000 IBM 0x45444f43) @ 0x3ffefe80 (XEN) ACPI: DSDT (v001 IBM SEREMRLD 0x00001000 MSFT 0x0100000b) @ 0x00000000 (XEN) ACPI: Local APIC address 0xfee00000 (XEN) ACPI: LAPIC (acpi_id[0x00] lapic_id[0x03] enabled) (XEN) Processor #3 6:11 APIC version 17 (XEN) ACPI: LAPIC (acpi_id[0x01] lapic_id[0x00] enabled) (XEN) Processor #0 6:11 APIC version 17 (XEN) ACPI: IOAPIC (id[0x0e] address[0xfec00000] gsi_base[0]) (XEN) IOAPIC[0]: apic_id 14, version 17, address 0xfec00000, GSI 0-15 (XEN) ACPI: IOAPIC (id[0x0d] address[0xfec01000] gsi_base[16]) (XEN) IOAPIC[1]: apic_id 13, version 17, address 0xfec01000, GSI 16-31 (XEN) ACPI: INT_SRC_OVR (bus 0 bus_irq 3 global_irq 30 dfl dfl) (XEN) ACPI: IRQ3 used by override. (XEN) Enabling APIC mode: Flat. Using 2 I/O APICs (XEN) Using ACPI (MADT) for SMP configuration information (XEN) Initializing CPU#0 (XEN) Detected 1130.180 MHz processor. (XEN) CPU: L1 I cache: 16K, L1 D cache: 16K (XEN) CPU: L2 cache: 512K (XEN) Intel machine check architecture supported. (XEN) Intel machine check reporting enabled on CPU#0. (XEN) CPU0: Intel(R) Pentium(R) III CPU family 1133MHz stepping 01 (XEN) Booting processor 1/0 eip 90000 (XEN) Initializing CPU#1 (XEN) CPU: L1 I cache: 16K, L1 D cache: 16K (XEN) CPU: L2 cache: 512K (XEN) Intel machine check architecture supported. (XEN) Intel machine check reporting enabled on CPU#1. (XEN) CPU1: Intel(R) Pentium(R) III CPU family 1133MHz stepping 01 (XEN) Total of 2 processors activated. (XEN) ENABLING IO-APIC IRQs (XEN) -> Using new ACK method (XEN) ..TIMER: vector=0xF0 apic1=0 pin1=0 apic2=-1 pin2=-1 (XEN) ..MP-BIOS bug: 8254 timer not connected to IO-APIC (XEN) ...trying to set up timer (IRQ0) through the 8259A ... failed. (XEN) ...trying to set up timer as Virtual Wire IRQ... works. (XEN) checking TSC synchronization across 2 CPUs: passed. (XEN) Platform timer is 1.193MHz PIT (XEN) Brought up 2 CPUs (XEN) Machine check exception polling timer started. (XEN) *** LOADING DOMAIN 0 *** (XEN) Domain 0 kernel supports features = { 0000001f }. (XEN) Domain 0 kernel requires features = { 00000000 }. (XEN) PHYSICAL MEMORY ARRANGEMENT: (XEN) Dom0 alloc.: 3c000000->3e000000 (8192 pages to be allocated) (XEN) VIRTUAL MEMORY ARRANGEMENT: (XEN) Loaded kernel: c0100000->c05d5d74 (XEN) Init. ramdisk: c05d6000->c0f7c400 (XEN) Phys-Mach map: c0f7d000->c0f8d000 (XEN) Start info: c0f8d000->c0f8e000 (XEN) Page tables: c0f8e000->c0f94000 (XEN) Boot stack: c0f94000->c0f95000 (XEN) TOTAL: c0000000->c1400000 (XEN) ENTRY ADDRESS: c0100000 (XEN) Dom0 has maximum 2 VCPUs (XEN) Initrd len 0x9a6400, start at 0xc05d6000 (XEN) Scrubbing Free RAM: ...........done. (XEN) Xen trace buffers: disabled (XEN) Xen is relinquishing VGA console. (XEN) *** Serial input -> DOM0 (type 'CTRL-a' three times to switch input to Xen). Linux version 2.6.16.13-xen (root@n4h8) (gcc version 3.3.5 (Debian 1:3.3.5-13)) #3 SMP Fri Aug 4 10:57:44 PDT 2006 BIOS-provided physical RAM map: Xen: 0000000000000000 - 0000000004800000 (usable) 0MB HIGHMEM available. 72MB LOWMEM available. DMI 2.3 present. ACPI: LAPIC (acpi_id[0x00] lapic_id[0x03] enabled) ACPI: LAPIC (acpi_id[0x01] lapic_id[0x00] enabled) ACPI: IOAPIC (id[0x0e] address[0xfec00000] gsi_base[0]) IOAPIC[0]: apic_id 14, version 17, address 0xfec00000, GSI 0-15 ACPI: IOAPIC (id[0x0d] address[0xfec01000] gsi_base[16]) IOAPIC[1]: apic_id 13, version 17, address 0xfec01000, GSI 16-31 ACPI: INT_SRC_OVR (bus 0 bus_irq 3 global_irq 30 dfl dfl) Enabling APIC mode: Flat. Using 2 I/O APICs Using ACPI (MADT) for SMP configuration information Allocating PCI resources starting at 50000000 (gap: 40000000:bec00000) Built 1 zonelists Kernel command line: root=801 ro console=ttyS0 ramdisk_size=32768 Enabling fast FPU save and restore... done. Enabling unmasked SIMD FPU exception support... done. Initializing CPU#0 PID hash table entries: 512 (order: 9, 8192 bytes) Xen reported: 1130.113 MHz processor. Console: colour VGA+ 80x25 Dentry cache hash table entries: 16384 (order: 4, 65536 bytes) Inode-cache hash table entries: 8192 (order: 3, 32768 bytes) Software IO TLB enabled: Aperture: 2 megabytes Kernel range: 0x00000000c10f0000 - 0x00000000c12f0000 vmalloc area: c5000000-fb7fe000, maxmem 33ffe000 Memory: 47400k/73728k available (3389k kernel code, 18068k reserved, 1044k data, 208k init, 0k highmem) Checking if this processor honours the WP bit even in supervisor mode... Ok. Calibrating delay using timer specific routine.. 2261.60 BogoMIPS (lpj=11308023) Security Framework v1.0.0 initialized Capability LSM initialized Mount-cache hash table entries: 512 CPU: L1 I cache: 16K, L1 D cache: 16K CPU: L2 cache: 512K Checking 'hlt' instruction... OK. ENABLING IO-APIC IRQs Enabling SMP... Brought up 2 CPUs Initializing CPU#1 migration_cost=1895 checking if image is initramfs... it is Freeing initrd memory: 9881k freed Grant table initialized NET: Registered protocol family 16 ACPI: bus type pci registered PCI: Using configuration type 1 ACPI: Subsystem revision 20060127 ACPI: Interpreter enabled ACPI: Using IOAPIC for interrupt routing ACPI: PCI Root Bridge [PCI0] (0000:00) ACPI: PCI Interrupt Link [LPE1] (IRQs *10) ACPI: PCI Interrupt Link [LPE2] (IRQs *10) ACPI: PCI Interrupt Link [LPVI] (IRQs) *0, disabled. ACPI: PCI Interrupt Link [LPUS] (IRQs *7) ACPI: PCI Root Bridge [PCI1] (0000:01) ACPI: PCI Interrupt Link [LPSA] (IRQs *9) ACPI: PCI Interrupt Link [LP1A] (IRQs) *0, disabled. ACPI: PCI Interrupt Link [LP1B] (IRQs) *0, disabled. ACPI: PCI Interrupt Link [LP2A] (IRQs *9) ACPI: PCI Interrupt Link [LP2B] (IRQs) *0, disabled. Linux Plug and Play Support v0.97 (c) Adam Belay xen_mem: Initialising balloon driver. SCSI subsystem initialized usbcore: registered new driver usbfs usbcore: registered new driver hub PCI: Using ACPI for IRQ routing PCI: If a device doesn't work, try "pci=routeirq". If it helps, post a report IA-32 Microcode Update Driver: v1.14-xen <tigran@xxxxxxxxxxx> VFS: Disk quotas dquot_6.5.1 Dquot-cache hash table entries: 1024 (order 0, 4096 bytes) JFS: nTxBlock = 512, nTxLock = 4096 SGI XFS with ACLs, security attributes, realtime, large block numbers, no debug enabled Initializing Cryptographic API io scheduler noop registered io scheduler anticipatory registered (default) io scheduler deadline registered io scheduler cfq registered PNP: No PS/2 controller found. Probing ports directly. serio: i8042 AUX port at 0x60,0x64 irq 12 serio: i8042 KBD port at 0x60,0x64 irq 1 RAMDISK driver initialized: 16 RAM disks of 32768K size 1024 blocksize Xen virtual console successfully installed as ttyS0 Event-channel device installed. blkif_init: reqs=64, pages=704, mmap_vstart=0xc0800000 Uniform Multi-Platform E-IDE driver Revision: 7.00alpha2 ide: Assuming 33MHz system bus speed for PIO modes; override with idebus=xx SvrWks OSB4: IDE controller at PCI slot 0000:00:0f.1 SvrWks OSB4: chipset revision 0 SvrWks OSB4: not 100% native mode: will probe irqs later ide0: BM-DMA at 0x0700-0x0707, BIOS settings: hda:DMA, hdb:DMA ide1: BM-DMA at 0x0708-0x070f, BIOS settings: hdc:DMA, hdd:DMA hda: LG CD-ROM CRN-8245B, ATAPI CD/DVD-ROM drive ide0 at 0x1f0-0x1f7,0x3f6 on irq 14 hda: ATAPI 24X CD-ROM drive, 128kB Cache, (U)DMA Uniform CD-ROM driver Revision: 3.20 ide-floppy driver 0.99.newide ACPI: PCI Interrupt 0000:01:03.0[A] -> GSI 28 (level, low) -> IRQ 16 scsi0 : Adaptec AIC7XXX EISA/VLB/PCI SCSI HBA DRIVER, Rev 7.0 <Adaptec aic7892 Ultra160 SCSI adapter> aic7892: Ultra160 Wide Channel A, SCSI Id=7, 32/253 SCBs ACPI: PCI Interrupt 0000:01:05.0[A] -> GSI 20 (level, low) -> IRQ 17 scsi1 : IBM PCI ServeRAID 7.12.05 Build 761 <ServeRAID 4L> Vendor: IBM Model: SERVERAID Rev: 1.00 Type: Direct-Access ANSI SCSI revision: 02 Vendor: IBM Model: SERVERAID Rev: 1.00 Type: Direct-Access ANSI SCSI revision: 02 Vendor: IBM Model: SERVERAID Rev: 1.00 Type: Processor ANSI SCSI revision: 02 Vendor: IBM Model: FTlV1 S2 Rev: 0 Type: Processor ANSI SCSI revision: 02 Fusion MPT base driver 3.03.07 Copyright (c) 1999-2005 LSI Logic Corporation Fusion MPT SPI Host driver 3.03.07 Fusion MPT misc device (ioctl) driver 3.03.07 mptctl: Registered with Fusion MPT base driver mptctl: /dev/mptctl @ (major,minor=10,220) usbmon: debugfs is not available usbcore: registered new driver libusual mice: PS/2 mouse device common for all mice md: md driver 0.90.3 MAX_MD_DEVS=256, MD_SB_DISKS=27 md: bitmap version 4.39 NET: Registered protocol family 2 input: AT Translated Set 2 keyboard as /class/input/input0 IP route cache hash table entries: 1024 (order: 0, 4096 bytes) TCP established hash table entries: 4096 (order: 3, 32768 bytes) TCP bind hash table entries: 4096 (order: 3, 32768 bytes) TCP: Hash tables configured (established 4096 bind 4096) TCP reno registered Initializing IPsec netlink socket NET: Registered protocol family 1 NET: Registered protocol family 17 NET: Registered protocol family 8 NET: Registered protocol family 20 Using IPI No-Shortcut mode Freeing unused kernel memory: 208k freed Loading, please wait... Begin: Loading essential drivers... ... logips2pp: Detected unknown logitech mouse model 0 eepro100.c:v1.09j-t 9/29/99 Donald Becker http://www.scyld.com/network/eepro100.html eepro100.c: $Revision: 1.36 $ 2000/11/17 Modified by Andrey V. Savochkin <saw@xxxxxxxxxxxxx> and others ACPI: PCI Interrupt 0000:00:02.0[A] -> GSI 27 (level, low) -> IRQ 18 eth0: 0000:00:02.0, 00:02:55:C7:CA:D8, IRQ 18. Board assembly 754338-001, Physical connectors present: RJ45 Primary interface chip i82555 PHY #1. General self-test: passed. Serial sub-system self-test: passed. Internal registers self-test: passed. ROM checksum self-test: passed (0x04f4518b). ACPI: PCI Interrupt 0000:00:0a.0[A] -> GSI 25 (level, low) -> IRQ 19 eth1: 0000:00:0a.0, 00:02:55:C7:CA:D9, IRQ 19. Board assembly 754338-001, Physical connectors present: RJ45 Primary interface chip i82555 PHY #1. General self-test: passed. Serial sub-system self-test: passed. Internal registers self-test: passed. ROM checksum self-test: passed (0x04f4518b). Intel(R) PRO/1000 Network Driver - version 6.3.9-k4 Copyright (c) 1999-2005 Intel Corporation. Done. Begin: Running /scripts/init-premount ... input: PS/2 Logitech Mouse as /class/input/input1 Done. Begin: Mounting root file system... ... Begin: Running /scripts/local-top ... device-mapper: 4.5.0-ioctl (2005-10-04) initialised: dm-devel@xxxxxxxxxx /scripts/local-top/lvm: 36: vgchange: not found Done. SCSI device sda: 71096320 512-byte hdwr sectors (36401 MB) sda: assuming Write Enabled sda: assuming drive cache: write through SCSI device sda: 71096320 512-byte hdwr sectors (36401 MB) sda: assuming Write Enabled sda: assuming drive cache: write through sda: sda1 sda2 < sda5 > sd 1:0:0:0: Attached scsi disk sda SCSI device sdb: 71096320 512-byte hdwr sectors (36401 MB) sdb: assuming Write Enabled sdb: assuming drive cache: write through SCSI device sdb: 71096320 512-byte hdwr sectors (36401 MB) sdb: assuming Write Enabled sdb: assuming drive cache: write through sdb: unknown partition table sd 1:0:1:0: Attached scsi disk sdb Begin: Running /scripts/local-premount ... Done. EXT3-fs: INFO: recovery required on readonly filesystem. EXT3-fs: write access will be enabled during recovery. kjournald starting. Commit interval 5 seconds EXT3-fs: recovery complete. EXT3-fs: mounted filesystem with ordered data mode. Begin: Running /scripts/log-bottom ... Done. Done. Begin: Running /scripts/init-bottom ... Done. INIT: version 2.86 booting Starting the hotplug events dispatcher: udevd. Synthesizing the initial hotplug events...done. Waiting for /dev to be fully populated...done. Loading /etc/console/boottime.kmap.gz Activating swap. Adding 1019880k swap on /dev/sda5. Priority:-1 extents:1 across:1019880k Checking root file system... fsck 1.37 (21-Mar-2005) /dev/sda1: clean, 58437/512000 files, 445341/1023996 blocks EXT3 FS on sda1, internal journal System time was Sat Aug 12 00:22:00 UTC 2006. Setting the System Clock using the Hardware Clock as reference... System Clock set. System local time is now Sat Aug 12 00:22:01 UTC 2006. Calculating module dependencies...done. Loading modules... auto FATAL: Module auto not found. 3c59x softdog Software Watchdog Timer: 0.07 initialized. soft_noboot=0 soft_margin=60 sec (nowayout= 0) tulip Linux Tulip driver version 1.1.13 (May 11, 2002) e100 e100: Intel(R) PRO/100 Network Driver, 3.5.10-k2-NAPI e100: Copyright(c) 1999-2005 Intel Corporation All modules loaded. Loading device-mapper support. Starting Enterprise Volume Management System: device-mapper: dm-linear: Device lookup failed device-mapper: error adding target to table device-mapper: dm-linear: Device lookup failed device-mapper: error adding target to table device-mapper: dm-linear: Device lookup failed device-mapper: error adding target to table device-mapper: dm-linear: Device lookup failed device-mapper: error adding target to table device-mapper: dm-linear: Device lookup failed device-mapper: error adding target to table device-mapper: dm-linear: Device lookup failed device-mapper: error adding target to table device-mapper: dm-linear: Device lookup failed device-mapper: error adding target to table device-mapper: dm-linear: Device lookup failed device-mapper: error adding target to table evms. Checking all file systems... fsck 1.37 (21-Mar-2005) Setting kernel variables ... ... done. Loading the saved-state of the serial devices... Cannot get serial info: Invalid argument Cannot get serial info: Invalid argument /dev/ttyS1: No such file or directory /dev/ttyS1: No such file or directory Mounting local filesystems... Cleaning /tmp /var/run /var/lock. Cleaning: /etc/network/ifstate. Setting up IP spoofing protection: rp_filter. Configuring network interfaces: done. Starting portmap daemon: portmap. Setting the System Clock using the Hardware Clock as reference... System Clock set. Local time: Fri Aug 11 17:22:07 PDT 2006 Recovering jove files ... Done. Running ntpdate to synchronize clock. Initializing random number generator...done. Recovering nvi editor sessions... done. Setting up X server socket directory /tmp/.X11-unix...done. Setting up ICE socket directory /tmp/.ICE-unix...done. INIT: Entering runlevel: 2 Starting system log daemon: syslogd. Starting kernel log daemon: klogd. Starting virtual private network daemon:. Starting portmap daemon: portmap. Starting NFS common utilities: statd lockd. Starting internet superserver: inetd. * The mptctl module is missing. Please have a look at the README.Debian.gz. . Exporting directories for NFS kernel daemon...Installing knfsd (copyright (C) 1996 okir@xxxxxxxxxxxx). 10.27.3.21:/export/xen/fs/stevegt/crashme1/root: No such file or directory 10.27.3.18:/export/xen/fs/stevegt/xentest2/root: No such file or directory 10.27.3.17:/export/xen/fs/stevegt/xentest1/root: No such file or directory 10.27.3.8:/export/xen/fs/cclarke/umltest1/root: No such file or directory hotaru.chaosring.Org:/export/xen/fs/stevegt/crashme1/root: No such file or directory rn-2.refactored-networks.com:/export/xen/fs/stevegt/xentest2/root: No such file or directory xentest1.stevegt.TerraLuna.Org:/export/xen/fs/stevegt/xentest1/root: No such file or directory umltest1.cclarke.TerraLuna.Org:/export/xen/fs/cclarke/umltest1/root: No such file or directory 10.27.3.26:/export/xen/fs/stevegt/gforge/root: No such file or directory 10.27.3.23:/export/xen/fs/baseline/sarge/root: No such file or directory 10.27.3.22:/export/xen/fs/stevegt/build1/root: No such file or directory 10.27.3.16:/export/xen/fs/baseline/woody/root: No such file or directory bugs.t7a.Org:/export/xen/fs/stevegt/gforge/root: No such file or directory sarge.baseline.TerraLuna.Org:/export/xen/fs/baseline/sarge/root: No such file or directory build1.stevegt.TerraLuna.Org:/export/xen/fs/stevegt/build1/root: No such file or directory woody.baseline.TerraLuna.Org:/export/xen/fs/baseline/woody/root: No such file or directory 10.27.3.19:/export/xen/fs/cclarke/ebond/root: No such file or directory ebond.cclarke.TerraLuna.Org:/export/xen/fs/cclarke/ebond/root: No such file or directory 10.27.3.29:/export/xen/fs/dmasten/mss0/root: No such file or directory 10.27.3.27:/export/xen/fs/cclarke/ccms/root: No such file or directory 10.27.3.25:/export/xen/fs/baseline/sid/root: No such file or directory 10.27.3.24:/export/xen/fs/stevegt/cvs1/root: No such file or directory sid.stevegt.TerraLuna.Org:/export/xen/fs/baseline/sid/root: No such file or directory cvs1.stevegt.TerraLuna.Org:/export/xen/fs/stevegt/cvs1/root: No such file or directory 10.27.3.28:/export/xen/fs/stevegt/t7a/root: No such file or directory 10.27.3.20:/export/xen/fs/stevegt/tcx/root: No such file or directory tcx.TerraLuna.Org:/export/xen/fs/stevegt/tcx/root: No such file or directory n2h54.prd.TerraLuna.Org:/export/xen: No such file or directory n2h53.prd.TerraLuna.Org:/export/xen: No such file or directory n2h51.prd.TerraLuna.Org:/export/xen: No such file or directory n2h41.prd.TerraLuna.Org:/export/xen: No such file or directory done. Starting NFS kernel daemon: nfsdNFSD: Using /var/lib/nfs/v4recovery as the NFSv4 state recovery directory NFSD: unable to find recovery directory /var/lib/nfs/v4recovery NFSD: starting 90-second grace period mountd. Starting OpenBSD Secure Shell server: sshdNET: Registered protocol family 10 lo: Disabled Privacy Extensions IPv6 over IPv4 tunneling driver . Starting NTP server: ntpd. AFS module /lib/modules/2.6.16.13-xen/fs/openafs.mp.o does not exist. Not starting AFS. Please consider building kernel modules using instructions in /usr/share/doc/openafs-client/README.modules Starting DRBD resources: drbd: initialised. Version: 0.7.20 (api:79/proto:74) drbd: SVN Revision: 2260 build by root@n4h8, 2006-08-04 11:00:44 drbd: registered as block device major 147 [ d0 drbd7: resync bitmap: bits=1048576 words=32768 drbd7: size = 4095 MB (4194303 KB) drbd7: 0 KB marked out-of-sync by on disk bit-map. drbd7: No usable activity log found. drbd7: drbdsetup [3825]: cstate Unconfigured --> StandAlone d1 drbd8: resync bitmap: bits=1048576 words=32768 drbd8: size = 4095 MB (4194303 KB) drbd8: 0 KB marked out-of-sync by on disk bit-map. drbd8: Found 4 transactions (46 active extents) in activity log. drbd8: Marked additional 156 MB as out-of-sync based on AL. drbd8: drbdsetup [3829]: cstate Unconfigured --> StandAlone d2 drbd9: resync bitmap: bits=2621440 words=81920 drbd9: size = 9 GB (10485759 KB) drbd9: 0 KB marked out-of-sync by on disk bit-map. drbd9: Found 4 transactions (192 active extents) in activity log. drbd9: drbdsetup [3833]: cstate Unconfigured --> StandAlone d3 drbd10: resync bitmap: bits=1835008 words=57344 drbd10: size = 7167 MB (7340031 KB) drbd10: 0 KB marked out-of-sync by on disk bit-map. drbd10: Found 4 transactions (192 active extents) in activity log. drbd10: Marked additional 508 MB as out-of-sync based on AL. drbd10: drbdsetup [3837]: cstate Unconfigured --> StandAlone d4 drbd11: resync bitmap: bits=786432 words=24576 drbd11: size = 3071 MB (3145727 KB) drbd11: 0 KB marked out-of-sync by on disk bit-map. drbd11: Found 4 transactions (192 active extents) in activity log. drbd11: drbdsetup [3841]: cstate Unconfigured --> StandAlone d5 drbd12: resync bitmap: bits=786432 words=24576 drbd12: size = 3071 MB (3145727 KB) drbd12: 0 KB marked out-of-sync by on disk bit-map. drbd12: Found 4 transactions (136 active extents) in activity log. drbd12: Marked additional 508 MB as out-of-sync based on AL. drbd12: drbdsetup [3845]: cstate Unconfigured --> StandAlone s0 s1 s2 s3 s4 s5 n0 drbd7: drbdsetup [3883]: cstate StandAlone --> Unconnected drbd7: drbd7_receiver [3884]: cstate Unconnected --> WFConnection n1 drbd8: drbdsetup [3891]: cstate StandAlone --> Unconnected drbd8: drbd8_receiver [3892]: cstate Unconnected --> WFConnection n2 drbd9: drbdsetup [3899]: cstate StandAlone --> Unconnected drbd9: drbd9_receiver [3900]: cstate Unconnected --> WFConnection n3 drbd10: drbdsetup [3907]: cstate StandAlone --> Unconnected drbd10: drbd10_receiver [3908]: cstate Unconnected --> WFConnection n4 drbd11: drbdsetup [3915]: cstate StandAlone --> Unconnected drbd11: drbd11_receiver [3916]: cstate Unconnected --> WFConnection n5 drbd12: drbdsetup [3923]: cstate StandAlone --> Unconnected drbd12: drbd12_receiver [3924]: cstate Unconnected --> WFConnection ]. .......... *************************************************************** DRBD's startup script waits for the peer node(s) to appear. - In case this node was already a degraded cluster before the reboot the timeout is 120 seconds. [degr-wfc-timeout] - If the peer was available before the reboot the timeout will expire after 0 seconds. [wfc-timeout] (These values are for resource 'ds1.t7a.org'; 0 sec -> wait forever) To abort waiting enter 'yes' [ 18]:drbd7: drbd7_receiver [3884]: cstate WFConnection --> WFReportParams drbd7: Handshake successful: DRBD Network Protocol version 74 drbd7: Connection established. drbd7: I am(S): 1:00000002:00000001:00000018:00000002:01 drbd7: Peer(S): 1:00000002:00000001:00000019:00000002:10 drbd7: drbd7_receiver [3884]: cstate WFReportParams --> WFBitMapT drbd7: Secondary/Unknown --> Secondary/Secondary drbd7: drbd7_receiver [3884]: cstate WFBitMapT --> SyncTarget drbd7: Resync started as SyncTarget (need to sync 520192 KB [130048 bits set]). drbd9: drbd9_receiver [3900]: cstate WFConnection --> WFReportParams drbd9: Handshake successful: DRBD Network Protocol version 74 drbd9: Connection established. drbd9: I am(S): 1:00000002:00000001:0000000d:00000001:01 drbd9: Peer(S): 1:00000002:00000001:0000000e:00000001:10 drbd9: drbd9_receiver [3900]: cstate WFReportParams --> WFBitMapT drbd9: Secondary/Unknown --> Secondary/Secondary drbd9: drbd9_receiver [3900]: cstate WFBitMapT --> SyncTarget drbd9: Resync started as SyncTarget (need to sync 290816 KB [72704 bits set]). drbd9: drbd9_receiver [3900]: cstate SyncTarget --> PausedSyncT drbd9: Syncer waits for sync group. drbd11: drbd11_receiver [3916]: cstate WFConnection --> WFReportParams drbd11: Handshake successful: DRBD Network Protocol version 74 drbd11: Connection established. drbd11: I am(S): 1:00000002:00000001:00000008:00000001:01 drbd11: Peer(S): 1:00000002:00000001:00000009:00000001:10 drbd11: drbd11_receiver [3916]: cstate WFReportParams --> WFBitMapT drbd11: Secondary/Unknown --> Secondary/Secondary drbd11: drbd11_receiver [3916]: cstate WFBitMapT --> SyncTarget drbd11: Resync started as SyncTarget (need to sync 274432 KB [68608 bits set]). drbd11: drbd11_receiver [3916]: cstate SyncTarget --> PausedSyncT drbd11: Syncer waits for sync group. drbd8: drbd8_receiver [3892]: cstate WFConnection --> WFReportParams drbd8: Handshake successful: DRBD Network Protocol version 74 drbd8: Connection established. drbd8: I am(S): 1:00000002:00000001:00000017:00000001:11 drbd8: Peer(S): 1:00000002:00000001:00000017:00000001:01 drbd8: drbd8_receiver [3892]: cstate WFReportParams --> WFBitMapS drbd8: Secondary/Unknown --> Secondary/Secondary drbd8: drbd8_receiver [3892]: cstate WFBitMapS --> SyncSource drbd8: Resync started as SyncSource (need to sync 159744 KB [39936 bits set]). drbd8: drbd8_receiver [3892]: cstate SyncSource --> PausedSyncS drbd8: Syncer waits for sync group. drbd12: drbd12_receiver [3924]: cstate WFConnection --> WFReportParams drbd12: Handshake successful: DRBD Network Protocol version 74 drbd12: Connection established. drbd12: I am(S): 1:00000002:00000001:00000001:00000001:11 drbd12: Peer(S): 1:00000002:00000001:00000001:00000001:01 drbd12: drbd12_receiver [3924]: cstate WFReportParams --> WFBitMapS drbd12: Secondary/Unknown --> Secondary/Secondary drbd12: drbd12_receiver [3924]: cstate WFBitMapS --> SyncSource drbd12: Resync started as SyncSource (need to sync 520192 KB [130048 bits set]). drbd12: drbd12_receiver [3924]: cstate SyncSource --> PausedSyncS drbd12: Syncer waits for sync group. drbd10: drbd10_receiver [3908]: cstate WFConnection --> WFReportParams drbd10: Handshake successful: DRBD Network Protocol version 74 drbd10: Connection established. drbd10: I am(S): 1:00000002:00000001:0000000b:00000001:11 drbd10: Peer(S): 1:00000002:00000001:0000000b:00000001:01 drbd10: drbd10_receiver [3908]: cstate WFReportParams --> WFBitMapS drbd10: Secondary/Unknown --> Secondary/Secondary drbd10: drbd10_receiver [3908]: cstate WFBitMapS --> SyncSource drbd10: Resync started as SyncSource (need to sync 520192 KB [130048 bits set]). drbd10: drbd10_receiver [3908]: cstate SyncSource --> PausedSyncS drbd10: Syncer waits for sync group. Heartbeat not configured: /etc/ha.d/ha.cf not found. Heartbeat failure [rc=1]. Failed. Starting deferred execution scheduler: atd. Starting periodic command scheduler: cron. Starting watchdog daemon: watchdog. openMosix: ERROR: Cannot find the /proc/hpc directory. openMosix: ERROR: Looks like this is not an openMosix enabled kernel. openMosix: ERROR: Configuration ABORTED. openMosix: HINT: need to recompile your kernel with openMosix support enabled? openMosix: HINT: need to update your boot manager (lilo, grub, etc)? openMosix: HINT: have you read the docs at http://openMosix.sf.net/ ? Bridge firewalling registered device vif0.0 entered promiscuous mode xenbr0: port 1(vif0.0) entering learning state xenbr0: topology change detected, propagating xenbr0: port 1(vif0.0) entering forwarding state device peth0 entered promiscuous mode xenbr0: port 2(peth0) entering learning state xenbr0: topology change detected, propagating xenbr0: port 2(peth0) entering forwarding state Starting auto Xen domains: afs1.t7a.orgdrbd9: Secondary/Secondary --> Secondary/Primary device vif1.0 entered promiscuous mode ADDRCONF(NETDEV_UP): vif1.0: link is not ready ip_tables: (C) 2000-2006 Netfilter Core Team Error: Device 2049 (vbd) could not be connected. Backend device not found. xenbr0: port 3(vif1.0) entering disabled state ! build2.t7a.orgdevice vif1.0 left promiscuous mode xenbr0: port 3(vif1.0) entering disabled state device vif2.0 entered promiscuous mode ADDRCONF(NETDEV_UP): vif2.0: link is not ready drbd10: Secondary/Secondary --> Primary/Secondary ADDRCONF(NETDEV_CHANGE): vif2.0: link becomes ready xenbr0: port 3(vif2.0) entering learning state xenbr0: topology change detected, propagating xenbr0: port 3(vif2.0) entering forwarding state ds1.t7a.orgdrbd7: Secondary/Secondary --> Secondary/Primary device vif3.0 entered promiscuous mode ADDRCONF(NETDEV_UP): vif3.0: link is not ready Error: Device 2049 (vbd) could not be connected. Backend device not found. ! kdc1.t7a.orgxenbr0: port 4(vif3.0) entering disabled state device vif3.0 left promiscuous mode xenbr0: port 4(vif3.0) entering disabled state drbd8: Secondary/Secondary --> Primary/Secondary device vif4.0 entered promiscuous mode ADDRCONF(NETDEV_UP): vif4.0: link is not ready ADDRCONF(NETDEV_CHANGE): vif4.0: link becomes ready xenbr0: port 4(vif4.0) entering learning state xenbr0: topology change detected, propagating xenbr0: port 4(vif4.0) entering forwarding state w1.cdint.comdrbd11: Secondary/Secondary --> Secondary/Primary device vif5.0 entered promiscuous mode ADDRCONF(NETDEV_UP): vif5.0: link is not ready Error: Device 2049 (vbd) could not be connected. Backend device not found. xenbr0: port 5(vif5.0) entering disabled state ! webkdc.t7a.orgdevice vif5.0 left promiscuous mode xenbr0: port 5(vif5.0) entering disabled state device vif6.0 entered promiscuous mode ADDRCONF(NETDEV_UP): vif6.0: link is not ready drbd12: Secondary/Secondary --> Primary/Secondary ADDRCONF(NETDEV_CHANGE): vif6.0: link becomes ready xenbr0: port 5(vif6.0) entering learning state xenbr0: topology change detected, propagating xenbr0: port 5(vif6.0) entering forwarding state [done] _sudZUZ#Z#XZo=_ DDDD EEEEEE BBBB IIIIII AAAA NN NN _jmZZ2!!~---~!!X##wa DD DD EE BB BB II AA AA NNN NN .<wdP~~ -!YZL, DD DD EEEEE BBBBB II AAAAAA NNNN NN .mX2' _%aaa__ XZ[. DD DD EE BB BB II AA AA NN NNNN oZ[ _jdXY!~?S#wa ]Xb; DDDD EEEEEE BBBBB IIIIII AA AA NN NN _#e' .]X2( ~Xw| )XXc .2Z` ]X[. xY| ]oZ( Linux Version 2.6.16.13-xen .2#; )3k; _s!~ jXf` Compiled #3 SMP Fri Aug 4 10:57:44 PDT 2006 1Z> -]Xb/ ~ __#2( Two 1.13GHz Intel Intel(R) Pentium(R) III CPU family 1133MHz Processors -Zo; +!4ZwaaaauZZXY' 72M RAM *#[, ~-?!!!!!!-~ 4523.20 Bogomips Total XUb;. n4h34 )YXL,, +3#bc, -)SSL,, ~~~~~ Updating the Linuxlogo... done. Debian GNU/Linux 3.0 n4h34 ttyS0 n4h34 login: drbd7: Resync done (total 137 sec; paused 0 sec; 3796 K/sec) drbd7: drbd7_worker [3826]: cstate SyncTarget --> Connected drbd8: Syncer continues. drbd8: drbd7_worker [3826]: cstate PausedSyncS --> SyncSource drbd9: Syncer continues. drbd9: drbd7_worker [3826]: cstate PausedSyncT --> SyncTarget drbd10: Syncer continues. drbd10: drbd7_worker [3826]: cstate PausedSyncS --> SyncSource drbd11: Syncer continues. drbd11: drbd7_worker [3826]: cstate PausedSyncT --> SyncTarget drbd12: Syncer continues. drbd12: drbd7_worker [3826]: cstate PausedSyncS --> SyncSource drbd8: Resync done (total 211 sec; paused 137 sec; 2156 K/sec) drbd8: drbd8_worker [3830]: cstate SyncSource --> Connected drbd11: Resync done (total 239 sec; paused 137 sec; 2688 K/sec) drbd11: drbd11_worker [3842]: cstate SyncTarget --> Connected drbd9: Resync done (total 243 sec; paused 137 sec; 2740 K/sec) drbd9: drbd9_worker [3834]: cstate SyncTarget --> Connected Debian GNU/Linux 3.0 n4h34 ttyS0 n4h34 login: Debian GNU/Linux 3.0 n4h34 ttyS0 n4h34 login: Debian GNU/Linux 3.0 n4h34 ttyS0 n4h34 login: Debian GNU/Linux 3.0 n4h34 ttyS0 n4h34 login: Debian GNU/Linux 3.0 n4h34 ttyS0 n4h34 login: Debian GNU/Linux 3.0 n4h34 ttyS0 n4h34 login: drbd12: Resync done (total 306 sec; paused 136 sec; 3056 K/sec) drbd12: drbd12_worker [3846]: cstate SyncSource --> Connected Debian GNU/Linux 3.0 n4h34 ttyS0 n4h34 login: Debian GNU/Linux 3.0 n4h34 ttyS0 n4h34 login: drbd10: Resync done (total 310 sec; paused 136 sec; 2988 K/sec) drbd10: drbd10_worker [3838]: cstate SyncSource --> Connected Debian GNU/Linux 3.0 n4h34 ttyS0 n4h34 login: Unable to handle kernel paging request at virtual address c0976590 printing eip: c03d6ed7 *pde = ma 3cf91067 pa 00f91067 *pte = ma 00000000 pa fffff000 Oops: 0000 [#1] SMP Modules linked in: iptable_filter ip_tables x_tables bridge drbd ipv6 nfsd lockd sunrpc e100 tulip softdog 3c59x evdev sd_mod dm_mod thermal processor fan e1000 eepro100 mii tg3 CPU: 0 EIP: 0061:[<c03d6ed7>] Not tainted VLI EFLAGS: 00010206 (2.6.16.13-xen #3) EIP is at skb_copy_bits+0x127/0x280 eax: c0976000 ebx: 000005a8 ecx: 0000016a edx: c3711720 esi: c0976590 edi: c37110e0 ebp: 000005a8 esp: c00818c8 ds: 007b es: 007b ss: 0069 Process drbd12_receiver (pid: 3924, threadinfo=c0080000 task=c3efda90) Stack: <0>c1012ec0 00000002 c03d65ef c1d51a00 000005ea 00000042 00000000 00000000 00000020 c3edc800 c376fd64 c03d6b0f c376fd64 00000042 c37110e0 000005a8 c1a70000 c54690c0 00000000 c1d51ac0 c3edc800 c376fd64 c03dc478 c376fd64 Call Trace: [<c03d65ef>] pskb_expand_head+0xdf/0x140 [<c03d6b0f>] __pskb_pull_tail+0x7f/0x320 [<c54690c0>] br_nf_dev_queue_xmit+0x0/0x50 [bridge] [<c03dc478>] dev_queue_xmit+0x328/0x370 [<c5462f7e>] br_dev_queue_push_xmit+0xbe/0x140 [bridge] [<c5469212>] br_nf_post_routing+0x102/0x1c0 [bridge] [<c54690c0>] br_nf_dev_queue_xmit+0x0/0x50 [bridge] [<c5462ec0>] br_dev_queue_push_xmit+0x0/0x140 [bridge] [<c03f43f8>] nf_iterate+0x78/0x90 [<c5462ec0>] br_dev_queue_push_xmit+0x0/0x140 [bridge] [<c5462ec0>] br_dev_queue_push_xmit+0x0/0x140 [bridge] [<c03f447e>] nf_hook_slow+0x6e/0x110 [<c5462ec0>] br_dev_queue_push_xmit+0x0/0x140 [bridge] [<c5463061>] br_forward_finish+0x61/0x70 [bridge] [<c5462ec0>] br_dev_queue_push_xmit+0x0/0x140 [bridge] [<c5468995>] br_nf_forward_finish+0x75/0x130 [bridge] [<c5463000>] br_forward_finish+0x0/0x70 [bridge] [<c5468b38>] br_nf_forward_ip+0xe8/0x190 [bridge] [<c5468920>] br_nf_forward_finish+0x0/0x130 [bridge] [<c5463000>] br_forward_finish+0x0/0x70 [bridge] [<c03f43f8>] nf_iterate+0x78/0x90 [<c5463000>] br_forward_finish+0x0/0x70 [bridge] [<c5463000>] br_forward_finish+0x0/0x70 [bridge] [<c03f447e>] nf_hook_slow+0x6e/0x110 [<c5463000>] br_forward_finish+0x0/0x70 [bridge] [<c5463167>] __br_forward+0x77/0x80 [bridge] [<c5463000>] br_forward_finish+0x0/0x70 [bridge] [<c5463fbf>] br_handle_frame_finish+0xdf/0x160 [bridge] [<c5463ee0>] br_handle_frame_finish+0x0/0x160 [bridge] [<c5467d89>] br_nf_pre_routing_finish+0xf9/0x370 [bridge] [<c5463ee0>] br_handle_frame_finish+0x0/0x160 [bridge] [<c0322e3a>] loopback_start_xmit+0xba/0x110 [<c0400d70>] ip_finish_output+0x0/0x220 [<c03dc07e>] dev_hard_start_xmit+0x5e/0x130 [<c03dc3b5>] dev_queue_xmit+0x265/0x370 [<c03f43f8>] nf_iterate+0x78/0x90 [<c5467c90>] br_nf_pre_routing_finish+0x0/0x370 [bridge] [<c5467c90>] br_nf_pre_routing_finish+0x0/0x370 [bridge] [<c03f447e>] nf_hook_slow+0x6e/0x110 [<c5467c90>] br_nf_pre_routing_finish+0x0/0x370 [bridge] [<c5463ee0>] br_handle_frame_finish+0x0/0x160 [bridge] [<c54685fc>] br_nf_pre_routing+0x26c/0x520 [bridge] [<c5467c90>] br_nf_pre_routing_finish+0x0/0x370 [bridge] [<c03f43f8>] nf_iterate+0x78/0x90 [<c5463ee0>] br_handle_frame_finish+0x0/0x160 [bridge] [<c5463ee0>] br_handle_frame_finish+0x0/0x160 [bridge] [<c03f447e>] nf_hook_slow+0x6e/0x110 [<c5463ee0>] br_handle_frame_finish+0x0/0x160 [bridge] [<c546422d>] br_handle_frame+0x1ed/0x230 [bridge] [<c5463ee0>] br_handle_frame_finish+0x0/0x160 [bridge] [<c03dcb21>] netif_receive_skb+0x1a1/0x330 [<c03dcd87>] process_backlog+0xd7/0x190 [<c03dcf2a>] net_rx_action+0xea/0x230 [<c0125915>] __do_softirq+0xf5/0x120 [<c01259d5>] do_softirq+0x95/0xa0 [<c0125a42>] local_bh_enable+0x62/0xa0 [<c0406bd1>] tcp_prequeue_process+0x71/0x80 [<c04070e9>] tcp_recvmsg+0x349/0x750 [<c50eebfc>] dm_request+0xbc/0x100 [dm_mod] [<c03d5085>] sock_common_recvmsg+0x55/0x70 [<c03d11cf>] sock_recvmsg+0xef/0x110 [<c03143fa>] force_evtchn_callback+0xa/0x10 [<c0147163>] mempool_alloc+0x33/0xe0 [<c01367d0>] autoremove_wake_function+0x0/0x60 [<c50eebfc>] dm_request+0xbc/0x100 [dm_mod] [<c02b2ee0>] generic_make_request+0xf0/0x160 [<c529c680>] drbd_recv+0x90/0x190 [drbd] [<c529cdec>] drbd_recv_header+0x2c/0xf0 [drbd] [<c529e580>] receive_DataRequest+0x0/0x7d0 [drbd] [<c52a045c>] drbdd+0x1c/0x150 [drbd] [<c52a105a>] drbdd_init+0x7a/0x1a0 [drbd] [<c52a7136>] drbd_thread_setup+0x86/0xf0 [drbd] [<c52a70b0>] drbd_thread_setup+0x0/0xf0 [drbd] [<c0102f75>] kernel_thread_helper+0x5/0x10 Code: 8b 4c 24 30 8b 7c 24 34 8b 91 a0 00 00 00 8b 4c 24 18 0f b7 74 ca 18 8b 4c 24 14 8d 34 06 01 fe 29 ce 8b 7c 24 38 89 d9 c1 e9 02 <f3> a5 89 d9 83 e1 03 74 02 f3 a4 89 04 24 ba 02 00 00 00 89 54 <0>Kernel panic - not syncing: Fatal exception in interrupt (XEN) Domain 0 crashed: rebooting machine in 5 seconds. -- Stephen G. Traugott (KG6HDQ) Managing Partner, TerraLuna LLC stevegt@xxxxxxxxxxxxx -- http://www.t7a.org _______________________________________________ Xen-devel mailing list Xen-devel@xxxxxxxxxxxxxxxxxxx http://lists.xensource.com/xen-devel
|
 |
Lists.xenproject.org is hosted with RackSpace, monitoring our |