Re: [Xen-devel] PVH DomU panics on boot on Xen 4.5.0 whereas it was fine on 4.4.1

[Konrad Rzeszutek Wilk <konrad.wilk@xxxxxxxxxx>]

On Mon, Mar 16, 2015 at 11:08:50PM +0000, Ian Murray wrote:
> On 16/03/15 14:12, Konrad Rzeszutek Wilk wrote:
> > On Sun, Mar 15, 2015 at 09:34:16PM +0000, Ian Murray wrote:
> >> Hi,
> >>
> >> I have a domU guest that booted fine under Xen 4.4.1 with pvh=1 but now 
> >> fails to boot with it under Xen 4.5.0. Removing pvh=1, i.e. booting it as 
> >> traditional PV results in it booting fine.
> >>
> >> The only odd thing is that I had to compile with debug=y on Config.mk to 
> >> avoid a compiler warning that was causing compilation to fail outright. I 
> >> will create another mail about that. DomU is Ubuntu 14.10 and Dom0 is 
> >> 12.04.5
> >>
> > there were some incompatible changes in xen 4.5 in regards to
> > PVH which were then updated in Linux 3.19 (or was it 3.18?)
> >
> > I would recommend you rev up to the latest version of Linux.
> 
> I tried the mainline support kernel for Ubuntu (GNU/Linux
> 3.19.1-031901-generic x86_64) and it booted fine.
> 
> Thanks for the assistance and happy to assist if anyone wants to treat
> it as a regression.

Nah, it is labelled 'expermintal' for that exact reason - as we did
realized we made a mistake in Xen 4.4 that we ended up fixing
in Xen 4.5 - and the fixed it in Linux.

Sorry thought that it was not widely mentioned and it made your
day a bit sad. Was there an specific webpage you looked first for
help? (Asking so I can at least edit it to mention this).
> 
> 
> 
> >
> >> Here is the boot output when booting with pvh=1
> >>
> >> [    0.000000] Initializing cgroup subsys cpuset
> >> [    0.000000] Initializing cgroup subsys cpu
> >> [    0.000000] Initializing cgroup subsys cpuacct
> >> [    0.000000] Linux version 3.16.0-31-generic (buildd@batsu) (gcc version 
> >> 4.9.1 (Ubuntu 4.9.1-16ubuntu6) ) #41-Ubuntu SMP Tue Feb 10 15:24:04 UTC 
> >> 2015 (Ubuntu 3.16.0-31.41-generic 3.16.7-ckt5)
> >> [    0.000000] Command line: 
> >> root=UUID=edfcef2a-dcf1-4c77-ad69-22456606702e ro  nomodeset 
> >> xen-fbfront.video=16,1024,768
> >> [    0.000000] KERNEL supported cpus:
> >> [    0.000000]   Intel GenuineIntel
> >> [    0.000000]   AMD AuthenticAMD
> >> [    0.000000]   Centaur CentaurHauls
> >> [    0.000000] ACPI in unprivileged domain disabled
> >> [    0.000000] e820: BIOS-provided physical RAM map:
> >> [    0.000000] Xen: [mem 0x0000000000000000-0x000000003fffffff] usable
> >> [    0.000000] NX (Execute Disable) protection: active
> >> [    0.000000] DMI not present or invalid.
> >> [    0.000000] AGP: No AGP bridge found
> >> [    0.000000] e820: last_pfn = 0x40000 max_arch_pfn = 0x400000000
> >> [    0.000000] Scanning 1 areas for low memory corruption
> >> [    0.000000] init_memory_mapping: [mem 0x00000000-0x000fffff]
> >> [    0.000000] init_memory_mapping: [mem 0x3fe00000-0x3fffffff]
> >> [    0.000000] init_memory_mapping: [mem 0x3c000000-0x3fdfffff]
> >> [    0.000000] init_memory_mapping: [mem 0x00100000-0x3bffffff]
> >> [    0.000000] RAMDISK: [mem 0x023f6000-0x0589ffff]
> >> [    0.000000] NUMA turned off
> >> [    0.000000] Faking a node at [mem 0x0000000000000000-0x000000003fffffff]
> >> [    0.000000] Initmem setup node 0 [mem 0x00000000-0x3fffffff]
> >> [    0.000000]   NODE_DATA [mem 0x3fffb000-0x3fffffff]
> >> [    0.000000] Zone ranges:
> >> [    0.000000]   DMA      [mem 0x00001000-0x00ffffff]
> >> [    0.000000]   DMA32    [mem 0x01000000-0xffffffff]
> >> [    0.000000]   Normal   empty
> >> [    0.000000] Movable zone start for each node
> >> [    0.000000] Early memory node ranges
> >> [    0.000000]   node   0: [mem 0x00001000-0x0009ffff]
> >> [    0.000000]   node   0: [mem 0x00100000-0x3fffffff]
> >> [    0.000000] SFI: Simple Firmware Interface v0.81 
> >> http://simplefirmware.org
> >> [    0.000000] smpboot: Allowing 2 CPUs, 0 hotplug CPUs
> >> [    0.000000] PM: Registered nosave memory: [mem 0x000a0000-0x000fffff]
> >> [    0.000000] e820: [mem 0x40000000-0xffffffff] available for PCI devices
> >> [    0.000000] Booting paravirtualized kernel with PVH extensions on Xen
> >> [    0.000000] Xen version: 4.5.0
> >> [    0.000000] setup_percpu: NR_CPUS:256 nr_cpumask_bits:256 nr_cpu_ids:2 
> >> nr_node_ids:1
> >> [    0.000000] PERCPU: Embedded 28 pages/cpu @ffff88003fc00000 s83328 
> >> r8192 d23168 u1048576
> >> [    0.000000] Built 1 zonelists in Node order, mobility grouping on.  
> >> Total pages: 257930
> >> [    0.000000] Policy zone: DMA32
> >> [    0.000000] Kernel command line: 
> >> root=UUID=edfcef2a-dcf1-4c77-ad69-22456606702e ro  nomodeset 
> >> xen-fbfront.video=16,1024,768
> >> [    0.000000] PID hash table entries: 4096 (order: 3, 32768 bytes)
> >> [    0.000000] xsave: enabled xstate_bv 0x3, cntxt size 0x240
> >> [    0.000000] AGP: Checking aperture...
> >> [    0.000000] AGP: No AGP bridge found
> >> [    0.000000] Memory: 958828K/1048188K available (7743K kernel code, 
> >> 1197K rwdata, 3612K rodata, 1360K init, 1308K bss, 89360K reserved)
> >> [    0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=2, Nodes=1
> >> [    0.000000] Hierarchical RCU implementation.
> >> [    0.000000]     RCU dyntick-idle grace-period acceleration is enabled.
> >> [    0.000000]     RCU restricting CPUs from NR_CPUS=256 to nr_cpu_ids=2.
> >> [    0.000000]     Offload RCU callbacks from all CPUs
> >> [    0.000000]     Offload RCU callbacks from CPUs: 0-1.
> >> [    0.000000] RCU: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=2
> >> [    0.000000] NR_IRQS:16640 nr_irqs:288 16
> >> [    0.000000] xen:events: Using FIFO-based ABI
> >> [    0.000000] xen:events: Xen HVM callback vector for event delivery is 
> >> enabled
> >> [    0.000000] Console: colour dummy device 80x25
> >> [    0.000000] console [tty0] enabled
> >> [    0.000000] console [hvc0] enabled
> >> [    0.000000] allocated 4194304 bytes of page_cgroup
> >> [    0.000000] please try 'cgroup_disable=memory' option if you don't want 
> >> memory cgroups
> >> [    0.000000] installing Xen timer for CPU 0
> >> [    0.000000] tsc: Detected 2893.516 MHz processor
> >> [    0.004000] Calibrating delay loop (skipped), value calculated using 
> >> timer frequency.. 5787.03 BogoMIPS (lpj=11574064)
> >> [    0.004000] pid_max: default: 32768 minimum: 301
> >> [    0.004000] Security Framework initialized
> >> [    0.004000] AppArmor: AppArmor initialized
> >> [    0.004000] Yama: becoming mindful.
> >> [    0.004000] Dentry cache hash table entries: 131072 (order: 8, 1048576 
> >> bytes)
> >> [    0.004000] Inode-cache hash table entries: 65536 (order: 7, 524288 
> >> bytes)
> >> [    0.004000] Mount-cache hash table entries: 2048 (order: 2, 16384 bytes)
> >> [    0.004000] Mountpoint-cache hash table entries: 2048 (order: 2, 16384 
> >> bytes)
> >> [    0.004000] Initializing cgroup subsys memory
> >> [    0.004000] Initializing cgroup subsys devices
> >> [    0.004000] Initializing cgroup subsys freezer
> >> [    0.004000] Initializing cgroup subsys net_cls
> >> [    0.004000] Initializing cgroup subsys blkio
> >> [    0.004000] Initializing cgroup subsys perf_event
> >> [    0.004000] Initializing cgroup subsys net_prio
> >> [    0.004000] Initializing cgroup subsys hugetlb
> >> [    0.004000] ENERGY_PERF_BIAS: Set to 'normal', was 'performance'
> >> [    0.004000] ENERGY_PERF_BIAS: View and update with 
> >> x86_energy_perf_policy(8)
> >> [    0.004000] CPU: Physical Processor ID: 0
> >> [    0.004000] CPU: Processor Core ID: 0
> >> [    0.004000] Last level iTLB entries: 4KB 512, 2MB 8, 4MB 8
> >> [    0.004000] Last level dTLB entries: 4KB 512, 2MB 32, 4MB 32, 1GB 0
> >> [    0.004000] tlb_flushall_shift: 6
> >> [    0.030028] ftrace: allocating 29316 entries in 115 pages
> >> [    0.036106] cpu 0 spinlock event irq 17
> >> [    0.041478] Performance Events: unsupported p6 CPU model 42 no PMU 
> >> driver, software events only.
> >> [    0.042905] NMI watchdog: disabled (cpu0): hardware events not enabled
> >> [    0.042995] installing Xen timer for CPU 1
> >> [    0.043022] cpu 1 spinlock event irq 24
> >> [    0.043048] SMP alternatives: switching to SMP code
> >> [    0.057278] x86: Booted up 1 node, 2 CPUs
> >> [    0.057406] devtmpfs: initialized
> >> [    0.059043] evm: security.selinux
> >> [    0.059062] evm: security.SMACK64
> >> [    0.059075] evm: security.SMACK64EXEC
> >> [    0.059087] evm: security.SMACK64TRANSMUTE
> >> [    0.059099] evm: security.SMACK64MMAP
> >> [    0.059111] evm: security.ima
> >> [    0.059123] evm: security.capability
> >> [    0.060911] pinctrl core: initialized pinctrl subsystem
> >> [    0.060997] regulator-dummy: no parameters
> >> [    0.081449] RTC time: 165:165:165, date: 165/165/65
> >> [    0.081508] NET: Registered protocol family 16
> >> [    0.081533] xen:grant_table: Grant tables using version 1 layout
> >> [    0.081558] Grant table initialized
> >> [    0.081648] PCI: setting up Xen PCI frontend stub
> >> [    0.088163] ACPI: Interpreter disabled.
> >> [    0.088163] xen:balloon: Initialising balloon driver
> >> [    0.096046] xen_balloon: Initialising balloon driver
> >> [    0.096071] vgaarb: loaded
> >> [    0.096248] SCSI subsystem initialized
> >> [    0.096277] usbcore: registered new interface driver usbfs
> >> [    0.096277] usbcore: registered new interface driver hub
> >> [    0.096277] usbcore: registered new device driver usb
> >> [    0.096277] PCI: System does not support PCI
> >> [    0.096277] PCI: System does not support PCI
> >> [    0.096277] NetLabel: Initializing
> >> [    0.096277] NetLabel:  domain hash size = 128
> >> [    0.096277] NetLabel:  protocols = UNLABELED CIPSOv4
> >> [    0.096277] NetLabel:  unlabeled traffic allowed by default
> >> [    0.096277] Switched to clocksource xen
> >> [    0.103970] AppArmor: AppArmor Filesystem Enabled
> >> [    0.104082] pnp: PnP ACPI: disabled
> >> [    0.111189] NET: Registered protocol family 2
> >> [    0.111459] TCP established hash table entries: 8192 (order: 4, 65536 
> >> bytes)
> >> [    0.111520] TCP bind hash table entries: 8192 (order: 5, 131072 bytes)
> >> [    0.111563] TCP: Hash tables configured (established 8192 bind 8192)
> >> [    0.111606] TCP: reno registered
> >> [    0.111624] UDP hash table entries: 512 (order: 2, 16384 bytes)
> >> [    0.111651] UDP-Lite hash table entries: 512 (order: 2, 16384 bytes)
> >> [    0.111727] NET: Registered protocol family 1
> >> [    0.111846] Trying to unpack rootfs image as initramfs...
> >> [    0.160145] Freeing initrd memory: 53928K (ffff8800023f6000 - 
> >> ffff8800058a0000)
> >> [    0.160340] platform rtc_cmos: registered platform RTC device (no PNP 
> >> device found)
> >> [    0.160459] RAPL PMU detected, hw unit 2^-16 Joules, API unit is 2^-32 
> >> Joules, 3 fixed counters 163840 ms ovfl timer
> >> [    0.160558] microcode: CPU0 sig=0x206a7, pf=0x2, revision=0x26
> >> [    0.160578] microcode: CPU1 sig=0x206a7, pf=0x2, revision=0x26
> >> [    0.160808] microcode: Microcode Update Driver: v2.00 
> >> <tigran@xxxxxxxxxxxxxxxxxxxx>, Peter Oruba
> >> [    0.160856] Scanning for low memory corruption every 60 seconds
> >> [    0.161477] futex hash table entries: 512 (order: 3, 32768 bytes)
> >> [    0.161523] Initialise system trusted keyring
> >> [    0.161577] audit: initializing netlink subsys (disabled)
> >> [    0.181335] audit: type=2000 audit(1426454667.912:1): initialized
> >> [    0.181879] HugeTLB registered 2 MB page size, pre-allocated 0 pages
> >> [    0.183205] zpool: loaded
> >> [    0.183216] zbud: loaded
> >> [    0.183640] VFS: Disk quotas dquot_6.5.2
> >> [    0.183715] Dquot-cache hash table entries: 512 (order 0, 4096 bytes)
> >> [    0.184551] fuse init (API version 7.23)
> >> [    0.184727] msgmni has been set to 1978
> >> [    0.184790] Key type big_key registered
> >> [    0.185745] Key type asymmetric registered
> >> [    0.185757] Asymmetric key parser 'x509' registered
> >> [    0.185801] Block layer SCSI generic (bsg) driver version 0.4 loaded 
> >> (major 252)
> >> [    0.185894] io scheduler noop registered
> >> [    0.185905] io scheduler deadline registered (default)
> >> [    0.185958] io scheduler cfq registered
> >> [    0.186036] pci_hotplug: PCI Hot Plug PCI Core version: 0.5
> >> [    0.186061] pciehp: PCI Express Hot Plug Controller Driver version: 0.4
> >> [    0.186779] Serial: 8250/16550 driver, 32 ports, IRQ sharing enabled
> >> [    0.188858] Linux agpgart interface v0.103
> >> [    0.190320] brd: module loaded
> >> [    0.191147] loop: module loaded
> >> [    0.195014] libphy: Fixed MDIO Bus: probed
> >> [    0.195030] tun: Universal TUN/TAP device driver, 1.6
> >> [    0.195040] tun: (C) 1999-2004 Max Krasnyansky <maxk@xxxxxxxxxxxx>
> >> [    0.195156] PPP generic driver version 2.4.2
> >> [    0.195247] xen_netfront: Initialising Xen virtual ethernet driver
> >> [    0.199063] ehci_hcd: USB 2.0 'Enhanced' Host Controller (EHCI) Driver
> >> [    0.199088] ehci-pci: EHCI PCI platform driver
> >> [    0.199123] ehci-platform: EHCI generic platform driver
> >> [    0.199145] ohci_hcd: USB 1.1 'Open' Host Controller (OHCI) Driver
> >> [    0.199160] ohci-pci: OHCI PCI platform driver
> >> [    0.199177] ohci-platform: OHCI generic platform driver
> >> [    0.199194] uhci_hcd: USB Universal Host Controller Interface driver
> >> [    0.199258] i8042: PNP: No PS/2 controller found. Probing ports 
> >> directly.
> >> [    0.209793] blkfront: xvda: flush diskcache: enabled; persistent 
> >> grants: enabled; indirect descriptors: enabled;
> >> [    0.255889]  xvda: xvda1 xvda2 < xvda5 >
> >> [    1.209575] i8042: No controller found
> >> [    1.209830] mousedev: PS/2 mouse device common for all mice
> >> [    1.269874] rtc_cmos rtc_cmos: rtc core: registered rtc_cmos as rtc0
> >> [    1.270004] rtc_cmos: probe of rtc_cmos failed with error -38
> >> [    1.270096] device-mapper: uevent: version 1.0.3
> >> [    1.270226] device-mapper: ioctl: 4.27.0-ioctl (2013-10-30) 
> >> initialised: dm-devel@xxxxxxxxxx
> >> [    1.270261] Intel P-state driver initializing.
> >> [    1.270283] ledtrig-cpu: registered to indicate activity on CPUs
> >> [    1.270385] TCP: cubic registered
> >> [    1.270494] NET: Registered protocol family 10
> >> [    1.270706] NET: Registered protocol family 17
> >> [    1.270731] Key type dns_resolver registered
> >> [    1.270745] mce: Unable to init device /dev/mcelog (rc: -5)
> >> [    1.271098] Loading compiled-in X.509 certificates
> >> [    1.272466] Loaded X.509 cert 'Magrathea: Glacier signing key: 
> >> a112c18a448a560047ef1f4db74d497f61a45099'
> >> [    1.272528] registered taskstats version 1
> >> [    1.272947] traps: modprobe[60] trap invalid opcode ip:7fab5997f1aa 
> >> sp:7ffff0f5f6e8 error:0 in ld-linux-x86-64.so.2[7fab59967000+23000]
> >> [    1.273384] traps: modprobe[62] trap invalid opcode ip:7f34024f51aa 
> >> sp:7fff4beae188 error:0 in ld-linux-x86-64.so.2[7f34024dd000+23000]
> >> [    1.273719] Key type trusted registered
> >> [    1.273940] traps: modprobe[66] trap invalid opcode ip:7f4d34ded1aa 
> >> sp:7fffdcf3b8c8 error:0 in ld-linux-x86-64.so.2[7f4d34dd5000+23000]
> >> [    1.274230] traps: modprobe[68] trap invalid opcode ip:7f4990a341aa 
> >> sp:7fff1cb42358 error:0 in ld-linux-x86-64.so.2[7f4990a1c000+23000]
> >> [    1.274727] traps: modprobe[72] trap invalid opcode ip:7f019bace1aa 
> >> sp:7fffb5e59d48 error:0 in ld-linux-x86-64.so.2[7f019bab6000+23000]
> >> [    1.275024] traps: modprobe[74] trap invalid opcode ip:7fc8f4e2c1aa 
> >> sp:7fff46604c78 error:0 in ld-linux-x86-64.so.2[7fc8f4e14000+23000]
> >> [    1.275350] Key type encrypted registered
> >> [    1.275381] AppArmor: AppArmor sha1 policy hashing enabled
> >> [    1.275393] ima: No TPM chip found, activating TPM-bypass!
> >> [    1.275419] evm: HMAC attrs: 0x1
> >> [    1.275449] xenbus_probe_frontend: Device with no driver: device/vfb/0
> >> [    1.275461] xenbus_probe_frontend: Device with no driver: device/vkbd/0
> >> [    1.275556]   Magic number: 1:252:3141
> >> [    1.275592] /build/buildd/linux-3.16.0/drivers/rtc/hctosys.c: unable to 
> >> open rtc device (rtc0)
> >> [    1.275673] BIOS EDD facility v0.16 2004-Jun-25, 0 devices found
> >> [    1.275685] EDD information not available.
> >> [    1.276100] Freeing unused kernel memory: 1360K (ffffffff81d2d000 - 
> >> ffffffff81e81000)
> >> [    1.276118] Write protecting the kernel read-only data: 12288k
> >> [    1.276900] Freeing unused kernel memory: 436K (ffff880001793000 - 
> >> ffff880001800000)
> >> [    1.277030] Freeing unused kernel memory: 484K (ffff880001b87000 - 
> >> ffff880001c00000)
> >> [    1.277197] traps: init[1] trap invalid opcode ip:7f1c5a4441aa 
> >> sp:7fffe6457778 error:0 in ld-linux-x86-64.so.2[7f1c5a42c000+23000]
> >> [    1.277267] Kernel panic - not syncing: Attempted to kill init! 
> >> exitcode=0x00000004
> >> [    1.277267] 
> >> [    1.277286] CPU: 1 PID: 1 Comm: init Not tainted 3.16.0-31-generic 
> >> #41-Ubuntu
> >> [    1.277299]  ffff88003d9dfd30 ffff88003d9dfca0 ffffffff81782d0b 
> >> ffffffff81a7bbe8
> >> [    1.277318]  ffff88003d9dfd20 ffffffff8177ecca ffffffff00000010 
> >> ffff88003d9dfd30
> >> [    1.277337]  ffff88003d9dfcc8 ffff88003da80000 0000000000000004 
> >> ffff88003da804a8
> >> [    1.277357] Call Trace:
> >> [    1.277367]  [<ffffffff81782d0b>] dump_stack+0x45/0x56
> >> [    1.277378]  [<ffffffff8177ecca>] panic+0xc6/0x1fa
> >> [    1.277389]  [<ffffffff81072ec1>] do_exit+0xaa1/0xab0
> >> [    1.277400]  [<ffffffff81072f53>] do_group_exit+0x43/0xc0
> >> [    1.277410]  [<ffffffff810832ff>] get_signal_to_deliver+0x29f/0x8d0
> >> [    1.277423]  [<ffffffff810125b8>] do_signal+0x48/0xbf0
> >> [    1.277434]  [<ffffffff811e8f5b>] ? do_execve_common.isra.24+0x65b/0x750
> >> [    1.277446]  [<ffffffff811e8f5b>] ? do_execve_common.isra.24+0x65b/0x750
> >> [    1.281074]  [<ffffffff810131c9>] do_notify_resume+0x69/0xb0
> >> [    1.281074]  [<ffffffff8178bb22>] retint_signal+0x48/0x86
> >> [    1.281074] Kernel Offset: 0x0 from 0xffffffff81000000 (relocation 
> >> range: 0xffffffff80000000-0xffffffffbfffffff)
> >>
> >> Thanks for reading,
> >>
> >> Ian.
> >>
> >> _______________________________________________
> >> Xen-devel mailing list
> >> Xen-devel@xxxxxxxxxxxxx
> >> http://lists.xen.org/xen-devel
> 

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxx
http://lists.xen.org/xen-devel