RE: Dom0 crashing when built with c/s 881 (Was: [Xen-devel] blktap2: need more than 3 values to unpack)

[Dan Magenheimer <dan.magenheimer@xxxxxxxxxx>]

Well, hmmm.

>  1. It only adds extra functionality which by default would 
> not do anything.
> Just reading the patch I can see it's harmless by itself as it merely
> introduces a new notifier chain which noone registers on.

I won't pretend to understand the code, but I did
reproduce the problem with 881 by itself and with
889 with 881 reverted.

>  2. Changeset 882 depends on 881. If you revert just 881 then 
> the tree will not build.

This doesn't appear to be the case for me, though
I'm certainly willing to believe there's something
wrong with my process:

1) I build* 889 successfully.  (It fails to run
   and I reboot to Linux.)
2) I revert 881 with

# cd .../xen-unstable.hg
# cd linux-2.6.18-xen.hg
# hg diff -r880 -r881 | patch -p1 -R
# # the patch output shows the same four files as 881
# cd ..

3) I insert a syntax error in include/linux/device.h
   and build* again.  Build fails (as expected).
4) I remove the syntax error and build* again,
   build succeeds, installs and reboots fine.

* My build process is:
# cd .../xen-unstable.hg
# KERNELS=linux-2.6-xen0 make linux-2.6-xen-clean
# yes "" | KERNELS=linux-2.6-xen0 make \
  linux-2.6-xen-config CONFIGMODE=oldconfig
# KERNELS=linux-2.6-xen0 make linux-2.6-xen-build
# KERNELS=linux-2.6-xen0 make linux-2.6-xen-install

and I always check /boot/vmlinux-2.6.18.8-xen to
ensure the install succeeded.

> -----Original Message-----
> From: Keir Fraser [mailto:keir.fraser@xxxxxxxxxxxxx]
> Sent: Wednesday, June 03, 2009 2:47 PM
> To: Dan Magenheimer; Dulloor; Isaku Yamahata
> Cc: Xen-Devel (E-mail)
> Subject: Re: Dom0 crashing when built with c/s 881 (Was: [Xen-devel]
> blktap2: need more than 3 values to unpack)
> 
> 
> Changeset 881 is "pcie io space multiplex: backport of bus event
> notification patch", right? I don't believe that you are 
> reverting just that
> patch, for two reasons:
>  1. It only adds extra functionality which by default would 
> not do anything.
> Just reading the patch I can see it's harmless by itself as it merely
> introduces a new notifier chain which noone registers on.
>  2. Changeset 882 depends on 881. If you revert just 881 then 
> the tree will
> not build.
> 
>  -- Keir
> 
> On 03/06/2009 21:15, "Dan Magenheimer" 
> <dan.magenheimer@xxxxxxxxxx> wrote:
> 
> >>> Is it better to have the new option added by
> >>> c/s(CONFIG_PCI_IOMULTI) default 'n' ?
> > 
> > Turning off that option doesn't stop my dom0 from crashing.
> > Reverting 881 DOES stop my dom0 from crashing.  Attached are
> > the boot output for fail and success (the only difference
> > between the two dom0's is the success has 881 reverted).
> > 
> > Of note, the fail log has three lines of the type:
> > 
> > "Driver 'xxx' needs updating - please use bus_type methods"
> > 
> > (xxx = usbfs, hub, usb) and the success log has a line that says:
> > 
> > "PCI IO multiplexer device installed"
> > 
> > which is missing in the fail log.
> > 
> >> -----Original Message-----
> >> From: Keir Fraser [mailto:keir.fraser@xxxxxxxxxxxxx]
> >> Sent: Wednesday, June 03, 2009 12:45 PM
> >> To: Dulloor
> >> Cc: Isaku Yamahata; Dan Magenheimer; Xen-Devel (E-mail)
> >> Subject: Re: Dom0 crashing when built with c/s 881 (Was: 
> [Xen-devel]
> >> blktap2: need more than 3 values to unpack)
> >> 
> >> 
> >> Depends on the feature's impact when unused.
> >> 
> >>  -- Keir
> >> 
> >> On 03/06/2009 18:57, "Dulloor" <dulloor@xxxxxxxxx> wrote:
> >> 
> >>> Is it better to have the new option added by
> >> c/s(CONFIG_PCI_IOMULTI) default
> >>> 'n' ?
> >>> 
> >>> -dulloor
> >>> 
> >>> On Wed, Jun 3, 2009 at 1:48 PM, Keir Fraser
> >> <keir.fraser@xxxxxxxxxxxxx> wrote:
> >>>> Perhaps post a diff of boot output with c/s 880 vs c/s
> >> 881. I've cc'ed
> >>>> Isaku, who submitted the patch that's causing your problem.
> >>>> 
> >>>>  -- Keir
> >>>> 
> >>>> On 03/06/2009 18:35, "Dan Magenheimer"
> >> <dan.magenheimer@xxxxxxxxxx> wrote:
> >>>> 
> >>>>> OK, it appears that linux-2.6.18-xen.hg c/s 881 is causing
> >>>>> my dom0 to crash.  Dom0 boots successfully at 880 and fails
> >>>>> with 881 or anything after.
> >>>>> 
> >>>>> Any ideas?
> >>>>> 
> >>>>>> -----Original Message-----
> >>>>>> From: Dan Magenheimer
> >>>>>> Sent: Tuesday, June 02, 2009 10:45 PM
> >>>>>> To: Dan Magenheimer; Keir Fraser; Dutch Meyer; 
> Xen-Devel (E-mail)
> >>>>>> Subject: RE: [Xen-devel] blktap2: need more than 3
> >> values to unpack
> >>>>>> 
> >>>>>> 
> >>>>>> Followup: my dom0 boots failed with 888, boots fine with 876,
> >>>>>> then to ensure no pilot error, I rebuilt 888 again and it
> >>>>>> failed again.
> >>>>>> 
> >>>>>>> -----Original Message-----
> >>>>>>> From: Dan Magenheimer
> >>>>>>> Sent: Tuesday, June 02, 2009 5:50 PM
> >>>>>>> To: Keir Fraser; Dutch Meyer; Xen-Devel (E-mail)
> >>>>>>> Subject: RE: [Xen-devel] blktap2: need more than 3
> >> values to unpack
> >>>>>>> 
> >>>>>>> 
> >>>>>>> Thanks.  It is indeed a pilot error on my part but a bit
> >>>>>>> more bizarre.  I apparently have a linux-2.6-xen.hg directory
> >>>>>>> as both a sister and a child to xen-unstable.hg.  In this
> >>>>>>> case the build apparently chooses the child.  I was
> >>>>>>> looking at and modifying the un-updated child so
> >>>>>>> blktap2 wasn't even present yet.  Removing the child
> >>>>>>> causes the sibling to build.  BUT! Now dom0 is
> >>>>>>> crashing early on during boot.  (This is an Intel
> >>>>>>> Weybridge box.)  I'll look into this further tomorrow.
> >>>>>>> 
> >>>>>>> ata2: SATA link up 1.5 Gbps (SStatus 113 SControl 300)
> >>>>>>> ata2.00: ATAPI, max UDMA/100
> >>>>>>> ata2.00: configured for UDMA/100
> >>>>>>> scsi2 : ahci
> >>>>>>> ata3: SATA link down (SStatus 0 SControl 300)
> >>>>>>> scsi3 : ahci
> >>>>>>> ata4: SATA link down (SStatus 0 SControl 300)
> >>>>>>>   Vendor: ATA       Model: ST3320620AS       Rev: 3.AA
> >>>>>>>   Type:   Direct-Access                      ANSI SCSI
> >> revision: 05
> >>>>>>> ata1: EH pending after completion, repeating EH (cnt=4)
> >>>>>>>   Vendor: LITE-ON   Model: DVDRW LH-20A1S    Rev: 9L03
> >>>>>>>   Type:   CD-ROM                             ANSI SCSI
> >> revision: 05
> >>>>>>> (XEN) PCI add device 00:1f.5
> >>>>>>> ata_piix 0000:00:1f.5: MAP [ P0 P2 P1 P3 ]
> >>>>>>> ACPI: PCI Interrupt 0000:00:1f.5[B] -> GSI 19 (level,
> >> low) -> IRQ 16
> >>>>>>> ata5: SATA max UDMA/133 cmd 0x0 ctl 0x2 bmdma 0x0 irq 16
> >>>>>>> ata6: SATA max UDMA/133 cmd 0x0 ctl 0x2 bmdma 0x0 irq 16
> >>>>>>> scsi4 : ata_piix
> >>>>>>> scsi5 : ata_piix
> >>>>>>> device-mapper: ioctl: 4.7.0-ioctl (2006-06-24) initialised:
> >>>>>>> dm-devel@xxxxxxxxxx
> >>>>>>> Kernel panic - not syncing: Attempted to kill init!
> >>>>>>>  (XEN) Domain 0 crashed: rebooting machine in 5 seconds.
> >>>>>>> 
> >>>>>>>> -----Original Message-----
> >>>>>>>> From: Keir Fraser [mailto:keir.fraser@xxxxxxxxxxxxx]
> >>>>>>>> Sent: Tuesday, June 02, 2009 4:45 PM
> >>>>>>>> To: Dan Magenheimer; Dutch Meyer; Xen-Devel (E-mail)
> >>>>>>>> Subject: Re: [Xen-devel] blktap2: need more than 3 values
> >>>>>> to unpack
> >>>>>>>> 
> >>>>>>>> 
> >>>>>>>> Probably you had an old .config hanging around in your build
> >>>>>>>> tree somewhere.
> >>>>>>>> c/s 889 should fix this for a fresh build.
> >>>>>>>> 
> >>>>>>>>  -- Keir
> >>>>>>>> 
> >>>>>>>> On 02/06/2009 18:33, "Dan Magenheimer"
> >>>>>>>> <dan.magenheimer@xxxxxxxxxx> wrote:
> >>>>>>>> 
> >>>>>>>>> Thanks.  Looks like a partial configuration patch 
> got checked
> >>>>>>>>> in for blktap2 (cs 886)?  CONFIG_XEN_BLKDEV_TAP2 must be
> >>>>>>> configured
> >>>>>>>>> but afaict is not turned on by default (yet?).  So a fresh
> >>>>>>>>> xen-unstable tip doesn't build the blktap2 driver.  See:
> >>>>>>>>> 
> >>>>>>>>> 
> >>>>>> 
> http://xenbits.xensource.com/linux-2.6.18-xen.hg?rev/3e01555dd227
> >>>>>>>>> 
> >>>>>>>>> (I'm guessing since this was submitted by Isaku that blktap2
> >>>>>>>>> shouldn't be the default on ia64?)
> >>>>>>>>> 
> >>>>>>>>> Should CONFIG_XEN_BLKDEV_TAP2 be turned on by 
> default, instead
> >>>>>>>>> of CONFIG_XEN_BLKDEV_TAP, at least on x86?
> >>>>>>>>> 
> >>>>>>>>> I tried modifying
> >>>>>>>>> 
> >>>>>>>>> linux-2.6.18-xen.hg/buildconfigs/linux-defconfig_xen0_x86_32
> >>>>>>>>> 
> >>>>>>>>> (and also
> >>>>>>>>> 
> >>>>>>>>> linux-2.6.18-xen.hg/buildconfigs/linux-defconfig_xen_x86_32)
> >>>>>>>>> 
> >>>>>>>>> followed by:
> >>>>>>>>> 
> >>>>>>>>> KERNELS=linux-2.6-xen0 make linux-2.6-xen-config
> >>>>>>>> CONFIGMODE=oldconfig
> >>>>>>>>> 
> >>>>>>>>> (I don't need or want to go through a manual config process)
> >>>>>>>>> 
> >>>>>>>>> but BLKDEV_TAP is always selected, not BLKDEV_TAP2.
> >>>>>>>>> 
> >>>>>>>>> Finally, I resorted to manually changing
> >>>>>>>>> 
> >>>>>>>>> linux-2.6.18-xen.hg/drivers/xen/Kconfig
> >>>>>>>>> 
> >>>>>>>>> and this succeeds in turning it on, but it just reverses the
> >>>>>>>>> above checked-in patch, so I suspect that's not the right
> >>>>>>>>> answer either.
> >>>>>>>>> 
> >>>>>>>>>> -----Original Message-----
> >>>>>>>>>> From: Dutch Meyer [mailto:dmeyer@xxxxxxxxx]
> >>>>>>>>>> Sent: Tuesday, June 02, 2009 9:05 AM
> >>>>>>>>>> To: Dan Magenheimer
> >>>>>>>>>> Subject: RE: [Xen-devel] blktap2: need more than 3
> >>>>>>> values to unpack
> >>>>>>>>>> 
> >>>>>>>>>> 
> >>>>>>>>>> I think that you don't have the blktap2 driver loaded in
> >>>>>>>>>> dom0.  A clean
> >>>>>>>>>> build/install of the dom0 kernel image should sort
> >> you out.  If
> >>>>>>>>>> drivers/xen/blktap2 is compiled in it should be setting up
> >>>>>>>>>> these paths.
> >>>>>>>>>> 
> >>>>>>>>>> Let me know if that fixes things and I'll make python
> >>>>>>> spit out more
> >>>>>>>>>> meaningful errors, otherwise we can try to figure out the
> >>>>>>>>>> blktap2 kernel
> >>>>>>>>>> code isn't working.
> >>>>>>>>>> 
> >>>>>>>>>> --Dutch
> >>>>>>>>>> 
> >>>>>>>>>> On Tue, 2 Jun 2009, Dan Magenheimer wrote:
> >>>>>>>>>> 
> >>>>>>>>>>> It replies with "didn't find blktap-control in /proc/misc"
> >>>>>>>>>>> 
> >>>>>>>>>>> If that fails, perhaps the path doesn't exist, 
> but I looked
> >>>>>>>>>>> and /sys/class/blktap2 doesn't exist.
> >>>>>>>>>>> 
> >>>>>>>>>>>> -----Original Message-----
> >>>>>>>>>>>> From: Dutch Meyer [mailto:dmeyer@xxxxxxxxx]
> >>>>>>>>>>>> Sent: Monday, June 01, 2009 10:37 PM
> >>>>>>>>>>>> To: Dan Magenheimer
> >>>>>>>>>>>> Subject: RE: [Xen-devel] blktap2: need more than 3
> >>>>>>>> values to unpack
> >>>>>>>>>>>> 
> >>>>>>>>>>>> 
> >>>>>>>>>>>> Can you try this from the command line:
> >>>>>>>>>>>> 
> >>>>>>>>>>>>      tapdisk2 -n aio:/pathto/file.img
> >>>>>>>>>>>> 
> >>>>>>>>>>>> If successful, this will create your aio device 
> and print a
> >>>>>>>>>>>> /dev device
> >>>>>>>>>>>> associated with it.
> >>>>>>>>>>>> 
> >>>>>>>>>>>> In that case you'll then be able to remove it with:
> >>>>>>>>>>>> 
> >>>>>>>>>>>>      echo 1 > /sys/class/blktap2/<disk>/remove
> >>>>>>>>>>>> 
> >>>>>>>>>>>> Where <disk> will be obvious from the output of the
> >>>>>>>>>> tapdisk2 command.
> >>>>>>>>>>>> 
> >>>>>>>>>>>> However, I expect that this will fail.
> >>>>>>>>>>>> 
> >>>>>>>>>>>> --Dutch
> >>>>>>>>>>>> 
> >>>>>>>>>>>> On Mon, 1 Jun 2009, Dan Magenheimer wrote:
> >>>>>>>>>>>> 
> >>>>>>>>>>>> Then I might be able to help, but I'm not sure how to
> >>>>>>>>>>>> reproduce it.  If
> >>>>>>>>>>>> you send a log file and config for this latter error I'll
> >>>>>>>>>>>> take a look.
> >>>>>>>>>>>> 
> >>>>>>>>>>>> Here ya go.
> >>>>>>>>>>>> 
> >>>>>>>>>>>> Thanks,
> >>>>>>>>>>>> Dan
> >>>>>>>>>>>> 
> >>>>>>>>>>>> -----Original Message-----
> >>>>>>>>>>>> From: Dutch Meyer [mailto:dmeyer@xxxxxxxxx]
> >>>>>>>>>>>> Sent: Monday, June 01, 2009 8:32 PM
> >>>>>>>>>>>> To: Dan Magenheimer
> >>>>>>>>>>>> Cc: Xen-Devel (E-mail)
> >>>>>>>>>>>> Subject: Re: [Xen-devel] blktap2: need more than 3
> >>>>>>>>>> values to unpack
> >>>>>>>>>>>> 
> >>>>>>>>>>>> 
> >>>>>>>>>>>> The tap:aio:/pathto/file.img syntax that you're
> >>>>>> using in your
> >>>>>>>>>>>> config was
> >>>>>>>>>>>> changed before blktap2 was introduced.
> >>>>>>>>>>>> tap:tapdisk:aio:/pathto/file.img is
> >>>>>>>>>>>> apparently the correct syntax now, though the
> >> README didn't
> >>>>>>>>>>>> get updated to
> >>>>>>>>>>>> reflect this.  Our blktap2 documentation is no better -
> >>>>>>>>>> I'll try to
> >>>>>>>>>>>> remedy that this week.
> >>>>>>>>>>>> 
> >>>>>>>>>>>> 
> >>>>>>>>>>>> If you're still seeing this error:
> >>>>>>>>>>>>     "Error: 'file' object has no attribute 'find'"
> >>>>>>>>>>>> 
> >>>>>>>>>>>> Then I might be able to help, but I'm not sure how to
> >>>>>>>>>>>> reproduce it.  If
> >>>>>>>>>>>> you send a log file and config for this latter error I'll
> >>>>>>>>>>>> take a look.
> >>>>>>>>>>>> Yang seems to be reporting the same thing in
> >>>>>> another thread.
> >>>>>>>>>>>> 
> >>>>>>>>>>>> --Dutch
> >>>>>>>>>>>> 
> >>>>>>>>>>>> On Mon, 1 Jun 2009, Dan Magenheimer wrote:
> >>>>>>>>>>>> 
> >>>>>>>>>>>> Hmmm... trying blktap2 for the first time, using 19682.
> >>>>>>>>>>>> I had thought that the syntax hadn't changed, but I am
> >>>>>>>>>>>> getting what appears to be a parsing error on
> >> my vbd line.
> >>>>>>>>>>>> 
> >>>>>>>>>>>> "ValueError: need more than 3 values to unpack"
> >>>>>>>>>>>> 
> >>>>>>>>>>>> Thinking maybe that "w!" was the culprit, I changed
> >>>>>>>>>>>> it to "w" with no change in result.
> >>>>>>>>>>>> 
> >>>>>>>>>>>> Looking at the python code that generated the error,
> >>>>>>>>>>>> I tried to figure out the syntax by experimentation
> >>>>>>>>>>>> but without luck.  I tried:
> >>>>>>>>>>>> 
> >>>>>>>>>>>> tap:tapdisk:aio:/pathto/file.img
> >>>>>>>>>>>> 
> >>>>>>>>>>>> but got "Error: 'file' object has no attribute 'find'"
> >>>>>>>>>>>> 
> >>>>>>>>>>>> To see if I could use the old blktap, I tried
> >>>>>>>>>>>> 
> >>>>>>>>>>>> tap:tapdisk:ioemu:/pathto/file.img
> >>>>>>>>>>>> 
> >>>>>>>>>>>> but got the dreaded "Error: Device 768 (tap)
> >> could not be
> >>>>>>>>>>>> connected. Hotplug scripts not working"
> >>>>>>>>>>>> 
> >>>>>>>>>>>> Am I missing something in the syntax for blktap2?
> >>>>>>>>>>>> Is there a how-to or readme I didn't find?  Or
> >>>>>>>>>>>> is there some required dependency I don't know about
> >>>>>>>>>>>> that is missing?
> >>>>>>>>>>>> 
> >>>>>>>>>>>> I thought maybe I had a bad install, so rebuilt and
> >>>>>>>>>>>> reinstalled with the same result.
> >>>>>>>>>>>> 
> >>>>>>>>>>>> xend.log and config file attached.
> >>>>>>>>>>>> 
> >>>>>>>>>>>> Thanks,
> >>>>>>>>>>>> Dan
> >>>>>>>>>>>> 
> >>>>>>>>>>>> P.S. I am trying blktap2 because both blktap and
> >>>>>>>>>>>> file-backed fail.  Blktap sometimes reads garbage
> >>>>>>>>>>>> from the file and
> >>>>>>>>>>>> 
> >>>>>>>>>>>> _______________________________________________
> >>>>>>>>>>>> Xen-devel mailing list
> >>>>>>>>>>>> Xen-devel@xxxxxxxxxxxxxxxxxxx
> >>>>>>>>>>>> http://lists.xensource.com/xen-devel
> >>>>>>>>>>>> 
> >>>>>>>>>>>> 
> >>>>>>>>>>> 
> >>>>>>>>>> 
> >>>>>>>>> 
> >>>>>>>>> _______________________________________________
> >>>>>>>>> Xen-devel mailing list
> >>>>>>>>> Xen-devel@xxxxxxxxxxxxxxxxxxx
> >>>>>>>>> http://lists.xensource.com/xen-devel
> >>>>>>>> 
> >>>>>>>> 
> >>>>>>>> 
> >>>>>>> 
> >>>>>>> _______________________________________________
> >>>>>>> Xen-devel mailing list
> >>>>>>> Xen-devel@xxxxxxxxxxxxxxxxxxx
> >>>>>>> http://lists.xensource.com/xen-devel
> >>>>>>> 
> >>>> 
> >>>> 
> >>>> 
> >>>> _______________________________________________
> >>>> Xen-devel mailing list
> >>>> Xen-devel@xxxxxxxxxxxxxxxxxxx
> >>>> http://lists.xensource.com/xen-devel
> >>> 
> >>> 
> >> 
> >> 
> >> 
> >> _______________________________________________
> >> Xen-devel mailing list
> >> Xen-devel@xxxxxxxxxxxxxxxxxxx
> >> http://lists.xensource.com/xen-devel
> >> 
> 
> 
>

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-devel