[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [Xen-devel] pvops DomU kernels [was Re: "Hotplug Scripts not working ..." error on jaunty]


  • To: Dulloor <dulloor@xxxxxxxxx>
  • From: Boris Derzhavets <bderzhavets@xxxxxxxxx>
  • Date: Thu, 22 Oct 2009 00:44:56 -0700 (PDT)
  • Cc: Xen-devel <xen-devel@xxxxxxxxxxxxxxxxxxx>, "Marc - A. Dahlhaus" <mad@xxxxxx>
  • Delivery-date: Thu, 22 Oct 2009 00:45:56 -0700
  • Domainkey-signature: a=rsa-sha1; q=dns; c=nofws; s=s1024; d=yahoo.com; h=Message-ID:X-YMail-OSG:Received:X-Mailer:Date:From:Subject:To:Cc:In-Reply-To:MIME-Version:Content-Type; b=FZhRqVy+GodqtKO5fOrMjy7NanpSOjNr9o05d+7+42yunP5+kX3WefXnsWnhgjdZQCpIZgZJSKuHUuV/O6rS0H+3JgbjCvaxT+JQkJBp2bhZPLz45a2Lqb/VWTRCOAL4ZEoBfmb1uLgJcC1O4PG04bzJQp3itCBprx0ZMSnLykc=;
  • List-id: Xen developer discussion <xen-devel.lists.xensource.com>

> The problem is with the pvops DomU kernel.

It works not only for xenified ( Xen Dom0 support forward ported kernels)  aka Suse
2.6.27,2.6.29.4, 2.6.31.4 kernels

F11,F10,Ubuntu 8.10,9.04 Servers PV DomUs have been setup with no problems at the same Dom0. Mentioned  kernels have Xen DomU Support via mainline Linux,
but they are not obviously pvops enabled kernels.


Boris.

--- On Thu, 10/22/09, Dulloor <dulloor@xxxxxxxxx> wrote:

From: Dulloor <dulloor@xxxxxxxxx>
Subject: [Xen-devel] pvops DomU kernels [was Re: "Hotplug Scripts not working ..." error on jaunty]
To: "Boris Derzhavets" <bderzhavets@xxxxxxxxx>
Cc: "Xen-devel" <xen-devel@xxxxxxxxxxxxxxxxxxx>, "Marc - A. Dahlhaus" <mad@xxxxxx>
Date: Thursday, October 22, 2009, 1:56 AM

console/network/everything is fine with the 2.6.18 DomU (on pvops DomU and on Ubuntu 9.04 server). I assume it would be so with the any other forward port too (2.6.27, for instance).

The problem is with the pvops DomU kernel.
1. With Linus' git tree (CONFIG_XEN and other DomU options compiled in), the kernel crashes with the following (in xm dmesg) :
(XEN) mm.c:840:d33 Error getting mfn 18c3 (pfn 21603) from L1 entry 80000000018c3061 for l1e_owner=33, pg_owner=33
(XEN) traps.c:465:d33 Unhandled invalid opcode fault/trap [#6] on VCPU 0 [ec=0000]
(XEN) domain_crash_sync called from entry.S
(XEN) Domain 33 (vcpu#0) crashed on cpu#7:
(XEN) ----[ Xen-3.5-unstable  x86_64  debug=y  Not tainted ]----
(XEN) CPU:    7

2. With Jeremy's git, i.e the same kernel as Dom0 (but with CONFIG_XEN and other DomU options compiled in), the kernel just freezes in init_intel function (start_kernel->...->identify_boot_cpu->...->c_init). To be more precise, in detect_extended_topology function. Also, for the DomU, the init_intel function in "arch/x86/kernel/cpu/intel.c" gets executed rather than the function in "setup-xen.c". I am not well-versed the setup code in pvops. Does that look fine ?

I have attached the configs for both 1 and 2.

thanks
dulloor

PS : Also, DomU compilation (with just DomU options compiled in) fails with the following error, if CONFIG_SMP is not defined -
arch/x86/kernel/pvclock.c: In function âpvclock_clocksource_vreadâ:
arch/x86/kernel/pvclock.c:172: error: implicit declaration of function âfix_to_virtâ
arch/x86/kernel/pvclock.c:172: error: âFIX_PVCLOCK_TIME_INFOâ undeclared (first use in this function)
arch/x86/kernel/pvclock.c:172: error: (Each undeclared identifier is reported only once


On Tue, Oct 20, 2009 at 11:23 AM, Boris Derzhavets <bderzhavets@xxxxxxxxx> wrote:
I used to work with NAT for DomUs only on Fedoras 12,11,10,8 (libvirtd & dnmask).
Due to absence libvirt ->xen driver on Ubuntu's Servers 9.04,8.10 i always configured
xen bridge via /etc/xen/xend-config.sxp. It works fine.

--- On Tue, 10/20/09, Boris Derzhavets <bderzhavets@xxxxxxxxx> wrote:

From: Boris Derzhavets <bderzhavets@xxxxxxxxx>

Subject: Re: [Xen-devel] Re: "Hotplug Scripts not working ..." error on jaunty
To: "Dulloor" <dulloor@xxxxxxxxx>

Cc: "Xen-devel" <xen-devel@xxxxxxxxxxxxxxxxxxx>, "Marc - A. Dahlhaus" <mad@xxxxxx>
Date: Tuesday, October 20, 2009, 10:39 AM


You have to take care of hvc0 like tty1 entry under /etc/event.d on:
1. Ubuntu's <= 9.04 pv guest
# cd /etc/event.d
# cp tty1 hvc0
and edit hvc0 file correspondently
2. /etc/inittab on CentOS 5.3 DomU
should have entry
hvc0:23:respawn:...:hvc0
like for tty1,tty2 and etc

and so on,  console=hvc0 at /boot/vmlinuz-xen  module's line requires
activating hvc0 via guest's specific location, view for instance :-

http://bderzhavets.wordpress.com/2009/05/21/install-ubuntu-jaunty-pv-hvm-domus-at-xen-3-4-dom0-kernel-2-6-30-rc6-tip/

or just tell exactly which one (linux) PV DomU is causing you trouble with serial console.

Boris.
P.S. F11, U 9.04 Server , OpenSuse 11.1(2)  PV DomUs may loaded via vfb as well,but U 9.10 Server (beta) requires serial console




--- On Tue, 10/20/09, Dulloor <dulloor@xxxxxxxxx> wrote:

From: Dulloor <dulloor@xxxxxxxxx>
Subject: Re: [Xen-devel] Re: "Hotplug Scripts not working ..." error on jaunty
To: "Boris Derzhavets" <bderzhavets@xxxxxxxxx>
Cc: "Xen-devel" <xen-devel@xxxxxxxxxxxxxxxxxxx>, "Marc - A. Dahlhaus" <mad@xxxxxx>
Date: Tuesday, October 20, 2009, 6:22 AM

Boris,

The "hotplug" error is gone with 9.04 server. The VMs boot fine (atleast per logs and xm list).

I am not able to access VM via console(hvc0) though. I have tried everything out there in your blogs, but the console just freezes at this same point :
"
[    0.010000] Initializing cgroup subsys memory
[    0.010000] Initializing cgroup subsys freezer
"
Any clues on how to go about debugging this ?

I use the default network-nat configuration for VMs(with addresses from 10.0.0.1/24). That doesn't work either. Is that expected too ?
I am sure the bridge configuration should work though (will try tomorrow).

-dulloor


On Sun, Oct 18, 2009 at 3:48 AM, Boris Derzhavets <bderzhavets@xxxxxxxxx> wrote:
> I will try 9.04-Server (not sure if it will help though)
1. I believe it will help.
2. Bruce's  Edge feed would be also very much wanted.

Boris.


--- On Sun, 10/18/09, Dulloor <dulloor@xxxxxxxxx> wrote:

From: Dulloor <dulloor@xxxxxxxxx>
Subject: Re: [Xen-devel] Re: "Hotplug Scripts not working ..." error on jaunty
To: "Boris Derzhavets" <bderzhavets@xxxxxxxxx>
Cc: "Marc - A. Dahlhaus" <mad@xxxxxx>, "Xen-devel" <xen-devel@xxxxxxxxxxxxxxxxxxx>
Date: Sunday, October 18, 2009, 3:09 AM


I use xen-unstable (at the tip again). I have been following your blog. Thinking of it, the only difference is I use 9.04-Desktop-amd64 for my base installation. I will try 9.04-Server (not sure if it will help though). Btw, these VM images were working fine with 8.04 installation.

-dulloor


On Sun, Oct 18, 2009 at 3:00 AM, Boris Derzhavets <bderzhavets@xxxxxxxxx> wrote:
Please, tell how you setup Xen 3.4.1 or Xen  Unstable on 9.04 server ?

Boris.

P.S. Working sample:-

http://bderzhavets.wordpress.com/2009/06/25/setup-xen-3-4-1-dom0-on-top-of-ubuntu-9-04-server-via-marc-a-dahlhauss-udev-patch/



--- On Sun, 10/18/09, Dulloor <dulloor@xxxxxxxxx> wrote:

From: Dulloor <dulloor@xxxxxxxxx>
Subject: Re: [Xen-devel] Re: "Hotplug Scripts not working ..." error on jaunty

To: "Boris Derzhavets" <bderzhavets@xxxxxxxxx>
Cc: "Marc - A. Dahlhaus" <mad@xxxxxx>, "Xen-devel" <xen-devel@xxxxxxxxxxxxxxxxxxx>
Date: Sunday, October 18, 2009, 1:37 AM


I am at the tip of xen/master branch of Jeremy's tree.

commit 2afad356210ac35cbbc81904e0ac8b514b7f6212
Merge: 9fa9003... 907d20f...
Author: Jeremy Fitzhardinge <jeremy.fitzhardinge@xxxxxxxxxx>
Date:   Thu Oct 15 11:34:28 2009 -0700

    Merge branch 'xen/vsyscall' into xen/master

    * xen/vsyscall:
      x86/vsyscall: fix 32-bit compile

-dulloor

On Sun, Oct 18, 2009 at 12:49 AM, Boris Derzhavets <bderzhavets@xxxxxxxxx> wrote:
> Boris Derzhavets schrieb:
>>
>> By some reasons i have to run  on regular basis run ./xen-hotplug-cleanup
>> on two Xen 3.4.1 on top of Ubuntu 9.04 Servers patched systems to
>> re-enable ability start DomUs without hotplug scripts failure.
>>
>> Boris.

It  was written on June 26, 2009 and was connected with some particular 9.04 instance.
Since then i did several Xen Host installs via official 3.4.1  tarball and via xen-unstable.hg  but, every time the most recent JF's pvops kernel had been built to
support Dom0. Problem with hot-plug scripts was gone far away.
My question :-
What kernel is supporting your Dom0 ?

Boris.
P.S. Seems to me , that was kernel related problem in June.

--- On Sat, 10/17/09, Dulloor <dulloor@xxxxxxxxx> wrote:

From: Dulloor <dulloor@xxxxxxxxx>
Subject: [Xen-devel] Re: "Hotplug Scripts not working ..." error on jaunty
To: "Marc - A. Dahlhaus" <mad@xxxxxx>Date: Saturday, October 17, 2009, 6:08 PM


- udev version is 141 (udevadm info -V)

- There is only a /etc/udev directory. It had two files : xend.rules
and xen-backend.rules. Removing them didn't help.

- I use pvops kernel. And, I use - make install-xen and make
install-tools PYTHON_PREFIX_ARG=""

- Following is the output from "udevadm monitor" :
KERNEL[1255816932.742809] add      /devices/vbd-22-2049 (xen-backend)
KERNEL[1255816932.768878] add      /devices/vbd-22-2050 (xen-backend)
KERNEL[1255816932.786000] add      /devices/vbd-22-2051 (xen-backend)
KERNEL[1255816932.817227] add      /devices/vif-22-0 (xen-backend)
UDEV  [1255816932.818643] add      /devices/vif-22-0 (xen-backend)
KERNEL[1255816933.008654] add      /devices/console-22-0 (xen-backend)
UDEV  [1255816933.009916] add      /devices/console-22-0 (xen-backend)
UDEV  [1255816933.657785] add      /devices/vbd-22-2049 (xen-backend)
UDEV  [1255816934.707977] add      /devices/vbd-22-2050 (xen-backend)
UDEV  [1255816935.761811] add      /devices/vbd-22-2051 (xen-backend)
KERNEL[1255817033.248057] remove   /devices/console-22-0 (xen-backend)
------------- ****After the error****
UDEV  [1255817033.334001] remove   /devices/console-22-0 (xen-backend)

-dulloor


On Sat, Oct 17, 2009 at 4:04 PM, Marc - A. Dahlhaus <mad@xxxxxx> wrote:
> Hello,
>
> Dulloor schrieb:
>>
>> I see that these patches made it to xen-unstable, but I still see the
>> "Hotplug Scripts not working ..." error on my jaunty dom0 machine.
>
> The patches only fixed the build-time and installation-time detection of
>  udev to work with version above of 124.
>
> Have no jaunty based box around here to get some impressions of what goes
> wrong in this setup.
>
> Could you check what udev version it has and if /etc/hotplug and/or
> /etc/udev contains xen related stuff?
>
> As udev is in charge of hotplug events on jaunty a
> "udevadm monitor" might get you some further details on what events are
> generated.
>
>> I am trying to run a simple para-VM. xend starts fine (had to build
>> tools with PYTHON_PREFIX_ARG set to ""). And, I tried
>> xen-hotplug-cleanup too, but still getting the error.
>>
>> Any ideas ?
>
> Check what files get installed in /etc/udev and /etc/hotplug during
> installation. If you find anything xen related in /etc/hotplug remove it and
> check if that fixes your problem.
>
> It would be interesting to know how you install xen...
>
> Marc
>
>> -dulloor
>>
>> On Sat, Jun 27, 2009 at 4:02 PM, Marc - A. Dahlhaus <mad@xxxxxx> wrote:
>>>
>>> Hello Boris,
>>>
>>> as the problem you are describing is unrelated to the patch (which
>>> changes
>>> the detection of udev during buildtime only),
>>> can you please change the subject line to something more appropriate and
>>> give some additional details about the problem?
>>>
>>>
>>> Marc
>>>
>>> Boris Derzhavets schrieb:
>>>>
>>>> By some reasons i have to run  on regular basis run
>>>> ./xen-hotplug-cleanup
>>>> on two Xen 3.4.1 on top of Ubuntu 9.04 Servers patched systems to
>>>> re-enable ability start DomUs without hotplug scripts failure.
>>>>
>>>> Boris.
>>>>
>>>> --- On *Fri, 6/26/09, Marc - A. Dahlhaus [ Administration | Westermann
>>>> GmbH ] /<mad@xxxxxx>/* wrote:
>>>>
>>>>
>>>>   From: Marc - A. Dahlhaus [ Administration | Westermann GmbH ]
>>>>   <mad@xxxxxx>
>>>>   Subject: Re: [Xen-devel] [PATCH] Fix buildsystem to detect udev >
>>>>   version 124
>>>>   To: "Boris Derzhavets" <bderzhavets@xxxxxxxxx>
>>>>   Cc: "Xen-devel" <xen-devel@xxxxxxxxxxxxxxxxxxx>, "Keir Fraser"
>>>>   <keir.fraser@xxxxxxxxxxxxx>
>>>>   Date: Friday, June 26, 2009, 5:12 AM
>>>>
>>>>   Could go into 3.4 and unstable IMO.
>>>>
>>>>   On 3.3 there is no tools/hotplug/Linux/Makefile so it must be altered
>>>>   there to use tools/examples/Makefile for the first chunk and
>>>>   should work
>>>>   fine there.
>>>>
>>>>
>>>>   Marc
>>>>
>>>>   Am Donnerstag, den 25.06.2009, 20:03 -0700 schrieb Boris Derzhavets:
>>>>   > Where this patch is supposed to go either xen-unstable.hg or
>>>>   > xen-3.4-testing.hg (both) ?
>>>>   > Systems affected Ubuntu 9.04 (now) , F12 (pretty soon).
>>>>   >
>>>>   > Boris
>>>>   >
>>>>   > --- On Wed, 6/24/09, Marc - A. Dahlhaus [ Administration |
>>>>   Westermann
>>>>   > GmbH ] <mad@xxxxxx </mc/compose?to=mad@xxxxxx>> wrote:
>>>>   >            >         From: Marc - A. Dahlhaus [ Administration |
>>>> Westermann
>>>>   GmbH ]
>>>>   >         <mad@xxxxxx </mc/compose?to=mad@xxxxxx>>
>>>>   >         Subject: [Xen-devel] [PATCH] Fix buildsystem to detect
>>>>   udev >
>>>>   >         version 124
>>>>   >         To: "Xen-devel" <xen-devel@xxxxxxxxxxxxxxxxxxx
>>>>   </mc/compose?to=xen-devel@xxxxxxxxxxxxxxxxxxx>>
>>>>   >         Cc: "Daniel Schroeder" <sec@xxxxxxxxxxxxxxx
>>>>   </mc/compose?to=sec@xxxxxxxxxxxxxxx>>, "Keir Fraser"
>>>>   >         <keir.fraser@xxxxxxxxxxxxx
>>>>   </mc/compose?to=keir.fraser@xxxxxxxxxxxxx>>
>>>>   >         Date: Wednesday, June 24, 2009, 12:26 PM
>>>>   >            >         udev removed the udevinfo symlink from versions
>>>> higher than
>>>>   >         123 and xens
>>>>   >         build-system could not detect if udev is in place and
>>>>   has the
>>>>   >         required
>>>>   >         version. Attached patch fixes this.
>>>>   >            >            >         Signed-off-by: Marc-A. Dahlhaus
>>>> <mad@xxxxxx
>>>>   </mc/compose?to=mad@xxxxxx>>
>>>>   >            >            >            >            >
>>>> -----Inline Attachment Follows-----
>>>>   >            >         _______________________________________________
>>>>   >         Xen-devel mailing list
>>>>   >         Xen-devel@xxxxxxxxxxxxxxxxxxx
>>>>   </mc/compose?to=Xen-devel@xxxxxxxxxxxxxxxxxxx>
>>>>   >         http://lists.xensource.com/xen-devel
>>>>   >            >
>>>>   > _______________________________________________
>>>>   > Xen-devel mailing list
>>>>   > Xen-devel@xxxxxxxxxxxxxxxxxxx
>>>>   </mc/compose?to=Xen-devel@xxxxxxxxxxxxxxxxxxx>
>>>>   > http://lists.xensource.com/xen-devel
>>>>
>>>>
>>>>   _______________________________________________
>>>>   Xen-devel mailing list
>>>>   Xen-devel@xxxxxxxxxxxxxxxxxxx
>>>>   </mc/compose?to=Xen-devel@xxxxxxxxxxxxxxxxxxx>
>>>>   http://lists.xensource.com/xen-devel
>>>>
>>>>
>>>> ------------------------------------------------------------------------
>>>>
>>>> _______________________________________________
>>>> Xen-devel mailing list
>>>> Xen-devel@xxxxxxxxxxxxxxxxxxx
>>>> http://lists.xensource.com/xen-devel
>>>>
>>> _______________________________________________
>>> Xen-devel mailing list
>>> Xen-devel@xxxxxxxxxxxxxxxxxxx
>>> http://lists.xensource.com/xen-devel
>>>
>
>

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-devel







-----Inline Attachment Follows-----

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-devel


-----Inline Attachment Follows-----

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-devel



-----Inline Attachment Follows-----

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-devel

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-devel

 


Rackspace

Lists.xenproject.org is hosted with RackSpace, monitoring our
servers 24x7x365 and backed by RackSpace's Fanatical Support®.