[Xen-tools] gdbserver-xen on x86_64 [2nd attempt]

[Tim Barbour <trb@xxxxxxxxx>]

Hi,

This message was originally delivered to the list server on 2007-4-22 (see the
log message from my mail server, below), but it does not appear in the list
archives for April, so I am resending it. According to the "Welcome" mail I
received from the list server, I subscribed to the list on 2007-4-19 (several
days prior to sending the message).

Apr 22 16:42:57 [postfix/smtp] 25B151361B: to=<xen-tools@xxxxxxxxxxxxxxxxxxx>, 
relay=host01-bcn-london.deploy.xenoserver.org[217.147.82.46]:25, delay=13, 
delays=0.25/0.01/9.5/3.1, dsn=2.0.0, status=sent (250 OK id=1Hflj2-0001oJ-2Y)

Why should the mailing-list server ignore my post ? I have not received any kind
of bounce message.

The text of my original message follows below.

I am trying to use gdbserver-xen to debug a paravirtualised XenoLinux VM on
x86_64, but without much success.

I currently have xen 3.0.4 and xen-tools 3.0.4, built using the ebuilds from the
Gentoo marineam-xen overlay. I built xen (not xen-tools), with debug=y (although
I only want to debug a VM, not xen itself). The dom0 and domU kernels are both
linux-2.6.16.40-xen (the latter built with CONFIG_DEBUG_KERNEL etc., and patched
slightly to support a kernel module I am writing). I have followed the
instructions at
http://lists.xensource.com/archives/html/xen-devel/2005-08/msg00981.html and
also tell xengdb "set architecture i386:x86-64:intel".

gdbserver-xen seems to attach to the VM fine, e.g.

trb@elysium ~ $ sudo gdbserver-xen 127.0.0.1:9999 --attach 3
Attached; pid = 3
Listening on port 9999
Remote debugging from host 127.0.0.1

xengdb talks to gdbserver-xen, but it does not quite work. Below is an example
xengdb session:

trb@elysium ~ $ xengdb /xen/boot/vmlinux-syms-2.6.16.40-xen-pmm

GNU gdb 6.2.1
Copyright 2004 Free Software Foundation, Inc.
GDB is free software, covered by the GNU General Public License, and you are
welcome to change it and/or distribute copies of it under certain conditions.
Type "show copying" to see the conditions.
There is absolutely no warranty for GDB.  Type "show warranty" for details.
This GDB was configured as "x86_64-unknown-linux-gnu"...Using host libthread_db 
library "/lib/libthread_db.so.1".

(gdb) set architecture i386:x86-64:intel
The target architecture is assumed to be i386:x86-64:intel
(gdb) target remote 127.0.0.1:9999
Remote debugging using 127.0.0.1:9999
[New Thread 0]
[Switching to Thread 0]
0xffffffff801073aa in hypercall_page ()
(gdb) bt
#0  0xffffffff801073aa in hypercall_page ()
Cannot access memory at address 0xffffffff80107000
(gdb) where
#0  0xffffffff801073aa in hypercall_page ()
(gdb) list
1       /*
2        *  linux/arch/x86_64/kernel/head64.c -- prepare to run common code
3        *
4        *  Copyright (C) 2000 Andrea Arcangeli <andrea@xxxxxxx> SuSE
5        *
6        *  $Id: head64.c,v 1.22 2001/07/06 14:28:20 ak Exp $
7        *
8        *  Jun Nakajima <jun.nakajima@xxxxxxxxx>
9        *      Modified for Xen.
10       */
(gdb) c
Continuing.
Warning:
Cannot insert breakpoint -2.
Error accessing memory address 0xffffffff80100000: Input/output error.

(gdb) break copy_process
Cannot access memory at address 0xffffffff80128f40
(gdb) q

The response to c (continue) is odd - I was not attempting to set a breakpoint
(subsequently I did attempt to, which also failed).

When xengdb exits, the VM typically continues fine. But in this case it was not
entirely unscathed. It was still booting when gdbserver-xen attached, and had
just printed the third line beginning with "NET:"; after gdb quit, it printed
the rest (and completed booting okay), where the "BUG: soft lockup detected on
CPU#0! etc." seems suspiciously related to the debugging.

TCP bic registered
NET: Registered protocol family 1
NET: Registered protocol family 17
NET: Registered protocol family 15
BUG: soft lockup detected on CPU#0!
CPU 0:
Modules linked in:
Pid: 0, comm: swapper Not tainted 2.6.16.40-xen-pmm #4
RIP: e030:[<ffffffff801073aa>] <ffffffff801073aa>{hypercall_page+938}
RSP: e02b:ffffffff8052ff28  EFLAGS: 00000246
RAX: 0000000000000000 RBX: 0000000000000000 RCX: ffffffff801073aa
RDX: 0000000000000001 RSI: 00000000deadbeef RDI: 00000000deadbeef
RBP: ffffffff8052ff50 R08: 0000000000000000 R09: ffff88000142e098
R10: 0000000000000001 R11: 0000000000000246 R12: 0000000000000000
R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000000
FS:  0000000000000000(0000) GS:ffffffff80518000(0000) knlGS:0000000000000000
CS:  e033 DS: 0000 ES: 0000

Call Trace: <ffffffff8010f339>{safe_halt+137} <ffffffff80109877>{xen_idle+119}
       <ffffffff8010a01c>{cpu_idle+92} <ffffffff80108406>{rest_init+38}
       <ffffffff80531865>{start_kernel+517} <ffffffff8053120d>{_sinittext+525}
Sending DHCP requests .., OK
IP-Config: Got DHCP answer from 0.0.0.0, my address is 10.137.1.125
[...]


I would appreciate any suggestions as to how to get it working. The only thing I
can think of is to try the latest unstable version of xen, but I hoped 3.0.4
would be new enough to support debugging a paravirtualised VM on x86_64 . Would
it help to use unstable, and if so, what version (e.g. Hg tag) ? Since this is
my main machine, I am a little nervous about running it on an unstable version
of xen... . I could use a different physical machine for testing, but then I
might as well just test on a native kernel, via kgdb.

Tim

_______________________________________________
Xen-tools mailing list
Xen-tools@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-tools