[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: Troubles analyzing crash dumps from xl dump-core



On 10.03.21 20:37, Igor Druzhinin wrote:
On 30.01.21 19:53, Roman Shaposhnik wrote:
On Fri, Jan 29, 2021 at 11:28 PM Jürgen Groß <jgross@xxxxxxxx> wrote:

On 29.01.21 21:12, Roman Shaposhnik wrote:
Hi!

I'm trying to see how much mileage I can get out of
crash(1) 7.2.8 (based on gdb 7.6) when it comes to analyzing crash
dumps taken via xl dump-core (this is all on x86_64 with stock Xen
v. 4.14).

The good news is that the image actually does load up but it throws
the following WARNINGs in the process:

WARNING: cannot access vmalloc'd module memory
crash: read error: kernel virtual address: ffffffff93613480  type:
"fill_task_struct"
WARNING: active task ffffffff93613480 on cpu 0 not found in PID hash
crash: read error: kernel virtual address: ffffffff93613480  type:
"fill_task_struct"
WARNING: cannot read log_buf contents

And then the info that it gives me around basic things like ps, mod,
log, etc. is really super limited (and I am now suspecting may even
be wrong).

Since I was primarily after dmesg/log initially, I tried:
crash> log
log: WARNING: cannot read log_buf contents

Then I tried taking an xl dump-core from the domU that was still
very much alive and happy and got similar results -- so it clearly
doesn't seem to be related to the state domU is in.

As matter of fact, I actually got to the desired dmesg output by
simply running strings(1) on the core file -- so the info is
definitely there -- but I guess some kind of index/reference maybe
broken.

With all that in mind, if there's anyone on this ML who has recently
done Xen DomU crash dump analysis -- I would definitely appreciate
the pointers!

For me it just works (openSUSE).

Can you please run:

crash --version and readelf -a XXXX (on the xl dump-core output) and
post the results?

# crash --version

crash 7.2.1

I tried to build this version but I still get the following while trying to 
open a dump file
produced by "xl dump-core":

[root@lcy2-dt92 crash]# ./crash ../vmlinux-5.8.0-44-generic ../xxx.dmp

crash 7.2.1
Copyright (C) 2002-2017  Red Hat, Inc.
Copyright (C) 2004, 2005, 2006, 2010  IBM Corporation
Copyright (C) 1999-2006  Hewlett-Packard Co
Copyright (C) 2005, 2006, 2011, 2012  Fujitsu Limited
Copyright (C) 2006, 2007  VA Linux Systems Japan K.K.
Copyright (C) 2005, 2011  NEC Corporation
Copyright (C) 1999, 2002, 2007  Silicon Graphics, Inc.
Copyright (C) 1999, 2000, 2001, 2002  Mission Critical Linux, Inc.
This program is free software, covered by the GNU General Public License,
and you are welcome to change it and/or distribute copies of it under
certain conditions.  Enter "help copying" to see the conditions.
This program has absolutely no warranty.  Enter "help warranty" for details.

GNU gdb (GDB) 7.6
Copyright (C) 2013 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.  Type "show copying"
and "show warranty" for details.
This GDB was configured as "x86_64-unknown-linux-gnu"...

crash: cannot determine base kernel version
crash: ../vmlinux-5.8.0-44-generic and ../xxx.dmp do not match!


Am I doing something wrong? How do I need to use crash for it to work?

Could you try the attached patch?


Juergen

Attachment: 0001-tools-libs-ctrl-fix-xc_core_arch_map_p2m-to-support-.patch
Description: Text Data

Attachment: OpenPGP_0xB0DE9DD628BF132F.asc
Description: application/pgp-keys

Attachment: OpenPGP_signature
Description: OpenPGP digital signature


 


Rackspace

Lists.xenproject.org is hosted with RackSpace, monitoring our
servers 24x7x365 and backed by RackSpace's Fanatical Support®.