RE: Troubles analyzing crash dumps from xl dump-core

[Igor Druzhinin <igor.druzhinin@xxxxxxxxxx>]

> On 30.01.21 19:53, Roman Shaposhnik wrote:
> > On Fri, Jan 29, 2021 at 11:28 PM Jürgen Groß <jgross@xxxxxxxx> wrote:
> >>
> >> On 29.01.21 21:12, Roman Shaposhnik wrote:
> >>> Hi!
> >>>
> >>> I'm trying to see how much mileage I can get out of
> >>> crash(1) 7.2.8 (based on gdb 7.6) when it comes to analyzing crash
> >>> dumps taken via xl dump-core (this is all on x86_64 with stock Xen
> >>> v. 4.14).
> >>>
> >>> The good news is that the image actually does load up but it throws
> >>> the following WARNINGs in the process:
> >>>
> >>> WARNING: cannot access vmalloc'd module memory
> >>> crash: read error: kernel virtual address: ffffffff93613480  type:
> >>> "fill_task_struct"
> >>> WARNING: active task ffffffff93613480 on cpu 0 not found in PID hash
> >>> crash: read error: kernel virtual address: ffffffff93613480  type:
> >>> "fill_task_struct"
> >>> WARNING: cannot read log_buf contents
> >>>
> >>> And then the info that it gives me around basic things like ps, mod,
> >>> log, etc. is really super limited (and I am now suspecting may even
> >>> be wrong).
> >>>
> >>> Since I was primarily after dmesg/log initially, I tried:
> >>> crash> log
> >>> log: WARNING: cannot read log_buf contents
> >>>
> >>> Then I tried taking an xl dump-core from the domU that was still
> >>> very much alive and happy and got similar results -- so it clearly
> >>> doesn't seem to be related to the state domU is in.
> >>>
> >>> As matter of fact, I actually got to the desired dmesg output by
> >>> simply running strings(1) on the core file -- so the info is
> >>> definitely there -- but I guess some kind of index/reference maybe
> >>> broken.
> >>>
> >>> With all that in mind, if there's anyone on this ML who has recently
> >>> done Xen DomU crash dump analysis -- I would definitely appreciate
> >>> the pointers!
> >>
> >> For me it just works (openSUSE).
> >
> > Can you please run:
> >
> > crash --version and readelf -a XXXX (on the xl dump-core output) and
> > post the results?
> 
> # crash --version
> 
> crash 7.2.1

I tried to build this version but I still get the following while trying to 
open a dump file
produced by "xl dump-core":

[root@lcy2-dt92 crash]# ./crash ../vmlinux-5.8.0-44-generic ../xxx.dmp

crash 7.2.1
Copyright (C) 2002-2017  Red Hat, Inc.
Copyright (C) 2004, 2005, 2006, 2010  IBM Corporation
Copyright (C) 1999-2006  Hewlett-Packard Co
Copyright (C) 2005, 2006, 2011, 2012  Fujitsu Limited
Copyright (C) 2006, 2007  VA Linux Systems Japan K.K.
Copyright (C) 2005, 2011  NEC Corporation
Copyright (C) 1999, 2002, 2007  Silicon Graphics, Inc.
Copyright (C) 1999, 2000, 2001, 2002  Mission Critical Linux, Inc.
This program is free software, covered by the GNU General Public License,
and you are welcome to change it and/or distribute copies of it under
certain conditions.  Enter "help copying" to see the conditions.
This program has absolutely no warranty.  Enter "help warranty" for details.

GNU gdb (GDB) 7.6
Copyright (C) 2013 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.  Type "show copying"
and "show warranty" for details.
This GDB was configured as "x86_64-unknown-linux-gnu"...

crash: cannot determine base kernel version
crash: ../vmlinux-5.8.0-44-generic and ../xxx.dmp do not match!


Am I doing something wrong? How do I need to use crash for it to work?

Igor