[Xen-devel] Re: Please help: domU becomes unresponsive

["Jerry Amundson" <jamundso@xxxxxxxxx>]

More info from crash... Can someone enlighten me as to what [swapper]
is attempting here?

   PID    PPID  CPU       TASK        ST  %MEM     VSZ    RSS  COMM
      0      0   0  ffffffff80322b40  RU   0.0       0      0  [swapper]
      0      1   1  ffffff800e7b1030  RU   0.0       0      0  [swapper]
      0      1   2  ffffff80006727f0  RU   0.0       0      0  [swapper]
      0      1   3  ffffff8000672030  RU   0.0       0      0  [swapper]
      0      1   4  ffffff80000057f0  RU   0.0       0      0  [swapper]
      0      1   5  ffffff8000005030  RU   0.0       0      0  [swapper]
      1      0   2  ffffff800e7b17f0  IN   0.0    4756    552  init
      2      1   0  ffffff80000037f0  IN   0.0       0      0  [migration/0]
      3      1   0  ffffff8000003030  IN   0.0       0      0  [ksoftirqd/0]
      4      1   0  ffffff80000a67f0  IN   0.0       0      0  [events/0]
      5      1   1  ffffff80000a6030  IN   0.0       0      0  [khelper]
      6      1   4  ffffff82800037f0  IN   0.0       0      0  [kthread]
      7      6   0  ffffff8280003030  IN   0.0       0      0  [xenwatch]
      8      6   0  ffffff82800127f0  IN   0.0       0      0  [xenbus]
     16      6   1  ffffff8280012030  IN   0.0       0      0  [migration/1]
     17      6   1  ffffff82800677f0  IN   0.0       0      0  [ksoftirqd/1]
     18      6   1  ffffff8280067030  RU   0.0       0      0  [events/1]
     20      6   2  ffffff82800697f0  IN   0.0       0      0  [migration/2]
     21      6   2  ffffff8280069030  IN   0.0       0      0  [ksoftirqd/2]
     22      6   2  ffffff82800b67f0  IN   0.0       0      0  [events/2]
     24      6   3  ffffff82800b6030  IN   0.0       0      0  [migration/3]
     25      6   3  ffffff80007047f0  IN   0.0       0      0  [ksoftirqd/3]

On Wed, Jul 16, 2008 at 9:50 AM, Jerry Amundson <jamundso@xxxxxxxxx> wrote:
> Hi all, sorry to intrude on xen-devel, but I think I need direction
> from the expertise here. I've admin'd Xen servers of various flavors
> for a couple years, but never seen this before. After a period ranging
> from several hours to several days, my  primary database and
> development DomU completely locks up. Net disconnects, but CPU(sec)
> continues to tick in xentop. No errors, and nothing logged. All dom's
> are CentOS, so I'm pasting below what I've already posted to
> centos-devel and centos-virt.
>
> On Mon, Jul 14, 2008 at 3:49 PM, Jerry Amundson <jamundso@xxxxxxxxx> wrote:
>> Two Dell 6950 (now called R905, 4 Dual-Core AMD Opteron 8200 series)
>> heartbeat/drbd nodes running the stock CentOS 5.2 Dom0. The domU's are
>> the only resources in heartbeat.
>> Dom1 is a perfectly running, updated, CentOS 5.2 Apache/MySQL/Samba
>> Dom2 is a CentOS 4.6 software development and database server
>
> So crash tells me that Dom2 gets to this point:
>  SYSTEM MAP: System.map-2.6.9-67.0.20.ELxenU
> DEBUG KERNEL: /usr/lib/debug/lib/modules/2.6.9-67.0.20.ELxenU/vmlinux
> (2.6.9-67.0.20.ELxenU)
>   DUMPFILE: /public/IntSys/tmp/m1.dmp
>       CPUS: 6
>       DATE: Mon Jul 14 11:53:59 2008
>     UPTIME: 6 days, 11:39:33
> LOAD AVERAGE: 548.07, 542.95, 434.99
>      TASKS: 2721
>   NODENAME: monolith
>    RELEASE: 2.6.9-67.0.20.ELxenU
>    VERSION: #1 SMP Thu Jun 26 08:36:44 EDT 2008
>    MACHINE: x86_64  (2194 Mhz)
>     MEMORY: 10 GB
>      PANIC: ""
>        PID: 0
>    COMMAND: "swapper"
>       TASK: ffffffff80322b40  (1 of 6)  [THREAD_INFO: ffffffff80426000]
>        CPU: 0
>      STATE: TASK_RUNNING
>    WARNING: panic task not found
>
> crash> bt
> PID: 0      TASK: ffffffff80322b40  CPU: 0   COMMAND: "swapper"
>  #0 [ffffffff80427ec0] schedule at ffffffff80294d9a
>  #1 [ffffffff80427f98] cpu_idle at ffffffff8010b85d
> crash> kmem -i
>             PAGES        TOTAL      PERCENTAGE
>  TOTAL MEM  2621696        10 GB         ----
>     FREE     8884      34.7 MB    0% of TOTAL MEM
>     USED  2612812        10 GB   99% of TOTAL MEM
>   SHARED        0            0    0% of TOTAL MEM
>  BUFFERS    59585     232.8 MB    2% of TOTAL MEM
>   CACHED  1325825       5.1 GB   50% of TOTAL MEM
>     SLAB   358565       1.4 GB   13% of TOTAL MEM
>
> TOTAL HIGH        0            0    0% of TOTAL MEM
>  FREE HIGH        0            0    0% of TOTAL HIGH
>  TOTAL LOW  2621696        10 GB  100% of TOTAL MEM
>  FREE LOW     8884      34.7 MB    0% of TOTAL LOW
>
> kmem: swap_info[0].swap_map at ffffff00001ea000 is unaccessible
>
> So I see where the DomU is, but how did it get there? Can I find out
> from crash, or do I need something "real-time" within the DomU? Of
> course, searching has given me nothing to go on, hence this post, but
> I'll continue...
>
> Tia,
> jerry
>
> --
> "Your life is trite and jaded, boring and confiscated." - Twisted Sister
>



-- 
"Your life is trite and jaded, boring and confiscated." - Twisted Sister

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-devel