[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [Xen-users] Xen / Linux 4.3.3



 Quoting Kojedzinszky RichÃrd <kojedzinszky.richard@xxxxxxxxxxxx>:

In our case, we were using 4.3 from debian jessie-backports. Upon
bootup, ganeti had started most of the domains, but at one domain the xl
command started to hang. As it is a production node, I did not have much
time to debug it, just restarted the machine. After it, exactly the same
happened, the same domUs got started and at the very same domain did the
xl command hang. After it I switched back to 4.2.

Patrick: you can even reboot your machine from console with this:
# echo 'b' > /proc/sysrq-trigger

Regards,

Kojedzinszky RichÃrd
Euronet Magyarorszag Informatika Zrt.

On Tue, 12 Jan 2016, Patrick Velder wrote:

Date: Tue, 12 Jan 2016 12:17:26 +0100
From: Patrick Velder <lists@xxxxxxxxx>
To: George Dunlap <George.Dunlap@xxxxxxxxxxxxx>
Cc: "xen-users@xxxxxxxxxxxxx" <xen-users@xxxxxxxxxxxxx>
Subject: Re: [Xen-users] Xen / Linux 4.3.3

Hi George

Thanks for your reply.

1) Yes, correct.

2) The domains in the autostart are started.

I can do nothing with XL. If I exec xl, I cannot stop it with CTRL+C /
kill -9. I can not even reboot the Domain0 with reboot -f.The server
needs to be resetted with IPMI or physical.

Yes, I'm using systemd. I will have a look at it.

Best Regards
Patrick

On 12.01.2016 11:57, George Dunlap wrote:
On Fri, Jan 8, 2016 at 9:38 PM, Patrick Velder <lists@xxxxxxxxx> wrote:
Hi

I just tried to boot kernel 4.3.3 on my xen host. xl list does not
work
(freezes), but the domUs are up. Reboot is impossible, and it's not
possible
to kill the hanging xl processes.
Any ideas? I'm using debian 8 with xen from the debian repositories.

Sorry, just to clarify:

1. You're running Debian 8 (Jessie) in dom0, and you built your own
4.3.3 Linux kernel

2. Either "xl create" works, or the "xendomains" script works, because
you are able to start domUs after rebooting into Linux 4.3.3?

Normally if "xl list" hangs it's because xenstored isn't responding;
but in that case, I wouldn't expect you to be able to create domU's
either (although I suppose it might work by accident).

Can you run "ps ax | grep xenstored" in your dom0 to see if it's
running?

Also, are you running sysvinit or systemd? If the latter, use
systemctl and journalctl to see the status of xenstored and any error
messages.

-George

_______________________________________________
Xen-users mailing list
Xen-users@xxxxxxxxxxxxx
http://lists.xen.org/xen-users

_______________________________________________
Xen-users mailing list
Xen-users@xxxxxxxxxxxxxxxxx://lists.xen.org/xen-users


I don't mean to get off-topic, but it might be worth mentioning that `echo b > /proc/sysrq-trigger` will reboot the machine *without* syncing the disks, meaning you could end up with filesystem damage (although the risk is no greater than physically rebooting). Better is to go through the REISUB sequence (see https://en.wikipedia.org/wiki/Magic_SysRq_key).

I've seen plenty of lockups (not Xen related) and they are usually caused by a) OOM conditions, b) disk thrashing, c) overheating, d) processes in uninterruptible sleep, which is often caused by e) a faulty disk, SCSI/SATA controller, filesystem, or driver thereof any of those. I realize this is rudimentary information but it might get us to the point where we can find out what exactly the problem is and whether it is Xen related.

A good place to start is Alt+SysRq+W which will write all processes in uninterruptible sleep (the kind that prevent the system from rebooting) to the console.

If you don't already have a shell, try Alt+SysRq+RK and wait a few seconds, if you still can't get a shell, Atl+SysRq+F, wait, Alt+SysRq+E, wait, Alt+SysRq+I, wait and if you still can't, Alt+SysRq+SUB will sync the filesystems, emergency remount them read-only, and finally reboot the system (wait a while in between those). You'll of course have to substitute the physical keypresses with `echo $key > /proc/sysrq-trigger` or IPMI if it provides that functionality. Hopefully you will get a shell before the last step, at which point you can do a `ps aux` and look for uninterruptible ('D' in the STATE column) processes, `journalctl -e -u xenstored`, or whatever you like.

Also keep in mind that you can get the log from the previous boot using `journalctl -e -b -1` provided that journalctl flushed the log buffer to the filesystem, and that the filesystem was synced to disk before the system was rebooted (Alt+SysRq+E and S should help with this).


-------------------------------------------------

ONLY AT VFEmail! - Use our Metadata Mitigator to keep your email out of the 
NSA's hands!
$24.95 ONETIME Lifetime accounts with Privacy Features! 15GB disk! No bandwidth quotas! Commercial and Bulk Mail Options!
_______________________________________________
Xen-users mailing list
Xen-users@xxxxxxxxxxxxx
http://lists.xen.org/xen-users

 


Rackspace

Lists.xenproject.org is hosted with RackSpace, monitoring our
servers 24x7x365 and backed by RackSpace's Fanatical Support®.