Re: [Xen-users] Xen / Linux 4.3.3

[sm8ax1@xxxxxxxxxxx]

 Quoting Kojedzinszky RichÃrd <kojedzinszky.richard@xxxxxxxxxxxx>:

> In our case, we were using 4.3 from debian jessie-backports. Upon
> bootup, ganeti had started most of the domains, but at one domain the xl
> command started to hang. As it is a production node, I did not have much
> time to debug it, just restarted the machine. After it, exactly the same
> happened, the same domUs got started and at the very same domain did the
> xl command hang. After it I switched back to 4.2.
>
> Patrick: you can even reboot your machine from console with this:
> # echo 'b' > /proc/sysrq-trigger
>
> Regards,
>
> Kojedzinszky RichÃrd
> Euronet Magyarorszag Informatika Zrt.
>
> On Tue, 12 Jan 2016, Patrick Velder wrote:
>
> > Date: Tue, 12 Jan 2016 12:17:26 +0100
> > From: Patrick Velder <lists@xxxxxxxxx>
> > To: George Dunlap <George.Dunlap@xxxxxxxxxxxxx>
> > Cc: "xen-users@xxxxxxxxxxxxx" <xen-users@xxxxxxxxxxxxx>
> > Subject: Re: [Xen-users] Xen / Linux 4.3.3
> >
> > Hi George
> >
> > Thanks for your reply.
> >
> > 1) Yes, correct.
> >
> > 2) The domains in the autostart are started.
> >
> > I can do nothing with XL. If I exec xl, I cannot stop it with CTRL+C /
> > kill -9. I can not even reboot the Domain0 with reboot -f.The server
> > needs to be resetted with IPMI or physical.
> >
> > Yes, I'm using systemd. I will have a look at it.
> >
> > Best Regards
> > Patrick
> >
> > On 12.01.2016 11:57, George Dunlap wrote:
> > > On Fri, Jan 8, 2016 at 9:38 PM, Patrick Velder <lists@xxxxxxxxx> wrote:
> > > > Hi
> > > >
> > > > I just tried to boot kernel 4.3.3 on my xen host. xl list does not
work
> > > > (freezes), but the domUs are up. Reboot is impossible, and it's not
> > > > possible
> > > > to kill the hanging xl processes.
> > > > Any ideas? I'm using debian 8 with xen from the debian repositories.
> > > Sorry, just to clarify:
> > >
> > > 1. You're running Debian 8 (Jessie) in dom0, and you built your own
> > > 4.3.3 Linux kernel
> > >
> > > 2. Either "xl create" works, or the "xendomains" script works, because
> > > you are able to start domUs after rebooting into Linux 4.3.3?
> > >
> > > Normally if "xl list" hangs it's because xenstored isn't responding;
> > > but in that case, I wouldn't expect you to be able to create domU's
> > > either (although I suppose it might work by accident).
> > >
> > > Can you run "ps ax | grep xenstored" in your dom0 to see if it's
> > > running?
> > >
> > > Also, are you running sysvinit or systemd? If the latter, use
> > > systemctl and journalctl to see the status of xenstored and any error
> > > messages.
> > >
> > > -George
> > >
> > > _______________________________________________
> > > Xen-users mailing list
> > > Xen-users@xxxxxxxxxxxxx
> > > http://lists.xen.org/xen-users
> > _______________________________________________
> > Xen-users mailing list
> > Xen-users@xxxxxxxxxxxxxxxxx://lists.xen.org/xen-users
I don't mean to get off-topic, but it might be worth mentioning that  
`echo b > /proc/sysrq-trigger` will reboot the machine *without*  
syncing the disks, meaning you could end up with filesystem damage  
(although the risk is no greater than physically rebooting). Better is  
to go through the REISUB sequence (see  
https://en.wikipedia.org/wiki/Magic_SysRq_key).
I've seen plenty of lockups (not Xen related) and they are usually  
caused by a) OOM conditions, b) disk thrashing, c) overheating, d)  
processes in uninterruptible sleep, which is often caused by e) a  
faulty disk, SCSI/SATA controller, filesystem, or driver thereof any  
of those. I realize this is rudimentary information but it might get  
us to the point where we can find out what exactly the problem is and  
whether it is Xen related.
A good place to start is Alt+SysRq+W which will write all processes in  
uninterruptible sleep (the kind that prevent the system from  
rebooting) to the console.
If you don't already have a shell, try Alt+SysRq+RK and wait a few  
seconds, if you still can't get a shell, Atl+SysRq+F, wait,  
Alt+SysRq+E, wait, Alt+SysRq+I, wait and if you still can't,  
Alt+SysRq+SUB will sync the filesystems, emergency remount them  
read-only, and finally reboot the system (wait a while in between  
those). You'll of course have to substitute the physical keypresses  
with `echo $key > /proc/sysrq-trigger` or IPMI if it provides that  
functionality. Hopefully you will get a shell before the last step, at  
which point you can do a `ps aux` and look for uninterruptible ('D' in  
the STATE column) processes, `journalctl -e -u xenstored`, or whatever  
you like.
Also keep in mind that you can get the log from the previous boot  
using `journalctl -e -b -1` provided that journalctl flushed the log  
buffer to the filesystem, and that the filesystem was synced to disk  
before the system was rebooted (Alt+SysRq+E and S should help with  
this).

-------------------------------------------------

ONLY AT VFEmail! - Use our Metadata Mitigator to keep your email out of the 
NSA's hands!
$24.95 ONETIME Lifetime accounts with Privacy Features!  
15GB disk! No bandwidth quotas!
Commercial and Bulk Mail Options!  

_______________________________________________
Xen-users mailing list
Xen-users@xxxxxxxxxxxxx
http://lists.xen.org/xen-users