Re: [Xen-users] Network connections stalling - Debian Linux domU's

[Steve Dawson <sdawson@xxxxxxxxxxxxxxxx>]

On 07/20/2014 12:35 PM, Henrik Størner wrote:
> Hi,
> 
> I am a long-time Linux user and mail admin (15+ years), but have only
> recently been dipping my toes into Xen.
> 
> This past week I have converted my main mailserver from a setup with a
> single dedicated server to a Xen virtual server configuration with
> separate MySQL, Mail and web functionality. All appeared well, so I let
> the users loose on the system. And immediately started getting
> complaints that they could not fetch mail using IMAP.
> 
> To cut a long story short, the problem appears to be with the domU
> hosting the IMAP server (Dovecot). When a user connects to the IMAP
> service, the network connection quickly stalls. A "netstat" on the domU
> shows a very large Send-Q (i.e. data sent, but not acknowledged by the
> remote end of the connection) - usually around 25000, it ought to be
> quite small. A tcpdump also shows very large intervals between packets,
> 10-30 seconds is not uncommon.
> 
> It is easy to replicate: Using Thunderbird, I create a new profile with
> the necessary settings. The initial sync of the mailbox will trigger the
> problem.
> 
> 
> After reading a bunch of docs, blogs etc, I tried these things:
>   - Dom0 memory fixed at 2 GB, ballooning disabled
>   - CPU 0+1 dedicated to dom0 (pinned), the domU's configured to
>     NOT use CPU 0+1
>   - CPU 4.5 dedicated to the mail server domU
>   - txqueuelen on the VIF changed to 1024
> 
> Neither of these appear to make any difference whatsoever (the server
> was rebooted after changing the dom0 settings). And it doesn't really
> seem that any of the dom's are cpu-starved.
> 
> So to at least get the users off my back, I am now running the mail
> server in dom0 (mounting the mailserver volume in dom0 and running from
> a chroot). This runs flawlessly, but it is obviously not how I would
> like to do it...
> 
> 
> The physical server where this happens is a quad-core/hyperthread box
> (so there appears to be 8 cpu's). It has 24 GB memory - 2 VCPU's and 2
> GB memory dedicated for dom0. The mailserver domU has the same
> ressources - 2 VCPU's, 2 GB memory. The domU's are all PV Linuxes.
> 
> 
> All of the software is stock Debian 7: Xen 4.1.4, Linux kernel 3.2.0.
> 
> 
> Any ideas are most welcome!
> 
> 
> Thanks in advance,
> Henrik
> 
> _______________________________________________
> Xen-users mailing list
> Xen-users@xxxxxxxxxxxxx
> http://lists.xen.org/xen-users
> 
It could be hyper threading, either disable it, or manually pin your domU's 
cpus in pairs. I had a similar issue running a bunch of citrix xenapp  (2008 
windows terminal) servers with hyper threading enabled, when the loads got to 
about 30% the load averages would skyrocket suddenly, pinning the cpus fixed 
the problem.

Regards.

Steve.

_______________________________________________
Xen-users mailing list
Xen-users@xxxxxxxxxxxxx
http://lists.xen.org/xen-users