Re: [Xen-users] Xen 10GBit Ethernet network performance (was: Re: Experience with Xen & AMD Opteron 4200 series?)

[Joseph Glanville <joseph.glanville@xxxxxxxxxxxxxx>]

On 9 July 2012 19:50, Linus van Geuns <linus@xxxxxxxxxxxxx> wrote:
> Hey Joseph,
>
> On Mon, Jul 2, 2012 at 8:42 PM, Joseph Glanville
> <joseph.glanville@xxxxxxxxxxxxxx> wrote:
>>
>> [..]
>>
>>
>> In terms of Linux bridging module performance... I can easily do 20
>> odd gigabits between VMs on the same host and can push around 13
>> gigabits between hosts (40gig infinbiand).
>> It's worth noting that with both OVS and the bridge module you will be
>> limited by PPS (packets per second) rather than throughput.
>> With very big MTUs ( I am using 64k over IB ) one can do many
>> gigabits, but using 1500 byte MTUs throughput drops below 3gigabits.
>> This is due to the max throughput of the both the linux bridge module
>> and OVS on a single core maxes out a a few 100k PPS.
>
>
> What version of Xen and which distros are you using for dom0/domU?
> And what hardware are you using?
> Did you measure plain dom0 to dom0 performance over your infiniband
> connection(s)?

Gentoo running Linux 3.2 (pretty heavily patched though, especially to
drive up IPoIB performance)
Yes, we do a ton of stuff between our dom0s. Performance is not
noticeably different in terms of throughput but VMs have a very
measurable drop in PPS.
it would not be uncommon for us to easily get between 11-12 Gigabits
between dom0s using big MTUs as noted above. 2044 MTU is a good
balance between working with stuff and not breaking multicast + great
throughput.

>
> I am still digging for the limiting bottleneck/ issue with dom0 to dom0
> performance over 10GBit Eth on my boxes.
>
> Regards, Linus
>



-- 
CTO | Orion Virtualisation Solutions | www.orionvm.com.au
Phone: 1300 56 99 52 | Mobile: 0428 754 846

_______________________________________________
Xen-users mailing list
Xen-users@xxxxxxxxxxxxx
http://lists.xen.org/xen-users