Re: [Xen-devel] RE: [Xen-users] How to: Improve Network Throughput of a XEN kernel?

[Pasi Kärkkäinen <pasik@xxxxxx>]

On Wed, Mar 03, 2010 at 10:44:38AM -0500, Mike Viau wrote:
>    >
>    > On Tue, Mar 02, 2010 at 07:50:59PM -0500, Mike Viau wrote:
>    > > >Any just first argument dummy=dummy should on kenel line.
>    > > > Read Wiki one more time. It works.
>    > >
>    > > I believe you that it does work. I was able to see (XEN) messages
>    coming from the hypervisor, but no matter how I changed the option I kept
>    getting a kernel fault.
>    > >
>    > > I feel like it was time to stop mucking around with getting Ubuntu and
>    the grub2 boot loader to work with XEN so I decided to install a headless
>    OpenSUSE 11.2 system.
>    > >
>    > > The openSUSE distribution supported Xen 3.4.1 tools, and had a
>    2.6.31.12 xenified kernel. On this setup I experience identical network
>    performance.
>    > >
>    > > ~1MB/s - w/ xen kernel
>    > > ~4.5 MB/s - w/ non-xen kernel
>    > >
>    > > I use the e1000e kernel module driver for my Ethernet card.
>    > >
>    > > Is this typical Dom0 (or Xen Host) network performance with a xenified
>    kernel or is just in my two setups that one encounters such a network
>    performance hit to the Dom0?
>    > >
>    >
>    > No, it's not typical.
>    >
>    > How are you measuring the performance? In a gigabit network you should
>    be able
>    > to push around 110 MB/sec.. 4.5 MB/sec sounds really slow.
> 
>    I am using Fast Ethernet (100MB). What sort of throughput should I expect
>    becasue I also believe ~1MB is too slow, even just to/from the Dom0,
>    without any network emulation to the DomUs.
>

100 Mbit ethernet can transfer around 11 MB/sec.
 
>    I simply used SCP, HTTP, and FTP file transfer protocols on the Xen host
>    to another computer on the LAN. The ~4.5MB/s bottleneck is the hard drive
>    on the other computer so I suspect the true bandwidth would be higher, but
>    getting past the ~1MB/s roadblock is my major concern.
> 

Try using normal FTP (without encryption), or iperf.

Or wget to /dev/null, like: wget -O /dev/null http://testserver/file

>    I have tried two linux distros and both forward ported xenified and
>    parvirtualized xen kernel.
> 
>    Does the Xen hypervisor have any effect on the network throughput? Would
>    updating the Intel E1000E kernel module driver affect the network
>    throughput?
> 

Shouldn't be big difference, when you measure from dom0.

-- Pasi

>    -M
> 
>    >
>    > -- Pasi
>    >
>    > > -M
>    > >
>    > > --- On Mon, 3/1/10, Mike Viau <viaum@xxxxxxxxxxxxxxx> wrote:
>    > >
>    > > From: Mike Viau <viaum@xxxxxxxxxxxxxxx>
>    > > Subject: RE: [Xen-devel] RE: [Xen-users] How to: Improve Network
>    Throughput of a XEN kernel?
>    > > To: bderzhavets@xxxxxxxxx
>    > > Cc: xen-devel@xxxxxxxxxxxxxxxxxxx, xen-users@xxxxxxxxxxxxxxxxxxx
>    > > Date: Monday, March 1, 2010, 2:52 PM
>    > >
>    > > Why using grub2 vs grub is baffling am not sure....
>    > >
>    > > I think I'd prefer to use the kernel command line in grub2 so that the
>    options may be changed easier laster on. However I still have been having
>    so success with boot
>    > > XEN with Ubuntu's grub2 boot loader.
>    > >
>    > > Am I correct to conclude CONFIG_CMDLINE_BOL must = N to allow GRUB2 to
>    pass kernel options?
>    > > [1]http://cateee.net/lkddb/web-lkddb/CMDLINE_BOOL.html
>    > >
>    > > I have read through the Wiki page at
>    [2]http://wiki.xensource.com/xenwiki/XenCommonProblems. I am running a
>    64-bit Xen Hypervisor.
>    > >
>    > > Thus far I have been having no success with using the example entry
>    found at
>    
> [3]http://old.nabble.com/Strange-interaction-from-grub2-and-XEN-td26464067.html
>    > >
>    > > A question regarding that entry is what would be the root device when
>    there are separate /boot and root partitions? I am using the partition
>    with the /boot (grub2) as
>    > > my 'set root' device right?
>    > >
>    > > I am also using the dummy=dummy as the first option on the first
>    module line, again as discovered on the wiki.
>    > >
>    > > Additionally posted on another thread here in the Xen mailing list
>    was:
>    > >
>    > > > On 1 March 2010 08:09, Luke Carrier <luke.carrier@xxxxxxxxx> wrote:
>    > > >>
>    > > >> Not sure about Debian, but I know Ubuntu isn*t ideal for Xen any
>    more
>    > > >> because of the presence of GRUB2 * the new boot loader doesn*t
>    support
>    > > >> passing certain command lines and the current multiboot
>    implementation
>    > > >> breaks the method Xen uses to load the
>    > > Dom0 kernel * you can*t use Ubuntu
>    > > >> 9.10 Server with Xen.
>    > >
>    > > I am confused now because Boris clearly created a wordpress.com weblog
>    in where Ubuntu 9.10 Server was
>    > > used with XEN.
>    > >
>    > > Lastly, out of curiosity, what (or what kind) of commands cannot be
>    passed with GRUB2?
>    > >
>    > > ThX!
>    > >
>    > >
>    > > -M
>    > >
>    > >
>    
> --------------------------------------------------------------------------------------------------------------------------------------------------------------------------
>    > >
>    > > Date: Sat, 27 Feb 2010 11:20:31 -0800
>    > > From: bderzhavets@xxxxxxxxx
>    > > Subject: Re: [Xen-devel] RE: [Xen-users] How to: Improve Network
>    Throughput of a XEN kernel?
>    > > To: mike.viau@xxxxxxxxxxxxxxx
>    > > CC: xen-devel@xxxxxxxxxxxxxxxxxxx; xen-users@xxxxxxxxxxxxxxxxxxx
>    > >
>    > > In Xen Wiki is mentioned , that parameters for kernel may be set via
>    command line as follows:-
>    > >
>    
> ************************************************************************************************************
>    > > module (hd1,10)/vmlinuz-2.6.31.8 dummy=dummy root=/dev/sda7 ro
>    console=tty0
>    > >
>    
> ************************************************************************************************************
>    > > # cat /proc/cmdline
>    > >
>    > > shows that GRUB2 skips first argument, what causes problem.
>    > >
>    > > Compiling built-in CMD_LINE is just an option.
>    > > Read all Wiki written by Pasi and you would never be surprised ;)
>    > >
>    > > > P.S: What is your feeling on using a recent Ubuntu system with
>    > > > grub-legacy as the boot loader? I noticed that Squeeze has adopted
>    grub2 > package as well too...
>    > >
>    > > I have development Xen 3.4.3 instance on top Ubuntu 9.10 Server with
>    GRUB2 ( baremetall) loading any from 3 mentioned bellow kernels under Xen
>    3.4.3
>    > >
>    > > 1.) 2.6.32.9 PVOPS
>    > > 2.) 2.6.31.6 PVOPS
>    > > 3.) 2.6.31.12 xenified
>    > >
>    > > with no problems with dummy=dummy as first parameter ( thanks Pasi).
>    > > It may also may be done via compiling kernels with built-in CMD_LINE.
>    > > It works fine as well.
>    > >
>    > > Boris.
>    > >
>    > > --- On Sat, 2/27/10, Mike Viau <viaum@xxxxxxxxxxxxxxx> wrote:
>    > >
>    > > From: Mike Viau <viaum@xxxxxxxxxxxxxxx>
>    > > Subject: [Xen-devel] RE: [Xen-users] How to: Improve Network
>    Throughput of a XEN kernel?
>    > > To: bderzhavets@xxxxxxxxx
>    > > Cc: xen-devel@xxxxxxxxxxxxxxxxxxx, xen-users@xxxxxxxxxxxxxxxxxxx
>    > > Date: Saturday, February 27, 2010, 12:03 PM
>    > >
>    > > Yes, thank you Boris for pointing that out. I found that was defiantly
>    true. The 3.2.1 xen hypervisor could only boot the 2.6.26-2 xen kernel and
>    the xenified
>    > > 2.6.31.12.
>    > >
>    > > Additionally I found that:
>    > >
>    > > Xend does not start when using pv_ops dom0 kernel?
>    > >
>    > > In December 2009 pv_ops dom0 kernel modules were renamed to have a
>    "xen-" prefix in them, ie. "evtchn.ko" became "xen-evtchn.ko".
>    > >
>    > > This makes Xen 3.4.x xend fail to start, because it tried to load
>    "evtchn.ko", but that doesn't exist. You need to load "xen-evtchn.ko" and
>    then start xend. Fedora 12
>    > > xen-3.4.2-2 rpms have this problem fixed.
>    > >
>    > > Also make sure you have xenfs mounted to "/proc/xen", that's needed
>    aswell.
>    > >
>    > > [Source: [4]http://wiki.xensource.com/xenwiki/XenParavirtOps]
>    > >
>    > > And it appears the Debian xen-3.4.2-2 debs (in squeeze) have not fixed
>    this problem.
>    > >
>    > > Lastly I am still doing some reading on the grub2 package and how to
>    configure it. It turns out that I have been using grub-legacy (or just
>    good old grub) for so long
>    > > that configuring the grub2 boot loader is turning out to be my own
>    personal kryptonite.
>    > >
>    > >
>    > >
>    > > I am still working on getting the hypervisor to load the dom0 kernel
>    in a Ubuntu system.
>    > >
>    > >
>    > >
>    > > I have noticed an example in your wordpress @ [5]Set up Xen 3.4.3-rc2
>    & Libvirt 0.7.0 Dom0 (with 2.6.31.8 xenified aka Suse kernel) on top of
>    Ubuntu 9.10 Server
>    > >
>    > >
>    > >
>    > > With CONFIG_CMDLINE="root=/dev/sdb11 ro console=tty0"
>    > >
>    > >
>    > >
>    > > menuentry "Xen 3.4 / Ubuntu 9.10 kernel 2.6.31.8 xenified" {
>    > > insmod ext2
>    > > set root=(hd1,10)
>    > > multiboot (hd1,10)/xen-3.4.gz
>    > > module (hd1,10)/vmlinuz-2.6.31.8
>    > > module (hd1,10)/initrd-2.6.31.8.img
>    > > }
>    > >
>    > >
>    > >
>    > > I am sure you wouldn't of mentioned the CONFIG_CMDLINE kernel option
>    is it was not important for using with grub2 right? I have not have to
>    compile a kernel with the
>    > > command line previous as I was just able to supply the arguments on
>    the kernel line of my menu.lst (grub configuration file).
>    > >
>    > > P.S: What is your feeling on using a recent Ubuntu system with
>    grub-legacy as the boot loader? I noticed that Squeeze has adopted grub2
>    package as well too...
>    > >
>    > >
>    > > -M
>    > >
>    > >
>    
> ------------------------------------------------------------------------------------------------------------------------------------------------------------------------
>    > >
>    > > Date: Sat, 27 Feb 2010 04:32:59 -0800
>    > > From: bderzhavets@xxxxxxxxx
>    > > Subject: Re: FW: [Xen-users] How to: Improve Network Throughput of a
>    XEN kernel?
>    > > To: jbeulich@xxxxxxxxxx; mike.viau@xxxxxxxxxxxxxxx
>    > > CC: xen-devel@xxxxxxxxxxxxxxxxxxx; xen-users@xxxxxxxxxxxxxxxxxxx
>    > >
>    > > You cannot load pvops kernel under xen-hypervisor 3.2.1 ( >=3.4.0 in
>    general)
>    > >
>    > > Boris.
>    > >
>    > > --- On Fri, 2/26/10, Mike Viau <viaum@xxxxxxxxxxxxxxx> wrote:
>    > >
>    > > From: Mike Viau <viaum@xxxxxxxxxxxxxxx>
>    > > Subject: FW: [Xen-users] How to: Improve Network Throughput of a XEN
>    kernel?
>    > > To: bderzhavets@xxxxxxxxx, jbeulich@xxxxxxxxxx
>    > > Cc: xen-users@xxxxxxxxxxxxxxxxxxx, xen-devel@xxxxxxxxxxxxxxxxxxx
>    > > Date: Friday, February 26, 2010, 3:19 PM
>    > >
>    > > Hello again,
>    > >
>    > > Just a quick update...
>    > >
>    > > I was really hoping it was just going to be an issue with the kernel
>    .config, however I copied exact configuration from the working Debain
>    2.6.26-2-xen-amd64 kernel
>    > > and used a make oldconfig but the end result was the same.
>    > >
>    > > After booting the xenified (2.6.31.12) or pvops kernel (2.6.31.6) via
>    both xen-hypervisor 3.2.1 or xen-hypervisor 3.4.2 the maximum data
>    transfer to/from the Dom0
>    > > (Xen host) is ~1MB/s.
>    > >
>    > > Next I plan on re-trying my efforts on a Ubuntu system to see if same
>    network bottlenecking is present.
>    > >
>    > >
>    
> ----------------------------------------------------------------------------------------------------------------------------------------------------------------------
>    > >
>    > > Date: Fri, 26 Feb 2010 09:16:59 -0800
>    > > From: bderzhavets@xxxxxxxxx
>    > > Subject: Re: [Xen-users] How to: Improve Network Throughput of a XEN
>    kernel?
>    > > To: xen-users@xxxxxxxxxxxxxxxxxxx; viaum@xxxxxxxxxxxxxxx
>    > >
>    > > Would escalate to xen-devel. 2.6.31.8 (12) aka Suse xenified was Jan
>    Beulich stuff.
>    > > He might have an answer.
>    > > Boris.
>    > >
>    > > --- On Fri, 2/26/10, Mike Viau <viaum@xxxxxxxxxxxxxxx> wrote:
>    > >
>    > > From: Mike Viau <viaum@xxxxxxxxxxxxxxx>
>    > > Subject: [Xen-users] How to: Improve Network Throughput of a XEN
>    kernel?
>    > > To: xen-users@xxxxxxxxxxxxxxxxxxx
>    > > Cc: waldi@xxxxxxxxxx
>    > > Date: Friday, February 26, 2010, 12:09 PM
>    > >
>    > > Hi there,
>    > >
>    > > I am starting a new thread to try and solve a problem I am having will
>    slow network throughput to the XEN host (not to a domU).
>    > >
>    > > I have confirmed my problem occurs with PVOPS 'xen/master' kernel as
>    well using
>    > >
>    
> [6]http://pasik.reaktio.net/xen/pv_ops-dom0-debug/config-2.6.31.6-pvops-dom0-xen-master-x86_64
>    as the .config.
>    > >
>    > > It also occurs with a forward ported xenified kernel (2.6.31.12) using
>    the attached config-11-xen (.config).
>    > >
>    > > However when using the xen kernel found in debain lenny (2.6.26-2) for
>    the amd64 architecture I get similiar transfer rates to my Xen host as I
>    do on a baremetel
>    > > kernel.
>    > >
>    > > Bastian Blank perhaps you could point me in the right direction in
>    regards to a debian specific customization or patch I should be aware of?
>    > > Maxium transfer rate (to/from hard disk) using the PVOPS 'xen/master'
>    kernel and forward ported xenified kernel (2.6.31.12) is bottlenecking at
>    ~1MB/s. On the
>    > > debain lenny xen kernel I am getting ~4.5MB/s.
>    > >
>    > > I was hoping one could suggest options in the .config that might be
>    modified in order to improve network throughput of a XEN kernel and
>    therefor increase the data
>    > > transfer rates over the netowork to and from the Xen host.
>    > >
>    > > Thanks alot for any help! As always I will be willing to accept any
>    suggestions :)
>    > >
>    > > -M
>    > >
>    > >
> 
>    --------------------------------------------------------------------------
> 
>    Live connected with Messenger on your phone [1]Learn more.
> 
> References
> 
>    Visible links
>    1. http://go.microsoft.com/?linkid=9712957

_______________________________________________
Xen-users mailing list
Xen-users@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-users