Re: [Xen-users] iscsi vs nfs for xen VMs

[Pasi Kärkkäinen <pasik@xxxxxx>]

On Sat, Jan 29, 2011 at 04:27:52PM +0100, Bart Coninckx wrote:
> On 01/29/11 16:09, Pasi Kärkkäinen wrote:
> > On Thu, Jan 27, 2011 at 09:35:38AM +0100, Adi Kriegisch wrote:
> >   
> >> Hi!
> >>
> >>     
> >>>> iSCSI tipically has a quite big overhead due to the protocol, FC, SAS,
> >>>> native infiniband, AoE have very low overhead.
> >>>>
> >>>>         
> >>> For iSCSI vs AoE, that isn't as true as you might think. TCP offload can
> >>> take care of a lot of the overhead. Any server class network adapter
> >>> these days should allow you to send 60kb packets to the network adapter
> >>> and it will take care of the segmentation, while AoE would be limited to
> >>> MTU sized packets. With AoE you need to checksum every packet yourself
> >>> while with iSCSI it is taken care of by the network adapter.
> >>>       
> >> What AoE actually does is sending a frame per block. Block size is 4K -- so
> >> no need for fragmentation. The overhead is pretty low, because we're
> >> talking about Ethernet frames.
> >> Most iSCSI issues I have seen are with reordering of packages due to
> >> transmission across several interfaces. So what most people recommend is to
> >> keep the number of interfaces to two. To keep performance up this means you
> >> have to use 10G, FC or similar which is quite expensive -- especially if
> >> you'd like to have a HA SAN network (HSRP and stuff like that is required).
> >>
> >> AoE does not suffer from those issues: Using 6 GBit interfaces is no
> >> problem at all, load balancing will happen automatically, as the load is
> >> distributed equally across all available interfaces. HA is very simple:
> >> just use two switches and connect one half of the interfaces to one switch
> >> and the other half to the other switch. (It is recommended to use switches
> >> that can do jumbo frames and flow control)
> >> IMHO most of the current recommendations and practises surrounding iSCSI
> >> are there to overcome the shortcomings of the protocol. AoE is way more
> >> robust and easier to handle.
> >>
> >>     
> > iSCSI does not have problems using multiple gige interfaces.
> > Just setup multipathing properly.
> >
> > -- Pasi
> >
> >
> > _______________________________________________
> > Xen-users mailing list
> > Xen-users@xxxxxxxxxxxxxxxxxxx
> > http://lists.xensource.com/xen-users
> >   
> 
> On this subject: am using multipathing to iSCSI too, hoping to have
> aggregated speed on top of path redundancy but the speed seems not to
> surpass the one of a single interface.
> 
> Is anyone successful at doing this?
> 

You're benchmarking sequential/linear IO, using big blocksizes, right?

Some questions:
        - Are you using multipath round robin path policy?
        - After how many IOs do you switch paths? You might need to lower the 
min_ios.


-- Pasi


_______________________________________________
Xen-users mailing list
Xen-users@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-users