Re: [MirageOS-devel] network performance question

[Takayuki Imada <takayuki.imada@xxxxxxxxx>]

Hi,

> So Linux VM drops from 45 Gbit/s to 14 Gbit/s, but that is still much higher 
that 2.4 Gbit/s.
> More ideas what could I change?
>
> Or maybe additional question. What number did other get with iperf on 
MirageOS?
> Takayuki, you are author of MirageOS iperf implementation). What numbers 
would you expect?
I think the results you obtained are not so different from what I observed.
(I confirmed two MirageOS Solo5/ukvm VMs for a client and a server achieved 
about 800Mbps on a physical server having Intel IvyBridge Xeon 2.4GHz)

I have been trying to increase the network performance by utilizing Netmap in 
MirageOS Solo5/ukvm, but performance by Linux VM would be higher ...

Kind regards,

--
Takayuki Imada


On 10/25/17 4:54 PM, Justin Cinkelj wrote:
> The particular Linux VM is used as build machine, and I guess most of used ram 
> was disk cache.
> I can run it with lower RAM, for sure. I can also set vCPU count to 1.
> I will also try to run it without vhost (if I figure out how to do that :).
>
> Ok, I added:
> |<interface type='bridge'>
>    <driver name='qemu'/>|
>
> And VM starts without vhost now (I also rmmod vhost/vhost_net, and they didn't 
> get loaded back at VM start), has 1 CPU and 512 MB memory.
> I had to also pin emulator threads (to pCPU 1 == 0x02 mask):
> [root@jcpc includeOS] # virsh emulatorpin mirageOS 1
> [root@jcpc includeOS] # virsh emulatorpin mirageOS
> emulator: CPU Affinity
> ----------------------------------
>         *: 1
> [root@jcpc includeOS] # virsh vcpuinfo mirageOS
> VCPU:           0
> CPU:            1
> State:          running
> CPU time:       212.1s
> CPU Affinity:   -y------
> [root@jcpc includeOS] # virsh iothreadinfo mirageOS
> No IOThreads found for the domain
>
> Result:
> justin_cinkelj@jcpc:~$ taskset 0x04 iperf -c 192.168.122.20 -t10
> ------------------------------------------------------------
> Client connecting to 192.168.122.20, TCP port 5001
> TCP window size:  119 KByte (default)
> ------------------------------------------------------------
> [  3] local 192.168.122.1 port 60196 connected with 192.168.122.20 port 5001
> [ ID] Interval       Transfer     Bandwidth
> [  3]  0.0-10.0 sec  16.2 GBytes  13.9 Gbits/sec
> top:
> %Cpu0  :  1.0 us,  0.0 sy,  0.0 ni, 99.0 id,  0.0 wa,  0.0 hi, 0.0 si,  0.0 st
> %Cpu1  : 61.4 us, 38.6 sy,  0.0 ni,  0.0 id,  0.0 wa,  0.0 hi, 0.0 si,  0.0 st
> %Cpu2  :  0.0 us, 17.0 sy,  0.0 ni, 82.0 id,  0.0 wa,  1.0 hi, 0.0 si,  0.0 st
> ...
>    PID USER      PR  NI    VIRT    RES    SHR S  %CPU %MEM TIME+ COMMAND
> 10172 qemu      20   0 5967952 695524  28268 R 100.0  2.1 7:47.57 
> qemu-system-x86
> 14303 justin_+  20   0  165816   1848   1680 S  16.8  0.0 0:00.71 iperf
>
> So Linux VM drops from 45 Gbit/s to 14 Gbit/s, but that is still much higher 
> that 2.4 Gbit/s.
> More ideas what could I change?
>
> Or maybe additional question. What number did other get with iperf on MirageOS?
> Takayuki, you are author of MirageOS iperf implementation). What numbers would 
> you expect?
>
> Thank you
> Justin
>
>
> On 10/24/2017 04:08 PM, Takayuki Imada wrote:
> > Justin,
> >
> > Thanks for providing the information.
> > I would like to add the following information:
> >
> > 1. total CPU utilization (host + VM)
> > >   4401 qemu      20   0 6038648 2.309g  22968 S 100.0  7.4  43:24.27 
> > qemu-system-x86
> > >   4422 root      20   0       0      0      0 R 83.2  0.0   0:22.90 vhost-4401
> > This indicates about 1.8 CPU cores were assigned for the Linux VM iperf 
> > receiver.
> > >   24788 root      20   0  528708  16148  16064 R 100.0  0.0   0:04.35 ukvm-bin
> > On the other hand, the MirageOS iperf server was running with just 1 CPU core 
> > in total.
> >
> > 2. Memory size assigned(used)
> > The above results indicate that the Linux VM was using 2.3G physical memory 
> > space,
> > but the MirageOS VM was running with 512MB physical memory size(this size is 
> > fixed in Solo5/ukvm).
> >
> > I guess that the above two points and networking implementation difference 
> > between Linux and MirageOS affected the performance difference you observed.
> >
> > Kind regards,
_______________________________________________
MirageOS-devel mailing list
MirageOS-devel@xxxxxxxxxxxxxxxxxxxx
https://lists.xenproject.org/cgi-bin/mailman/listinfo/mirageos-devel