[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [Xen-users] a lot of packet loss


  • To: xen-users@xxxxxxxxxxxxxxxxxxx
  • From: Attila Szamos <szamosa@xxxxxxxxx>
  • Date: Sat, 2 May 2009 01:42:01 +0200
  • Delivery-date: Fri, 01 May 2009 16:43:37 -0700
  • Domainkey-signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :content-type:content-transfer-encoding; b=IijbMUSVz73qqefUNvaPJ9PDA/LbF7kjyzL8dMLBeNSQtNBD6b8WgiLiIzVDTHxTkk PFyT72b1yzQsOX542ca7u5DeO5GRlziwp+Xo1aTxLMakDs3jrERU2IlMqcA+SX0N5dUH dJsUJfKC2tCGEvBgw5hT5/3GF4tri9vJ3onwQ=
  • List-id: Xen user discussion <xen-users.lists.xensource.com>

I tried another tcpdump-thing.

I pinged from the VM to another non-xen-kernel machine. I ran a
tcpdump on the xen host machine and the ping-target machine. The point
is that on the host machine, on which the ICMP packets flow through:

01:29:53.075672 IP perftest-vm2 > perftest7: ICMP echo request, id
24077, seq 8, length 64
01:29:53.075841 IP perftest7 > perftest-vm2: ICMP echo reply, id
24077, seq 8, length 64
01:29:54.075686 IP perftest-vm2 > perftest7: ICMP echo request, id
24077, seq 9, length 64
01:29:54.075884 IP perftest7 > perftest-vm2: ICMP echo reply, id
24077, seq 9, length 64
01:29:55.075700 IP perftest-vm2 > perftest7: ICMP echo request, id
24077, seq 10, length 64
01:29:55.075926 IP perftest7 > perftest-vm2: ICMP echo reply, id
24077, seq 10, length 64
01:29:56.075715 IP perftest-vm2 > perftest7: ICMP echo request, id
24077, seq 11, length 64
01:29:57.075728 IP perftest-vm2 > perftest7: ICMP echo request, id
24077, seq 12, length 64
01:29:58.075742 IP perftest-vm2 > perftest7: ICMP echo request, id
24077, seq 13, length 64
01:29:59.075761 IP perftest-vm2 > perftest7: ICMP echo request, id
24077, seq 14, length 64
01:30:00.075769 IP perftest-vm2 > perftest7: ICMP echo request, id
24077, seq 15, length 64
01:30:01.075783 IP perftest-vm2 > perftest7: ICMP echo request, id
24077, seq 16, length 64
01:30:02.075798 IP perftest-vm2 > perftest7: ICMP echo request, id
24077, seq 17, length 64
01:30:03.075812 IP perftest-vm2 > perftest7: ICMP echo request, id
24077, seq 18, length 64
01:30:04.075825 IP perftest-vm2 > perftest7: ICMP echo request, id
24077, seq 19, length 64
01:30:05.075840 IP perftest-vm2 > perftest7: ICMP echo request, id
24077, seq 20, length 64
01:30:05.075987 IP perftest7 > perftest-vm2: ICMP echo reply, id
24077, seq 20, length 64
01:30:10.082287 IP perftest-vm2 > perftest7: ICMP echo request, id
24077, seq 21, length 64
01:30:10.082448 IP perftest7 > perftest-vm2: ICMP echo reply, id
24077, seq 21, length 64
01:30:11.079923 IP perftest-vm2 > perftest7: ICMP echo request, id
24077, seq 22, length 64

On the perftest7 machine if a packet arrives, it gets a reply. So it
seems that the packets are lost on the network. It imples that the
problem is with the network, but If I dont use domU-s, there is no
unreplied packets.





On Sat, May 2, 2009 at 1:15 AM, Attila Szamos <szamosa@xxxxxxxxx> wrote:
> I commented out the resolv.conf, but nothing changed.
> I also tried the tcpdump issue. I experienced this:
>
> root@test5:~# ping 172.27.68.28
> PING 172.27.68.28 (172.27.68.28) 56(84) bytes of data.
> 64 bytes from 172.27.68.28: icmp_seq=10 ttl=64 time=0.189 ms
> 64 bytes from 172.27.68.28: icmp_seq=11 ttl=64 time=0.218 ms
>
> --- 172.27.68.28 ping statistics ---
> 16 packets transmitted, 2 received, 87% packet loss, time 15004ms
> rtt min/avg/max/mdev = 0.189/0.203/0.218/0.020 ms
>
>
> On the host:
> root@test6:~# cat dom0tcpdump > dom0tcpdump
> root@test6:~# cat dom0tcpdump | grep ICMP
> 01:03:19.108715 IP 172.27.68.114 > 172.27.68.28: ICMP echo request, id
> 7461, seq 10, length 64
> 01:03:19.108754 IP 172.27.68.28 > 172.27.68.114: ICMP echo reply, id
> 7461, seq 10, length 64
> 01:03:20.108733 IP 172.27.68.114 > 172.27.68.28: ICMP echo request, id
> 7461, seq 11, length 64
> 01:03:20.108770 IP 172.27.68.28 > 172.27.68.114: ICMP echo reply, id
> 7461, seq 11, length 64
>
> On the guest:
> root@test-vm2:~# tcpdump > domutcp
> root@test-vm2:~# cat domutcp | grep ICMP
> 01:03:19.142677 IP 172.27.68.114 > 172.27.68.28: ICMP echo request, id
> 7461, seq 10, length 64
> 01:03:19.142677 IP 172.27.68.28 > 172.27.68.114: ICMP echo reply, id
> 7461, seq 10, length 64
> 01:03:20.108578 IP 172.27.68.114 > 172.27.68.28: ICMP echo request, id
> 7461, seq 11, length 64
> 01:03:20.108578 IP 172.27.68.28 > 172.27.68.114: ICMP echo reply, id
> 7461, seq 11, length 64
>
> It is very interesting, because it seems that the ICMP packets even
> dont reach the host OS, but If I ping the host OS, each ICMP echo
> request got an ECHO reply.
>
> I read about this network problem in another forums, and someone had
> the same problem. He tought it is scheduling problem.
>
>
>
>
> On Sat, May 2, 2009 at 12:49 AM, Bhasker C V <bhasker@xxxxxxxxxxxxx> wrote:
>> On Fri, 1 May 2009, Attila Szamos wrote:
>>
>>> I've fix-ed the timesyncronization problem. But I don't know where to
>>> start with the network problem.
>>> If I ping the VM a lot of packet didn't get an echo reply.
>>>
>>> root@test6:~# ping perftest-vm2
>>> PING test-vm2 (172.27.68.28) 56(84) bytes of data.
>>> 64 bytes from test-vm2 (172.27.68.28): icmp_seq=1 ttl=64 time=0.346 ms
>>> 64 bytes from test-vm2 (172.27.68.28): icmp_seq=2 ttl=64 time=0.048 ms
>>> 64 bytes from test-vm2 (172.27.68.28): icmp_seq=3 ttl=64 time=0.039 ms
>>> 64 bytes from test-vm2 (172.27.68.28): icmp_seq=4 ttl=64 time=0.041 ms
>>> 64 bytes from test-vm2 (172.27.68.28): icmp_seq=6 ttl=64 time=0.032 ms
>>> 64 bytes from test-vm2 (172.27.68.28): icmp_seq=7 ttl=64 time=0.044 ms
>>> 64 bytes from test-vm2 (172.27.68.28): icmp_seq=8 ttl=64 time=0.038 ms
>>> 64 bytes from test-vm2 (172.27.68.28): icmp_seq=43 ttl=64 time=8.05 ms
>>> 64 bytes from test-vm2 (172.27.68.28): icmp_seq=56 ttl=64 time=0.042 ms
>>> 64 bytes from test-vm2 (172.27.68.28): icmp_seq=57 ttl=64 time=0.036 ms
>>> 64 bytes from test-vm2 (172.27.68.28): icmp_seq=58 ttl=64 time=0.041 ms
>>> 64 bytes from test-vm2 (172.27.68.28): icmp_seq=59 ttl=64 time=0.038 ms
>>> 64 bytes from test-vm2 (172.27.68.28): icmp_seq=60 ttl=64 time=0.041 ms
>>> 64 bytes from test-vm2 (172.27.68.28): icmp_seq=61 ttl=64 time=0.038 ms
>>> 64 bytes from test-vm2 (172.27.68.28): icmp_seq=62 ttl=64 time=0.033 ms
>>>
>>> --- test-vm2 ping statistics ---
>>> 64 packets transmitted, 15 received, 76% packet loss, time 63064ms
>>> rtt min/avg/max/mdev = 0.032/0.594/8.056/1.995 ms
>>
>> Does the ping directly to IP address too gives the same issue ?
>> sometimes DNS is a pain...
>> also on the domU side, try commenting out the complete resolv.conf
>> just to take DNS out of the way and try direct IP ping.
>>
>> you can also on the domU side run a tcpdump and check why the particular
>> icmp sequence number is missing. you can see the replies from domU and
>> if the reply does not come to the dom0, then there could be a problem with
>> xen.
>> else
>> ...
>>
>>>
>>> I've tried to switch the networking to 'route' from 'bridge', but it
>>> didn't help. Any suggestions?
>>>
>>> _______________________________________________
>>> Xen-users mailing list
>>> Xen-users@xxxxxxxxxxxxxxxxxxx
>>> http://lists.xensource.com/xen-users
>>>
>>
>> Bhasker C V
>> Registered linux user #306349
>>
>>
>>
>

_______________________________________________
Xen-users mailing list
Xen-users@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-users


 


Rackspace

Lists.xenproject.org is hosted with RackSpace, monitoring our
servers 24x7x365 and backed by RackSpace's Fanatical Support®.