[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

[Xen-users] [SPAM] Re: Xen bridge network issue


  • To: xen-users@xxxxxxxxxxxxxxxxxxx
  • From: Liron Gabay <gliron@xxxxxxxxx>
  • Date: Tue, 4 May 2010 19:54:20 +0300
  • Delivery-date: Tue, 04 May 2010 09:56:02 -0700
  • Domainkey-signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :content-type; b=lh59vJuQWR6NAESwqaa4px+CqBlTaU+cNQmKAew0m20u1CDmNZ/crSgk+nwkvFnjoT 4IijVSxxLt6tWiMaYrB0sfuklhMUulSU7xcuFo9wRNK38xBU5qZIRD85pSoOud4AIjfH 9uv4WJSGpqXRYjCEZ/q/fo8nRyKMlblMddci4=
  • Importance: Low
  • List-id: Xen user discussion <xen-users.lists.xensource.com>

Hi,
Still facing this problem, can anyoneÂtake a look?
Â
eth0 interface (Xen Virtual Interface) on the VM is showing growing Rx bytes on every ifconfig command,Âbut Tx bytes is not moving.
Also: qemu-dm log maybe has something:
Â
domid: 1
Warning: vlan 0 is not connected to host network
Watching /local/domain/0/device-model/1/logdirty/next-active
Watching /local/domain/0/device-model/1/command
char device redirected to /dev/pts/1
/builddir/build/BUILD/xen-3.4.0/tools/ioemu-dir/hw/xen_blktap.c:628: Init blktap pipes
xen be: console-0: backend state: Initialising -> InitWait
xen be: console-0: frontend not ready, ignoring
xen be: console-0: bind evtchn port 21
xen be: console-0: ring mfn 1277431, remote port 2, local port 21, limit 1048576
xen be: console-0: backend state: InitWait -> Connected
xs_read(): vncpasswd get error. /vm/edc7ed59-91d2-1c7f-9523-dd2f102eb74e/vncpasswd.
xen be: console-0: backend update: state
xen be: console-0: backend update: hotplug-status
xen be: console-0: backend update: state
xen be: console-0: backend update: state
xen be: console-0: frontend update: vnc-port
xen be: console-0: frontend update: tty
xen be: console-0: backlog piling up, nobody listening?
Â
Â
Â
Thanks!
Â
Â=================================
My Xen includes 2 guests. Xen itself (10.2.0.52) gets free access to the outside world and toÂits guests.
Both guests however (10.2.0.54/10.2.0.55) see each other but stayÂunder house arrest!
Not a singleÂping manages to go past the bridge (xenbr0) andÂget an answer from the default gateway (10.2.0.254.)
On the guest I keep seeing 'Destination Host Unreachable', i.e.:
#traceroute 10.2.0.254 comes back with the !H message.
Â
I have pasted all the info I gathered in attempt to understand who locks the guests in,Âbut will be happy toÂadd any furtherÂoutput required to get this resolved.
Â
I followed so many manuals and links, one that seriously caught my attention had to do with MAC addresses issue. Could this be my case?.
Â
Many thanks for any assistance,
Liron.
Â
Â
#tcpdump -nvvi xenbr0 | grep -i 10.2.0.55
---------------------------------------------
11:19:41.548806 arp who-has 10.2.0.254 tell 10.2.0.55
11:19:41.549059 IP (tos 0x0, ttl 64, id 0, offset 0, flags [DF], proto: ICMP (1), length: 84) 10.2.0.55 > 10.2.0.254: ICMP echo request, id 19463, seq 19, length 64


#tcpdump -nvvi vif1.0 host 10.2.0.55
------------------------------------
11:21:23.403063 IP (tos 0x0, ttl 64, id 0, offset 0, flags [DF], proto: ICMP (1), length: 84) 10.2.0.55 > 10.2.0.254: ICMP echo request, id 22279, seq 7, length 64


#tcpdump -vv -n -c 1000 | grep -i 10.2.0.254
--------------------------------------------
11:22:46.256389 arp who-has 10.2.0.254 tell 10.2.0.55
11:22:46.256662 IP (tos 0x0, ttl 64, id 0, offset 0, flags [DF], proto: ICMP (1), length: 84) 10.2.0.55 > 10.2.0.254: ICMP echo request, id 23047, seq 8, length 64


# iptables -L -vv -n
--------------------
Chain INPUT (policy ACCEPT 10869 packets, 769K bytes)
Âpkts bytes targetÂÂÂÂ prot opt inÂÂÂÂ outÂÂÂÂ sourceÂÂÂÂÂÂÂÂÂÂÂÂÂÂ destination
 0 0 ACCEPT tcp -- * * 0.0.0.0/0 0.0.0.0/0 state NEW tcp dpt:8003

Chain FORWARD (policy ACCEPT 0 packets, 0 bytes)
Âpkts bytes targetÂÂÂÂ prot opt inÂÂÂÂ outÂÂÂÂ sourceÂÂÂÂÂÂÂÂÂÂÂÂÂÂ destination
 0 0 ACCEPT all -- * * 0.0.0.0/0 0.0.0.0/0 state RELATED,ESTABLISHED PHYSDEV match --physdev-out vif1.0
 0 0 ACCEPT udp -- * * 0.0.0.0/0 0.0.0.0/0 PHYSDEV match --physdev-in vif1.0 udp spt:68 dpt:67
 0 0 ACCEPT all -- * * 0.0.0.0/0 0.0.0.0/0 state RELATED,ESTABLISHED PHYSDEV match --physdev-out vif1.0
 0 0 ACCEPT all -- * * 10.2.0.55 0.0.0.0/0 PHYSDEV match --physdev-in vif1.0

Chain OUTPUT (policy ACCEPT 8294 packets, 2149K bytes)
Âpkts bytes targetÂÂÂÂ prot opt inÂÂÂÂ outÂÂÂÂ sourceÂÂÂÂÂÂÂÂÂÂÂÂÂÂ destination

----------------------------------------------

As in http://wiki.xensource.com/xenwiki/XenNetworking#head-602e26cd4a03b992f3938fe1bea03fa0fea0ed8b

I tried:

#echo "1" > /proc/sys/net/ipv4/ip_forward

I also tried:

iptables -A FORWARD -m physdev --physdev-in eth0 --physdev-out '!' eth0Â -j ACCEPT
iptables -A FORWARD -m physdev --physdev-out eth0 --physdev-in '!' eth0Â -j ACCEPT

Still Nothing!

----------------------------------------------

# cat /etc/xen/xend-config.sxp | grep -v "^#" | grep "[a-z]"
-----------------------------------------------------------
(logfile /var/log/xen/xend.log)
(loglevel DEBUG)
(network-script network-bridge)
(vif-script vif-bridge)
(dom0-min-mem 196)
(enable-dom0-ballooning yes)
(dom0-cpus 0)
(vnc-listen '0.0.0.0')
(vncpasswd '')
(xend-domains-lock-path /opt/ovs-agent-2.3/utils/dlm.py)

-----------------------------------------------

# brctl show
bridge nameÂÂÂÂ bridge idÂÂÂÂÂÂÂÂÂÂÂÂÂÂ STP enabledÂÂÂÂ interfaces
xenbr0ÂÂÂÂÂÂÂÂÂ 8000.0050568b3ae4ÂÂÂÂÂÂ noÂÂÂÂÂÂÂÂÂÂÂÂÂ vif1.0
ÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂ eth0
--------------------------------------------------------------------

# xm list
NameÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂ IDÂÂ Mem VCPUsÂÂÂÂÂ StateÂÂ Time(s)
Domain-0ÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂ 0ÂÂ 564ÂÂÂÂ 2ÂÂÂÂ r-----ÂÂÂÂ 67.2
SiebelAppÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂ 1Â 1024ÂÂÂÂ 1ÂÂÂÂ -b----ÂÂÂÂ 32.6

---------------------------------------------------------------------

#xm network-list 1
-------------------
Idx BEÂÂÂÂ MAC Addr.ÂÂÂÂ handle state evt-ch tx-/rx-ring-ref BE-path
0ÂÂ 0Â 00:16:3e:07:03:49ÂÂÂ 0ÂÂÂÂ 4ÂÂÂÂÂ 10ÂÂÂ 815Â /816ÂÂÂÂ /local/domain/0/backend/vif/1/0

--------------------

# xm network-list 1 -l
(0
ÂÂÂ ((mac 00:16:3e:07:03:49)
ÂÂÂÂÂÂÂ (handle 0)
ÂÂÂÂÂÂÂ (protocol x86_32-abi)
ÂÂÂÂÂÂÂ (backend-id 0)
ÂÂÂÂÂÂÂ (state 4)
ÂÂÂÂÂÂÂ (backend /local/domain/0/backend/vif/1/0)
ÂÂÂÂÂÂÂ (tx-ring-ref 815)
ÂÂÂÂÂÂÂ (rx-ring-ref 816)
ÂÂÂÂÂÂÂ (event-channel 10)
ÂÂÂÂÂÂÂ (request-rx-copy 1)
ÂÂÂÂÂÂÂ (feature-rx-notify 1)
ÂÂÂÂÂÂÂ (feature-sg 1)
ÂÂÂÂÂÂÂ (feature-gso-tcpv4 1)
ÂÂÂ )
)

-------------------------------

# ifconfig -a --> ON 10.2.0.52 (Domain-0)
------------------------------------
eth0 Link encap:Ethernet HWaddr 00:50:56:8B:3A:E4
ÂÂÂÂÂÂÂÂÂ UP BROADCAST RUNNING MULTICASTÂ MTU:1500Â Metric:1
ÂÂÂÂÂÂÂÂÂ RX packets:18937 errors:0 dropped:0 overruns:0 frame:0
ÂÂÂÂÂÂÂÂÂ TX packets:13043 errors:0 dropped:0 overruns:0 carrier:0
ÂÂÂÂÂÂÂÂÂ collisions:0 txqueuelen:1000
ÂÂÂÂÂÂÂÂÂ RX bytes:1484679 (1.4 MiB)Â TX bytes:2669625 (2.5 MiB)
ÂÂÂÂÂÂÂÂÂ Interrupt:17 Base address:0x2000

loÂÂÂÂÂÂÂ Link encap:Local Loopback
ÂÂÂÂÂÂÂÂÂ inet addr:127.0.0.1Â Mask:255.0.0.0
ÂÂÂÂÂÂÂÂÂ UP LOOPBACK RUNNINGÂ MTU:16436Â Metric:1
ÂÂÂÂÂÂÂÂÂ RX packets:124 errors:0 dropped:0 overruns:0 frame:0
ÂÂÂÂÂÂÂÂÂ TX packets:124 errors:0 dropped:0 overruns:0 carrier:0
ÂÂÂÂÂÂÂÂÂ collisions:0 txqueuelen:0
ÂÂÂÂÂÂÂÂÂ RX bytes:17724 (17.3 KiB)Â TX bytes:17724 (17.3 KiB)

vif1.0 Link encap:Ethernet HWaddr FE:FF:FF:FF:FF:FF
ÂÂÂÂÂÂÂÂÂ UP BROADCAST RUNNING MULTICASTÂ MTU:1500Â Metric:1
ÂÂÂÂÂÂÂÂÂ RX packets:136 errors:0 dropped:0 overruns:0 frame:0
ÂÂÂÂÂÂÂÂÂ TX packets:5272 errors:0 dropped:154 overruns:0 carrier:0
ÂÂÂÂÂÂÂÂÂ collisions:0 txqueuelen:32
ÂÂÂÂÂÂÂÂÂ RX bytes:12878 (12.5 KiB)Â TX bytes:457533 (446.8 KiB)

xenbr0 Link encap:Ethernet HWaddr 00:50:56:8B:3A:E4
ÂÂÂÂÂÂÂÂÂ inet addr:10.2.0.52Â Bcast:0.0.0.0Â Mask:255.255.255.0
ÂÂÂÂÂÂÂÂÂ UP BROADCAST RUNNING MULTICASTÂ MTU:1500Â Metric:1
ÂÂÂÂÂÂÂÂÂ RX packets:18955 errors:0 dropped:0 overruns:0 frame:0
ÂÂÂÂÂÂÂÂÂ TX packets:12891 errors:0 dropped:0 overruns:0 carrier:0
ÂÂÂÂÂÂÂÂÂ collisions:0 txqueuelen:0
ÂÂÂÂÂÂÂÂÂ RX bytes:1224607 (1.1 MiB)Â TX bytes:2652734 (2.5 MiB)
--------------------------------------

# ip link
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 16436 qdisc noqueue
ÂÂÂ link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast qlen 1000
ÂÂÂ link/ether 00:50:56:8b:3a:e4 brd ff:ff:ff:ff:ff:ff
3: xenbr0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue
ÂÂÂ link/ether 00:50:56:8b:3a:e4 brd ff:ff:ff:ff:ff:ff
4: vif1.0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast qlen 32
ÂÂÂ link/ether fe:ff:ff:ff:ff:ff brd ff:ff:ff:ff:ff:ff
---------------------------------------------------------------

#vi /etc/sysconfig/network-scripts/ifcfg-eth0 (ON Domain-0 --> 10.2.0.52)
--------------------------------------------

# Advanced Micro Devices [AMD] 79c970 [PCnet32 LANCE]
DEVICE=eth0
BOOTPROTO=none
BROADCAST=10.2.0.255
HWADDR=00:50:56:8b:3a:e4
IPADDR=10.2.0.52
NETMASK=255.255.255.0
NETWORK=10.2.0.0
> GATEWAY=10.2.0.254
TYPE=Ethernet

---------------------------------------------

# arping -b -c 1 -I xenbr0 10.2.0.55 (to the Guest VM)
-------------------------------------
ARPING 10.2.0.55 from 10.2.0.52 xenbr0
Unicast reply from 10.2.0.55 [00:16:3E:07:03:49]Â 1.684ms
Sent 1 probes (1 broadcast(s))
Received 1 response(s)

----------------------------------------------------------
[root@LinuxDB network-scripts]# cd /etc/xen/scripts/
[root@LinuxDB scripts]# ./network-bridge status
============================================================
4: vif1.0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast qlen 32
ÂÂÂ link/ether fe:ff:ff:ff:ff:ff brd ff:ff:ff:ff:ff:ff
3: xenbr0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue
ÂÂÂ link/ether 00:50:56:8b:3a:e4 brd ff:ff:ff:ff:ff:ff
ÂÂÂ inet 10.2.0.52/24 scope global xenbr0

bridge nameÂÂÂÂ bridge idÂÂÂÂÂÂÂÂÂÂÂÂÂÂ STP enabledÂÂÂÂ interfaces
xenbr0ÂÂÂÂÂÂÂÂÂ 8000.0050568b3ae4ÂÂÂÂÂÂ noÂÂÂÂÂÂÂÂÂÂÂÂÂ vif1.0
ÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂ eth0

10.2.0.0/24 dev xenbr0 proto kernel scope link src 10.2.0.52
default via 10.2.0.254 dev xenbr0

Kernel IP routing table
DestinationÂÂÂÂ GatewayÂÂÂÂÂÂÂÂ GenmaskÂÂÂÂÂÂÂÂ Flags Metric RefÂÂÂ Use Iface
10.2.0.0ÂÂÂÂÂÂÂ 0.0.0.0ÂÂÂÂÂÂÂÂ 255.255.255.0ÂÂ UÂÂÂÂ 0ÂÂÂÂÂ 0ÂÂÂÂÂÂÂ 0 xenbr0
0.0.0.0ÂÂÂÂÂÂÂÂ 10.2.0.254ÂÂÂÂÂ 0.0.0.0ÂÂÂÂÂÂÂÂ UGÂÂÂ 0ÂÂÂÂÂ 0ÂÂÂÂÂÂÂ 0 xenbr0
============================================================

In my vm.cfg I added:

vif = ['ip=10.2.0.55,mac=00:16:3e:07:03:49,bridge=xenbr0']
===============================================================

#brctl showmacs xenbr0 (It's actually a very long list out of which I cut the last section that has local = 'yes')
----------------------
 1 00:50:56:8b:3a:e4 yes 0.00
 1 00:50:56:8b:40:87 no 20.43
 1 00:50:56:8b:48:08 no 260.19
 1 00:50:56:8b:48:11 no 152.62
 1 00:50:56:8b:50:d4 no 215.24
 1 00:50:56:8b:68:9a no 27.69
 1 00:50:56:b5:07:e2 no 170.61
 1 00:e0:07:03:40:88 no 5.22
 1 00:e0:4d:8b:8a:89 no 80.37
 1 0a:01:00:00:00:00 no 1.23
 2 fe:ff:ff:ff:ff:ff yes 0.00


I'm also adding a part of the "/var/log/xen/xend.log" In case it helps. If you need to see more from it, i'd be happy to add, but I don't see any errors there.
--------------------------------------------------------

[2010-04-30 11:13:23 2993] INFO (XendDomainInfo:2180) createDevice: vif : {'ip': '10.2.0.55', 'mac': '00:16:3e:07:03:49', 'uuid': 'e475dc37-4529-423f-157a-2b813c14afbc', 'bridge': 'xenbr0'}

[2010-04-30 11:13:23 2993] DEBUG (DevController:95) DevController: writing {'mac': '00:16:3e:07:03:49', 'handle': '0', 'protocol': 'x86_32-abi', 'backend-id': '0', 'state': '1', 'backend': '/local/domain/0/backend/vif/1/0'} to /local/domain/1/device/vif/0.
[2010-04-30 11:13:23 2993] DEBUG (DevController:97) DevController: writing {'bridge': 'xenbr0', 'domain': 'SiebelApp', 'handle': '0', 'uuid': 'e475dc37-4529-423f-157a-2b813c14afbc', 'script': '/etc/xen/scripts/vif-bridge', 'ip': '10.2.0.55', 'mac': '00:16:3e:07:03:49', 'frontend-id': '1', 'state': '1', 'online': '1', 'frontend': '/local/domain/1/device/vif/0'} to /local/domain/0/backend/vif/1/0.
[2010-04-30 11:13:23 2993] INFO (image:391) spawning device models: /usr/lib/xen/bin/qemu-dm ['/usr/lib/xen/bin/qemu-dm', '-d', '1', '-domain-name', 'SiebelApp', '-videoram', '4', '-vnc', '0.0.0.0:0', '-vncunused', '-serial', 'pty', '-M', 'xenpv']

[2010-04-30 11:13:23 2993] INFO (image:440) device model pid: 3367

[2010-04-30 11:13:23 2993] DEBUG (XendDomainInfo:3080) Storing VM details: {'on_xend_stop': 'ignore', 'shadow_memory': '0', 'uuid': '07382902-6cc8-ee56-18ad-f6d759aa3788', 'on_reboot': 'restart', 'start_time': '1272615203.93', 'on_poweroff': 'destroy', 'bootloader_args': '-q', 'on_xend_start': 'ignore', 'on_crash': 'restart', 'xend/restart_count': '0', 'vcpus': '1', 'vcpu_avail': '1', 'bootloader': '/usr/bin/pygrub', 'image': "(linux (kernel ) (videoram 4) (device_model /usr/lib/xen/bin/qemu-dm) (notes (FEATURES 'writable_page_tables|writable_descriptor_tables|auto_translated_physmap|pae_pgdir_above_4gb|supervisor_mode_kernel') (VIRT_BASE 3221225472) (GUEST_VERSION 2.6) (PADDR_OFFSET 3221225472) (GUEST_OS linux) (HYPERCALL_PAGE 3225423872) (LOADER generic) (PAE_MODE yes) (ENTRY 3225419776) (XEN_VERSION xen-3.0)))", 'name': 'SiebelApp'}

[2010-04-30 11:13:23 2993] INFO (image:528) waiting for sentinel_fifo

[2010-04-30 11:13:23 2993] DEBUG (XendDomainInfo:1640) Storing domain details: {'console/ring-ref': '1277431', 'image/entry': '3225419776', 'console/port': '2', 'store/ring-ref': '1277432', 'image/loader': 'generic', 'vm': '/vm/07382902-6cc8-ee56-18ad-f6d759aa3788', 'control/platform-feature-multiprocessor-suspend': '1', 'image/guest-os': 'linux', 'image/features/writable-descriptor-tables': '1', 'image/virt-base': '3221225472', 'memory/target': '1048576', 'image/guest-version': '2.6', 'image/features/supervisor-mode-kernel': '1', 'image/pae-mode': 'yes', 'console/limit': '1048576', 'image/paddr-offset': '3221225472', 'image/hypercall-page': '3225423872', 'cpu/0/availability': 'online', 'image/features/pae-pgdir-above-4gb': '1', 'image/features/writable-page-tables': '1', 'console/type': 'ioemu', 'image/features/auto-translated-physmap': '1', 'name': 'SiebelApp', 'domid': '1', 'image/xen-version': 'xen-3.0', 'store/port': '1'}


_______________________________________________
Xen-users mailing list
Xen-users@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-users

 


Rackspace

Lists.xenproject.org is hosted with RackSpace, monitoring our
servers 24x7x365 and backed by RackSpace's Fanatical Support®.