Hi,
Still facing this problem, can anyone take a look at the
latest info I have?
The full story is below.
eth0 interface (Xen Virtual Interface) on the VM is showing
growing Rx bytes on every ifconfig command, but Tx bytes is not moving.
Also: ‘qemu-dm.log’ maybe has something:
domid: 1
Warning: vlan 0 is not connected to host network
Watching /local/domain/0/device-model/1/logdirty/next-active
Watching /local/domain/0/device-model/1/command
char device redirected to /dev/pts/1
/builddir/build/BUILD/xen-3.4.0/tools/ioemu-dir/hw/xen_blktap.c:628:
Init blktap pipes
xen be: console-0: backend state: Initialising ->
InitWait
xen be: console-0: frontend not ready, ignoring
xen be: console-0: bind evtchn port 21
xen be: console-0: ring mfn 1277431, remote port 2, local
port 21, limit 1048576
xen be: console-0: backend state: InitWait -> Connected
xs_read(): vncpasswd get error.
/vm/edc7ed59-91d2-1c7f-9523-dd2f102eb74e/vncpasswd.
xen be: console-0: backend update: state
xen be: console-0: backend update: hotplug-status
xen be: console-0: backend update: state
xen be: console-0: backend update: state
xen be: console-0: frontend update: vnc-port
xen be: console-0: frontend update: tty
xen be: console-0: backlog piling up, nobody listening?
If I try to ping the vm 10.2.0.55 inside xen(102.0.52) and monitor
it with tcpdump –nvvi xenbr0 10.2.0.110 | grep –i 10.2.0.55
I see:
# tcpdump: listening on xenbr0, link-type EN10MB (Ethernet),
capture size 96 bytes
19:49:25.256958 arp who-has 10.2.0.55 tell 10.2.0.110
19:49:25.257378 arp reply 10.2.0.55 is-at 00:16:3e:07:03:49
So it is discoverable isn’t it? Why does the Tx byte
stands still?
Also following this immediately afterwards with ‘ip
neigh’ on the vm(10.2.0.55) returns
10.2.0.110 dev eth0 lladr 00:26:..etc STALE which I understand
is still usable, right?
Thanks!
=================================
My Xen includes 2 guests. Xen itself (10.2.0.52) gets free
access to the outside world and to its guests.
Both guests however (10.2.0.54/10.2.0.55) see each other but
stay under house arrest!
Not a single ping manages to go past the bridge (xenbr0) and
get an answer from the default gateway (10.2.0.254.)
On the guest I keep seeing 'Destination Host Unreachable',
i.e.:
#traceroute 10.2.0.254 comes back with the !H message.
I have pasted all the info I gathered in attempt to
understand who locks the guests in, but will be happy to add any further output
required to get this resolved.
I followed so many manuals and links, one that seriously
caught my attention had to do with MAC addresses issue. Could this be my case?.
Many thanks for any assistance,
Liron.
#tcpdump -nvvi xenbr0 | grep -i 10.2.0.55
---------------------------------------------
11:19:41.548806 arp who-has 10.2.0.254 tell 10.2.0.55
11:19:41.549059 IP (tos 0x0, ttl 64, id 0, offset 0,
flags [DF], proto: ICMP (1), length: 84) 10.2.0.55 > 10.2.0.254: ICMP echo
request, id 19463, seq 19, length 64
#tcpdump -nvvi vif1.0 host 10.2.0.55
------------------------------------
11:21:23.403063 IP (tos 0x0, ttl 64, id 0, offset 0,
flags [DF], proto: ICMP (1), length: 84) 10.2.0.55 > 10.2.0.254: ICMP echo
request, id 22279, seq 7, length 64
#tcpdump -vv -n -c 1000 | grep -i 10.2.0.254
--------------------------------------------
11:22:46.256389 arp who-has 10.2.0.254 tell 10.2.0.55
11:22:46.256662 IP (tos 0x0, ttl 64, id 0, offset 0,
flags [DF], proto: ICMP (1), length: 84) 10.2.0.55 > 10.2.0.254: ICMP echo
request, id 23047, seq 8, length 64
# iptables -L -vv -n
--------------------
Chain INPUT (policy ACCEPT 10869 packets, 769K bytes)
pkts bytes target prot opt
in out
source
destination
0 0
ACCEPT tcp --
* *
0.0.0.0/0
0.0.0.0/0 state NEW
tcp dpt:8003
Chain FORWARD (policy ACCEPT 0 packets, 0 bytes)
pkts bytes target prot opt
in out
source
destination
0 0
ACCEPT all --
* *
0.0.0.0/0
0.0.0.0/0 state
RELATED,ESTABLISHED PHYSDEV match --physdev-out vif1.0
0 0
ACCEPT udp --
* *
0.0.0.0/0
0.0.0.0/0 PHYSDEV
match --physdev-in vif1.0 udp spt:68 dpt:67
0 0
ACCEPT all --
* *
0.0.0.0/0
0.0.0.0/0 state
RELATED,ESTABLISHED PHYSDEV match --physdev-out vif1.0
0 0
ACCEPT all --
* *
10.2.0.55
0.0.0.0/0 PHYSDEV
match --physdev-in vif1.0
Chain OUTPUT (policy ACCEPT 8294 packets, 2149K bytes)
pkts bytes target prot opt
in out
source
destination
----------------------------------------------
As in
http://wiki.xensource.com/xenwiki/XenNetworking#head-602e26cd4a03b992f3938fe1bea03fa0fea0ed8b
I tried:
#echo "1" > /proc/sys/net/ipv4/ip_forward
I also tried:
iptables -A FORWARD -m physdev --physdev-in eth0
--physdev-out '!' eth0 -j ACCEPT
iptables -A FORWARD -m physdev --physdev-out eth0
--physdev-in '!' eth0 -j ACCEPT
Still Nothing!
----------------------------------------------
# cat /etc/xen/xend-config.sxp | grep -v "^#" |
grep "[a-z]"
-----------------------------------------------------------
(logfile /var/log/xen/xend.log)
(loglevel DEBUG)
(network-script network-bridge)
(vif-script vif-bridge)
(dom0-min-mem 196)
(enable-dom0-ballooning yes)
(dom0-cpus 0)
(vnc-listen '0.0.0.0')
(vncpasswd '')
(xend-domains-lock-path /opt/ovs-agent-2.3/utils/dlm.py)
-----------------------------------------------
# brctl show
bridge name bridge
id
STP enabled interfaces
xenbr0
8000.0050568b3ae4
no
vif1.0
eth0
--------------------------------------------------------------------
# xm list
Name
ID Mem VCPUs State
Time(s)
Domain-0
0 564 2
r----- 67.2
SiebelApp
1 1024 1
-b---- 32.6
---------------------------------------------------------------------
#xm network-list 1
-------------------
Idx BE MAC
Addr. handle state evt-ch tx-/rx-ring-ref BE-path
0 0 00:16:3e:07:03:49
0 4 10
815 /816 /local/domain/0/backend/vif/1/0
--------------------
# xm network-list 1 -l
(0
((mac 00:16:3e:07:03:49)
(handle 0)
(protocol
x86_32-abi)
(backend-id 0)
(state 4)
(backend
/local/domain/0/backend/vif/1/0)
(tx-ring-ref 815)
(rx-ring-ref 816)
(event-channel
10)
(request-rx-copy
1)
(feature-rx-notify
1)
(feature-sg 1)
(feature-gso-tcpv4 1)
)
)
-------------------------------
# ifconfig -a --> ON 10.2.0.52 (Domain-0)
------------------------------------
eth0 Link encap:Ethernet
HWaddr 00:50:56:8B:3A:E4
UP
BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX
packets:18937 errors:0 dropped:0 overruns:0 frame:0
TX
packets:13043 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX
bytes:1484679 (1.4 MiB) TX bytes:2669625 (2.5 MiB)
Interrupt:17 Base address:0x2000
lo Link
encap:Local Loopback
inet
addr:127.0.0.1 Mask:255.0.0.0
UP
LOOPBACK RUNNING MTU:16436 Metric:1
RX packets:124
errors:0 dropped:0 overruns:0 frame:0
TX
packets:124 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX
bytes:17724 (17.3 KiB) TX bytes:17724 (17.3 KiB)
vif1.0 Link encap:Ethernet HWaddr
FE:FF:FF:FF:FF:FF
UP
BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX
packets:136 errors:0 dropped:0 overruns:0 frame:0
TX
packets:5272 errors:0 dropped:154 overruns:0 carrier:0
collisions:0 txqueuelen:32
RX bytes:12878
(12.5 KiB) TX bytes:457533 (446.8 KiB)
xenbr0 Link encap:Ethernet HWaddr
00:50:56:8B:3A:E4
inet
addr:10.2.0.52 Bcast:0.0.0.0 Mask:255.255.255.0
UP
BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX
packets:18955 errors:0 dropped:0 overruns:0 frame:0
TX
packets:12891 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX
bytes:1224607 (1.1 MiB) TX bytes:2652734 (2.5 MiB)
--------------------------------------
# ip link
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 16436 qdisc noqueue
link/loopback 00:00:00:00:00:00 brd
00:00:00:00:00:00
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500
qdisc pfifo_fast qlen 1000
link/ether 00:50:56:8b:3a:e4 brd
ff:ff:ff:ff:ff:ff
3: xenbr0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500
qdisc noqueue
link/ether 00:50:56:8b:3a:e4 brd
ff:ff:ff:ff:ff:ff
4: vif1.0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500
qdisc pfifo_fast qlen 32
link/ether fe:ff:ff:ff:ff:ff brd
ff:ff:ff:ff:ff:ff
---------------------------------------------------------------
#vi /etc/sysconfig/network-scripts/ifcfg-eth0 (ON Domain-0
--> 10.2.0.52)
--------------------------------------------
# Advanced Micro Devices [AMD] 79c970 [PCnet32 LANCE]
DEVICE=eth0
BOOTPROTO=none
BROADCAST=10.2.0.255
HWADDR=00:50:56:8b:3a:e4
IPADDR=10.2.0.52
NETMASK=255.255.255.0
NETWORK=10.2.0.0
>
GATEWAY=10.2.0.254
TYPE=Ethernet
---------------------------------------------
# arping -b -c 1 -I xenbr0 10.2.0.55 (to the Guest VM)
-------------------------------------
ARPING 10.2.0.55 from 10.2.0.52 xenbr0
Unicast reply from 10.2.0.55 [00:16:3E:07:03:49]
1.684ms
Sent 1 probes (1 broadcast(s))
Received 1 response(s)
----------------------------------------------------------
[root@LinuxDB network-scripts]# cd /etc/xen/scripts/
[root@LinuxDB scripts]# ./network-bridge status
============================================================
4: vif1.0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500
qdisc pfifo_fast qlen 32
link/ether fe:ff:ff:ff:ff:ff brd
ff:ff:ff:ff:ff:ff
3: xenbr0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500
qdisc noqueue
link/ether 00:50:56:8b:3a:e4 brd
ff:ff:ff:ff:ff:ff
inet 10.2.0.52/24 scope global xenbr0
bridge name bridge
id STP
enabled interfaces
xenbr0
8000.0050568b3ae4
no
vif1.0
eth0
10.2.0.0/24 dev xenbr0 proto kernel scope
link src 10.2.0.52
default via 10.2.0.254 dev xenbr0
Kernel IP routing table
Destination
Gateway
Genmask Flags Metric
Ref Use Iface
10.2.0.0
0.0.0.0
255.255.255.0 U
0 0 0
xenbr0
0.0.0.0
10.2.0.254
0.0.0.0 UG
0 0 0
xenbr0
============================================================
In my vm.cfg I added:
vif = ['ip=10.2.0.55,mac=00:16:3e:07:03:49,bridge=xenbr0']
===============================================================
#brctl showmacs xenbr0 (It's actually a very long list out
of which I cut the last section that has local = 'yes')
----------------------
1
00:50:56:8b:3a:e4
yes
0.00
1
00:50:56:8b:40:87
no
20.43
1
00:50:56:8b:48:08
no
260.19
1
00:50:56:8b:48:11
no
152.62
1
00:50:56:8b:50:d4
no
215.24
1
00:50:56:8b:68:9a
no
27.69
1
00:50:56:b5:07:e2
no
170.61
1
00:e0:07:03:40:88 no
5.22
1
00:e0:4d:8b:8a:89
no
80.37
1
0a:01:00:00:00:00
no
1.23
2
fe:ff:ff:ff:ff:ff
yes
0.00
I'm also adding a part of the
"/var/log/xen/xend.log" In case it helps. If you need to see more
from it, i'd be happy to add, but I don't see any errors there.
--------------------------------------------------------
[2010-04-30 11:13:23 2993] INFO (XendDomainInfo:2180)
createDevice: vif : {'ip': '10.2.0.55', 'mac': '00:16:3e:07:03:49', 'uuid':
'e475dc37-4529-423f-157a-2b813c14afbc', 'bridge': 'xenbr0'}
[2010-04-30 11:13:23 2993] DEBUG (DevController:95)
DevController: writing {'mac': '00:16:3e:07:03:49', 'handle': '0', 'protocol':
'x86_32-abi', 'backend-id': '0', 'state': '1', 'backend':
'/local/domain/0/backend/vif/1/0'} to /local/domain/1/device/vif/0.
[2010-04-30 11:13:23 2993] DEBUG (DevController:97)
DevController: writing {'bridge': 'xenbr0', 'domain': 'SiebelApp', 'handle':
'0', 'uuid': 'e475dc37-4529-423f-157a-2b813c14afbc', 'script':
'/etc/xen/scripts/vif-bridge', 'ip': '10.2.0.55', 'mac': '00:16:3e:07:03:49',
'frontend-id': '1', 'state': '1', 'online': '1', 'frontend':
'/local/domain/1/device/vif/0'} to /local/domain/0/backend/vif/1/0.
[2010-04-30 11:13:23 2993] INFO (image:391) spawning device
models: /usr/lib/xen/bin/qemu-dm ['/usr/lib/xen/bin/qemu-dm', '-d', '1',
'-domain-name', 'SiebelApp', '-videoram', '4', '-vnc', '0.0.0.0:0',
'-vncunused', '-serial', 'pty', '-M', 'xenpv']
[2010-04-30 11:13:23 2993] INFO (image:440) device model
pid: 3367
[2010-04-30 11:13:23 2993] DEBUG (XendDomainInfo:3080)
Storing VM details: {'on_xend_stop': 'ignore', 'shadow_memory': '0', 'uuid':
'07382902-6cc8-ee56-18ad-f6d759aa3788', 'on_reboot': 'restart', 'start_time':
'1272615203.93', 'on_poweroff': 'destroy', 'bootloader_args': '-q',
'on_xend_start': 'ignore', 'on_crash': 'restart', 'xend/restart_count': '0',
'vcpus': '1', 'vcpu_avail': '1', 'bootloader': '/usr/bin/pygrub', 'image':
"(linux (kernel ) (videoram 4) (device_model /usr/lib/xen/bin/qemu-dm)
(notes (FEATURES
'writable_page_tables|writable_descriptor_tables|auto_translated_physmap|pae_pgdir_above_4gb|supervisor_mode_kernel')
(VIRT_BASE 3221225472) (GUEST_VERSION 2.6) (PADDR_OFFSET 3221225472) (GUEST_OS
linux) (HYPERCALL_PAGE 3225423872) (LOADER generic) (PAE_MODE yes) (ENTRY
3225419776) (XEN_VERSION xen-3.0)))", 'name': 'SiebelApp'}
[2010-04-30 11:13:23 2993] INFO (image:528) waiting for
sentinel_fifo
[2010-04-30 11:13:23 2993] DEBUG (XendDomainInfo:1640)
Storing domain details: {'console/ring-ref': '1277431', 'image/entry':
'3225419776', 'console/port': '2', 'store/ring-ref': '1277432', 'image/loader':
'generic', 'vm': '/vm/07382902-6cc8-ee56-18ad-f6d759aa3788',
'control/platform-feature-multiprocessor-suspend': '1', 'image/guest-os':
'linux', 'image/features/writable-descriptor-tables': '1', 'image/virt-base':
'3221225472', 'memory/target': '1048576', 'image/guest-version': '2.6',
'image/features/supervisor-mode-kernel': '1', 'image/pae-mode': 'yes',
'console/limit': '1048576', 'image/paddr-offset': '3221225472',
'image/hypercall-page': '3225423872', 'cpu/0/availability': 'online',
'image/features/pae-pgdir-above-4gb': '1', 'image/features/writable-page-tables':
'1', 'console/type': 'ioemu', 'image/features/auto-translated-physmap': '1',
'name': 'SiebelApp', 'domid': '1', 'image/xen-version': 'xen-3.0',
'store/port': '1'}
