[Xen-users] Networking problem: "routing" from domUs through dom0

["Jiaqi Tan" <jiaqi.tan@xxxxxxxxx>]

Hi,

I'm trying to get the following setup working: I have 2 physical
machines, both with Xen 3.1.0 running Fedora Core 6, and 2.6.18
kernels, both with 2 NICs each. 1 NIC on each machine is connected to
the Internet (eth0), and 1 NIC on each machine (eth1) is connected on
a private network. I have set up bridging using netdev=eth1 for the
domUs on each dom0. I want the domUs on both machines to have direct
access to each other, which works fine right now via the private
network on the 2nd NIC on each machine, and I also want to access the
Internet on each domU through dom0. I'm trying to do that using IP
masquerading on the dom0s.

Currently, my setup is as follows:
1. Each dom0 has network-bridge netdev=eth1
2. Each dom0 has the FORWARD chain set to ACCEPT, and additionally has
rules for each vif#.0 to allow forwarding (created by xend when I boot
the domUs):

Chain INPUT (policy ACCEPT 325 packets, 30149 bytes)
 pkts bytes target     prot opt in     out     source               destination

Chain FORWARD (policy ACCEPT 2 packets, 168 bytes)
 pkts bytes target     prot opt in     out     source               destination
   16  4760 ACCEPT     all  --  any    any     anywhere             anywhere

Chain OUTPUT (policy ACCEPT 266 packets, 39670 bytes)
 pkts bytes target     prot opt in     out     source               destination

3. In the NAT table, I have:
Chain PREROUTING (policy ACCEPT 12 packets, 776 bytes)
 pkts bytes target     prot opt in     out     source               destination

Chain POSTROUTING (policy ACCEPT 4 packets, 376 bytes)
 pkts bytes target     prot opt in     out     source               destination
    0     0 MASQUERADE  all  --  any    eth0    anywhere             anywhere

Chain OUTPUT (policy ACCEPT 2 packets, 208 bytes)
 pkts bytes target     prot opt in     out     source               destination

4. IPv4 forwarding is enabled as well:
$ sysctl net.ipv4.ip_forward
net.ipv4.ip_forward = 1

5. eth1 on dom0 has 10.1.1.3/255.255.255.0, and eth0 on domU has
10.1.1.103/255.255.255.0. Routing tables look like:

dom0:
Kernel IP routing table
Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
10.1.1.0        0.0.0.0         255.255.255.0   U     0      0        0 eth1
<net of internet connection>     0.0.0.0         ＜netmask＞   U     0
   0        0 eth0
0.0.0.0         <internet gateway>     0.0.0.0         UG    0      0
      0 eth0

domU:
Kernel IP routing table
Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
10.1.1.0        0.0.0.0         255.255.255.0   U     0      0        0 eth0
0.0.0.0         10.1.1.3        0.0.0.0         UG    0      0        0 eth0

Also, xenbr0 and peth1 have no IPs, and eth1 has 10.1.1.3/255.255.255.0.

I can ping both ways (ping the dom0 eth1 IP from domU, and the domU IP
from dom0).

But when I try to ping an external host from the domU, there's simply
no response. tcpdump shows the packet appearing on eth1, but not eth0.
When I do iptables -t nat -L -v, none of the packets matches the -o
eth0 -j MASQUERADE rule, but when I insert a rule for -o xenbr0, all
the packets are seen going out through xenbr0 and matching that rule.
All my packets are getting from domU to dom0, and it hits the FORWARD
and POSTROUTING chains, but the packets are all going out through
xenbr0 although xenbr0 is not even in the routing tables.

Any ideas?

Thanks in advance.

Jiaqi Tan

_______________________________________________
Xen-users mailing list
Xen-users@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-users