[Top] [All Lists]
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [Xen-users] network (NAT?) problem

Marcus
OK, I think I know what's happening here. Only the first packet from a stream ("connection") traverses the PRE- and POSTROUTING chains. Subsequent packets will be processed according to the connection tracking data.
In your case (which I've replicated and observed the same behaviour) the first packet from a connection initiated by dom2 enters dom0 for bridging back to dom1 via xen-br1. This packet traverses the PREROUTING, FORWARD and POSTROUTING chains and gets connection-tracked (yes, packets that are only bridged do traverse these chains). When dom1 routes that packet to dom0, dom0 finds that the packet matches a tracked connection so does not pass it over the PRE- or POSTROUTING chains and therefore doesn't hit the MASQUERADE rule.
A possible workaround is to use the NOTRACK target (does what it says on the tin, doesn't track that packet - I'm assuming this does not affect the second traversal which we do want to track) for packets that are just coming in to dom0 for bridging on xen-br1. This target can only be used on the 'raw' table. This requires CONFIG_IP_NF_RAW which I don't have in the kernel on the machine I'm using for testing, and it's a production machine that I can't reboot right now. Something like the following should work: 

iptables -t raw -A PREROUTING -i xen-br1 -J NOTRACK

Please try this and let me know if it works.

Regards,

James

Markus Lude wrote:

On Thu, Jun 23, 2005 at 10:55:11AM +0100, James Bulpin wrote:


Markus,
You say the routing of packets between dom2 and dom0 doesn't work as intended - can you elaborate on this, i.e. where do packets get to, do you see them if you tcpdump eth1 and eth0 in dom1? 


Ehm, the routing between dom2 and dom0 _does_ work, but something with NAT
in dom0 seems to go wrong for packets from dom2. I see the traffic between
dom0 and dom2 on the interfaces in dom1.
Do you have any IP tables rules in dom0 that would affect packets on xen-br1? 

Can you post the following for all domains:
- iptables details (both nat and filter tables)
- routing tables
- ifconfig
- cat /proc/sys/net/ipv4/ip_forward



dom2

xsarge2:~# iptables -nvL -t filter
modprobe: QM_MODULES: Function not implemented

modprobe: QM_MODULES: Function not implemented

modprobe: Can't locate module ip_tables
iptables v1.2.11: can't initialize iptables table `filter': iptables who? (do 
you need to insmod?)
Perhaps iptables or your kernel needs to be upgraded.

xsarge2:~# route -n
Kernel IP routing table
Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
172.30.1.0      0.0.0.0         255.255.255.0   U     0      0        0 eth0
0.0.0.0         172.30.1.21     0.0.0.0         UG    0      0        0 eth0

xsarge2:~# ifconfig
eth0 Link encap:Ethernet HWaddr 0A:00:00:00:01:20 inet addr:172.30.1.22 Bcast:172.30.255.255 Mask:255.255.255.0 
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:61 errors:0 dropped:0 overruns:0 frame:0
          TX packets:79 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000 RX bytes:5866 (5.7 KiB) TX bytes:6622 (6.4 KiB)
lo Link encap:Local Loopback inet addr:127.0.0.1 Mask:255.0.0.0 
          UP LOOPBACK RUNNING  MTU:16436  Metric:1
          RX packets:22 errors:0 dropped:0 overruns:0 frame:0
          TX packets:22 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0 RX bytes:2328 (2.2 KiB) TX bytes:2328 (2.2 KiB) 

xsarge2:~# cat /proc/sys/net/ipv4/ip_forward
1

---------------------------------------
dom1

xsarge1:~# iptables -nvL -t filter
modprobe: QM_MODULES: Function not implemented

modprobe: QM_MODULES: Function not implemented

modprobe: Can't locate module ip_tables
iptables v1.2.11: can't initialize iptables table `filter': iptables who? (do 
you need to insmod?)
Perhaps iptables or your kernel needs to be upgraded.

xsarge1:~# route -n
Kernel IP routing table
Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
172.30.1.0      0.0.0.0         255.255.255.0   U     0      0        0 eth0
172.30.0.0      0.0.0.0         255.255.255.0   U     0      0        0 eth1
0.0.0.0         172.30.0.1      0.0.0.0         UG    0      0        0 eth1

xsarge1:~# ifconfig
eth0 Link encap:Ethernet HWaddr 0A:00:00:00:01:10 inet addr:172.30.1.21 Bcast:172.30.255.255 Mask:255.255.255.0 
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:61 errors:0 dropped:0 overruns:0 frame:0
          TX packets:72 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000 RX bytes:5866 (5.7 KiB) TX bytes:6328 (6.1 KiB)
eth1 Link encap:Ethernet HWaddr 0A:00:00:00:10:11 inet addr:172.30.0.21 Bcast:172.30.255.255 Mask:255.255.255.0 
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:17 errors:0 dropped:0 overruns:0 frame:0
          TX packets:22 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000 RX bytes:1330 (1.2 KiB) TX bytes:1820 (1.7 KiB)
lo Link encap:Local Loopback inet addr:127.0.0.1 Mask:255.0.0.0 
          UP LOOPBACK RUNNING  MTU:16436  Metric:1
          RX packets:30 errors:0 dropped:0 overruns:0 frame:0
          TX packets:30 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0 RX bytes:2996 (2.9 KiB) TX bytes:2996 (2.9 KiB) 

xsarge1:~# cat /proc/sys/net/ipv4/ip_forward
1

---------------------------------------
dom0: arras

arras:~# iptables -nvL -t filter
Chain INPUT (policy ACCEPT 6004 packets, 450K bytes)
pkts bytes target prot opt in out source destination 
Chain FORWARD (policy ACCEPT 123 packets, 10332 bytes)
pkts bytes target prot opt in out source destination 
Chain OUTPUT (policy ACCEPT 5312 packets, 338K bytes)
pkts bytes target prot opt in out source destination arras:~# iptables -nvL -t nat Chain PREROUTING (policy ACCEPT 1866 packets, 401K bytes) pkts bytes target prot opt in out source destination 
Chain POSTROUTING (policy ACCEPT 82 packets, 6653 bytes)
pkts bytes target prot opt in out source destination 0 0 MASQUERADE all -- * xen-br0 172.30.0.0/16 0.0.0.0/0 
Chain OUTPUT (policy ACCEPT 22 packets, 1613 bytes)
pkts bytes target prot opt in out source destination arras:~# route -n 
Kernel IP routing table
Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
172.30.1.0      172.30.0.21     255.255.255.0   UG    0      0        0 xen-br10
172.30.0.0      0.0.0.0         255.255.255.0   U     0      0        0 xen-br10
134.2.8.0       0.0.0.0         255.255.248.0   U     0      0        0 xen-br0
0.0.0.0         134.2.15.254    0.0.0.0         UG    0      0        0 xen-br0

arras:~# ifconfig
eth0 Link encap:Ethernet HWaddr 00:0D:56:83:0C:A9 inet addr:134.2.11.153 Bcast:134.2.15.255 Mask:255.255.248.0 
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:49972 errors:0 dropped:0 overruns:0 frame:0
          TX packets:1000 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000 RX bytes:4690589 (4.4 MiB) TX bytes:95473 (93.2 KiB) Base address:0xdf40 Memory:feae0000-feb00000 lo Link encap:Local Loopback inet addr:127.0.0.1 Mask:255.0.0.0 
          UP LOOPBACK RUNNING  MTU:16436  Metric:1
          RX packets:4937 errors:0 dropped:0 overruns:0 frame:0
          TX packets:4937 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0 RX bytes:308890 (301.6 KiB) TX bytes:308890 (301.6 KiB)
vif1.0 Link encap:Ethernet HWaddr FE:FF:FF:FF:FF:FF UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 
          RX packets:116 errors:0 dropped:0 overruns:0 frame:0
          TX packets:61 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0 RX bytes:8176 (7.9 KiB) TX bytes:5866 (5.7 KiB)
vif1.1 Link encap:Ethernet HWaddr FE:FF:FF:FF:FF:FF UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 
          RX packets:93 errors:0 dropped:0 overruns:0 frame:0
          TX packets:65 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0 RX bytes:8050 (7.8 KiB) TX bytes:5866 (5.7 KiB)
vif2.0 Link encap:Ethernet HWaddr FE:FF:FF:FF:FF:FF UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 
          RX packets:105 errors:0 dropped:0 overruns:0 frame:0
          TX packets:61 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0 RX bytes:7714 (7.5 KiB) TX bytes:5866 (5.7 KiB)
xen-br0 Link encap:Ethernet HWaddr 00:0D:56:83:0C:A9 inet addr:134.2.11.153 Bcast:134.2.15.255 Mask:255.255.255.255 
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:46551 errors:0 dropped:0 overruns:0 frame:0
          TX packets:989 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0 RX bytes:2681480 (2.5 MiB) TX bytes:89935 (87.8 KiB)
xen-br1 Link encap:Ethernet HWaddr FE:FF:FF:FF:FF:FF UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 
          RX packets:1 errors:0 dropped:0 overruns:0 frame:0
          TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0 RX bytes:28 (28.0 b) TX bytes:0 (0.0 b)
xen-br10 Link encap:Ethernet HWaddr FE:FF:FF:FF:FF:FF inet addr:172.30.0.1 Bcast:172.30.255.255 Mask:255.255.255.0 
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:65 errors:0 dropped:0 overruns:0 frame:0
          TX packets:65 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0 RX bytes:5964 (5.8 KiB) TX bytes:5866 (5.7 KiB) 

arras:~# cat /proc/sys/net/ipv4/ip_forward
1



And brctl show for dom0.



arras:~# brctl show
bridge name     bridge id               STP enabled     interfaces
xen-br0         8000.000d56830ca9       no              eth0
xen-br10                8000.feffffffffff       no              vif1.1
xen-br1         8000.feffffffffff       no              vif1.0
                                                        vif2.0

hmm, is it normal, that xen-br1 and xen-br10 have the same bridge id?
If not, any hints were to change them?



Thanks,

James



Regards,
Markus

Markus Lude wrote:


Hello,

my current network setup looks like the following:

  dom2-eth0 (172.30.1.22) netmask /24
      |
      | xen-br1 (no ip)
      |
  dom1-eth0 (172.30.1.21) netmask /24
  dom1-eth1 (172.30.0.21) netmask /24
      |
      | xen-br10 (172.30.0.1) netmask /24
      |
    dom0
      |
      | xen-br0 (normal IP)
      |
     eth0 (same IP as xen-br0)
      |
      |
     LAN

Basically I want to route all traffic between dom0 and dom2 through dom1.
This does work as intended.
I further would like to have access to the LAN from dom1 and dom2 through
NAT in dom0. It works for dom1, but not for dom2. The addresses of packets



from dom2 were not changed (noticed with tcpdump -n -i eth0 in dom0).



In dom0 NAT is set up with:
iptables -t nat -A POSTROUTING -s 172.30.0.0/16 -o xen-br0 -j MASQUERADE

If I leave out the -s parameter nothing changes.

I'm running debian sarge in all doms and xen-2.0.6.

Any suggestions? What am I missing?

Regards,
Markus Lude




_______________________________________________
Xen-users mailing list
Xen-users@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-users


_______________________________________________
Xen-users mailing list
Xen-users@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-users

 

©2013 Xen Project, A Linux Foundation Collaborative Project. All Rights Reserved.
Linux Foundation is a registered trademark of The Linux Foundation.
Xen Project is a trademark of The Linux Foundation.

Rackspace

Lists.xenproject.org is hosted with RackSpace, monitoring our
servers 24x7x365 and backed by RackSpace's Fanatical Support®.