Re: [Xen-users] combination of bridged and routed networking problem

[b52@xxxxxxxxx]

> Hi,
>
> I have strange challenge regarding routing/networking on a xen host
> (called tachyon).
> We have setup the host using ubuntu 7.10 (gutsy), because this
> is Debian-based and has xen3.1 included.
>
> The vms should use either bridged network or routed network.
> Therefore, we used
> (network-script network-bridge)
> (vif-script vif-bridge)
> in xend-config.sxp.
>
> Two vms (called etch32 and etch64) are connected with vif2.0 and vif3.0
> to xenbr0. They have got IP addresses via DHCP and ip works
> well to and from these vms.
>
> The third vm (called lenny32) should be connected using a routed config.
> Therefore, we have deleted vif4.0 from the bridge and activated
> ARP, NAT, ... using the following commands on Dom0 (tachyon):
>
> ip link set vif4.0 arp on
> ip link set vif4.0 multicast on
> ip addr flush vif4.0
> ip link set vif4.0 addr 00:1E:0B:70:F6:9a
> ifconfig vif4.0 192.168.0.1
>
> iptables -t nat -A POSTROUTING -o eth0 -j MASQUERADE
> iptables -t nat -A PREROUTING -i xenbr0 -p tcp -m tcp --dport 222 -j DNAT
> --to-destination 192.168.0.2:22
> iptables -t nat -A PREROUTING -i xenbr0 -p tcp -m tcp --dport 443 -j DNAT
> --to-destination 192.168.0.2:443
>
> echo 1 >  /proc/sys/net/ipv4/ip_forward
>
> On lenny32, we have set the IP of eth0 to 192.168.0.2 statically.
>
> IP connections initiated from lenny32 work well but inbound connections
> from other hosts trying to connect to either port 222 or 443 on tachyon do
> not. These connections should be directed via DNAT to lenny32. The TCP/IP
> handshake works but the connection could not be established.
> A TCP-Reset was send by the client.
>
> Output of "tcpdump -i eth0 -s 0" at lenny32:
> 13:19:14.196801 arp who-has lenny32 tell 192.168.0.1
> 13:19:14.196827 arp reply lenny32 is-at 00:16:3e:54:f1:11 (oui Unknown)
> 13:19:14.196859 IP pc63002738.1054 > lenny32.ssh: S
> 1817694704:1817694704(0) win 65535 <mss 1460,nop,nop,sackOK>
> 13:19:14.196890 IP lenny32.ssh > pc63002738.wl.1054: S
> 1304397835:1304397835(0) ack 1817694705 win 5840 <mss 1460,nop,nop,sackOK>
> 13:19:14.197074 IP pc63002738.1054 > lenny32.ssh: . ack 1 win 65535
> 13:19:14.203841 IP lenny32.ssh > pc63002738.wl.1054: P 1:32(31) ack 1 win
> 5840 (SSH: Server Protocol: SSH-2.0-OpenSSH_4.7p1 Debian-2)
> 13:19:14.204048 IP pc63002738.1054 > lenny32.ssh: R
> 1817694705:1817694705(0) win 0 (Cients sends RESET !!???)
>
> A direct ssh connection from tachyon to lenny32 works!
> At lenny32 neither /etc/hosts.allow nor /etc/hosts.deny contains any
> entries.
>
> A similiar behaviour could be observed when connecting to 443 on tachyon.
> The command "iptables -Lvn" in the nat table shows that the DNAT rules
> match, the policy of all
> chains in the filter table is ACCEPT)
>
> Any help is appreciated !!
>
> Thanks in advance,
>
>  Valentin Rottmann
>
> ------------------------------------------------
> # etch64: fake eth0 -> vif2.0 -+
> #                              |
> # etch32: fake eth0 -> vif3.0 -+
> #                              |
> #                           xenbr0 -> peth0 -> the network
> #                              |
> #tachyon: fake eth0 -> vif0.0 -+
> #               |
> #  (routing, MASQUERADING, DNAT)
> #               |
> #           vif4.0  <-> fake eth0 (lenny32)
>
>
> root@tachyon:~# uname -a
> Linux tachyon 2.6.22-14-xen #1 SMP Tue Feb 12 04:26:15 UTC 2008 x86_64
> GNU/Linux
>
> root@tachyon:~# dpkg -l | grep xen
> ii  libxen3.1                                  3.1.0-0ubuntu18
> library interface for Xen, a Virtual Machine
> ii  linux-headers-2.6.22-14-xen                2.6.22-14.52
> Linux kernel headers for version 2.6.22 on T
> ii  linux-image-2.6.22-14-xen                  2.6.22-14.52
> Linux kernel image for version 2.6.22 on Thi
> ii  linux-image-xen                            2.6.22.14.21
> Linux kernel image on Xen
> ii  linux-restricted-modules-2.6.22-14-xen     2.6.22.4-14.10
> Non-free Linux 2.6.22 modules on Xen
> ii  linux-restricted-modules-xen               2.6.22.14.21
> Restricted Linux modules on Xen
> ii  linux-ubuntu-modules-2.6.22-14-xen         2.6.22-14.37
> Ubuntu supplied Linux modules for version 2.
> ii  linux-xen                                  2.6.22.14.21
> Complete Linux kernel on Xen
> ii  python-xen-3.1                             3.1.0-0ubuntu18
> python bindings for Xen, a Virtual Machine M
> ii  xen-hypervisor-3.1                         3.1.0-0ubuntu18         The
> Xen Hypervisor for i386, amd64 amd lpia
> ii  xen-ioemu-3.1                              3.1.0-0ubuntu18         XEN
> administrative tools
> ii  xen-utils-3.1                              3.1.0-0ubuntu18         XEN
> administrative tools
>
> root@tachyon:~# ip route show
> 192.168.0.0/24 dev vif4.0  proto kernel  scope link  src 192.168.0.1
> 10.35.18.0/24 dev eth0  proto kernel  scope link  src 10.35.18.38
> default via 10.32.18.1 dev eth0  metric 100
>
> root@tachyon:~# brctl show
> bridge name     bridge id               STP enabled     interfaces
> xenbr0          8000.feffffffffff       no              vif0.0
>                                                         peth0
>                                                         vif2.0
>                                                         vif3.0
> root@tachyon:~# iptables -t nat -L -v -n
> Chain PREROUTING (policy ACCEPT 1595 packets, 112K bytes)
>  pkts bytes target     prot opt in     out     source
> destination
>  1112 75928 LOG        0    --  *      *       0.0.0.0/0
> 0.0.0.0/0           LOG flags 0 level 4
>    15   860 DNAT       tcp  --  xenbr0 *       0.0.0.0/0
> 0.0.0.0/0           tcp dpt:222 to:192.168.0.2:22
>    32  1536 DNAT       tcp  --  xenbr0 *       0.0.0.0/0
> 0.0.0.0/0           tcp dpt:443 to:192.168.0.2:443
>
> Chain POSTROUTING (policy ACCEPT 6785 packets, 418K bytes)
>  pkts bytes target     prot opt in     out     source
> destination
>  1058 75471 MASQUERADE  0    --  *      eth0    0.0.0.0/0
> 0.0.0.0/0

Well, actually I don't see the reason but your NAT is quite strange!?
Shouldn't you only DNAT for eth0 port 222?
like: iptables -A PREROUTING -i eth0 -p tcp --dport 222 -j DNAT
--to-destination 192.168.0.2:22
Don't do this at the bridge and I think you don't need masquerade either.

Regards,
Holger

> Chain OUTPUT (policy ACCEPT 5342 packets, 313K bytes)
>  pkts bytes target     prot opt in     out     source
> destination
>
> root@tachyon:~# iptables  -L -v -n
> Chain INPUT (policy ACCEPT 97072 packets, 7289K bytes)
>  pkts bytes target     prot opt in     out     source
> destination
>
> Chain FORWARD (policy ACCEPT 1649K packets, 100M bytes)
>  pkts bytes target     prot opt in     out     source
> destination
>     0     0 ACCEPT     0    --  *      *       0.0.0.0/0
> 0.0.0.0/0           PHYSDEV match --physdev-in vif2.0
>     7  1489 ACCEPT     0    --  *      *       0.0.0.0/0
> 0.0.0.0/0           PHYSDEV match --physdev-in vif3.0
>    90  6972 ACCEPT     0    --  *      *       192.168.0.2
> 0.0.0.0/0           PHYSDEV match --physdev-in vif4.0
>     0     0 ACCEPT     udp  --  *      *       0.0.0.0/0
> 0.0.0.0/0           PHYSDEV match --physdev-in vif4.0 udp spt:68
> dpt:67
>
> Chain OUTPUT (policy ACCEPT 81259 packets, 21M bytes)
>  pkts bytes target     prot opt in     out     source
> destination
>
>
>
> --
> GMX startet ShortView.de. Hier findest Du Leute mit Deinen Interessen!
> Jetzt dabei sein: http://www.shortview.de/?mc=sv_ext_mf@gmx
>
> _______________________________________________
> Xen-users mailing list
> Xen-users@xxxxxxxxxxxxxxxxxxx
> http://lists.xensource.com/xen-users
>



_______________________________________________
Xen-users mailing list
Xen-users@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-users