[Xen-users] Solved: Re: Communication problem with virtual DMZ

["Diego Alvarez" <arcane.lord@xxxxxxxxx>]

Once again I was mistaken, the real problema was an incorrect TCP
Checksum (discovered with tcpdump), so the solution was to use
"ethtool -K eth0 tx off" on agustina.
I learned this from the list archives.

On 4/27/06, Diego Alvarez <arcane.lord@xxxxxxxxx> wrote:
> Hi all,
> I am running Xen 3.0.2-2 (taken from XenSource) with Linux kernel
> 2.6.16 (taken from Debian Sid), I compiled Xen and 2 kernels
> (dom0 and domU).
>
> Here is the ascii-art of my setup:
>
>   ------------                        -------------
>   |    LAN   |------------------------|   waste   | 192.168.0.94/24
>   ------------                        -------------
>        |
> ·····························
> ·      |                    ·
> ·      |        Dom0        ·
> ·      |                    ·
> ·  ---------                ·    ································
> ·  | peth0 |                ·    ·                              ·
> ·  ---------                ·    ·         DomU hades           ·
> ·      |                    ·    ·         (Firewall)           ·
> ·      |                    ·    ·                              ·
> · -----------   ----------  ·    ·    --------                  ·
> · | br-inet |---| hades0 |============| eth0 |  192.168.0.34/24 ·
> · -----------   |  (vif) |  ·    ·    --------                  ·
> ·      |        ----------  ·    ·                              ·
> ·      |                    ·    ·    --------                  ·
> · -----------               ·    ·    | eth1 |  192.168.0.34/32 ·
> · | vif0.0  |               ·    ·    --------                  ·
> · -----------               ·    ·       ||                     ·
> ·     ||                    ·    ········||······················
> ·     ||                    ·            ||
> ·     ||                    ·············||···········
> ·     ||                                 ||          ·
> ·  --------                              ||          ·
> ·  | eth0 | 192.168.0.22/24          ----------      ·
> ·  --------                          | hades1 |      ·
> ·                                    |  (vif) |      ·
> ·                                    ----------      ·
> ·                                        |           ·
> ·                -----------         ----------      ·
> ·                | pdummy0 |---------| br-dmz |      ·
> ·                -----------         ----------      ·
> ·                                        |           ·
> ·                                   -------------    ·
> ·                                   | agustina0 |    ·
> ·                                   |   (vif)   |    ·
> ·                                   -------------    ·
> ·                                        ||          ·
> ·                                        ||          ·
> ·········································||···········
>                                          ||
>                       ·-·-·-·-·-·-·-·-·-·||·-·-·-·-·-·-·-·-·
>                       ·                  ||                ·
>                       |        ··········||········        |
>                       ·        ·         ||       ·        ·
>                       |        ·     --------     ·        |
>                       ·        ·     | eth0 |     ·        ·
>                       |        ·     --------     ·        |
>                       ·        · 192.168.0.39/32  ·        ·
>                       |        ·                  ·        |
>                       ·        ·   DomU agustina  ·        ·
>                       |        ·    (DMZ Server)  ·        |
>                       ·        ·                  ·        ·
>                       |        ····················        |
>                       ·                                    ·
>                       |             Virtual DMZ            |
>                       ·                                    ·
>                       ·-·-·-·-·-·-·-·-·-·-·-·-·-·-·-·-·-·-·-
>
> --------
>
> Network configuration for Dom0:
>
> auto eth0
> iface eth0 inet static
>         address 192.168.0.22
>         netmask 255.255.255.0
>         gateway 192.168.0.2
>
> auto dummy0
> iface dummy0 inet static
>         address 10.1.1.1
>         netmask 255.255.255.255
>         up ifconfig dummy0 0.0.0.0 up
>
> --------
>
> Network configuration for DomU agustina (DMZ Server):
>
> auto eth0
> iface eth0 inet static
>         address 192.168.0.39
>         netmask 255.255.255.255
>         up route add -host 192.168.0.34 dev eth0
>         up route add default gw 192.168.0.34 dev eth0
>
> -------
>
> Network configuration for DomU hades (Firewall):
>
> auto eth0
> iface eth0 inet static
>   address 192.168.0.34
>   netmask 255.255.255.0
>   gateway 192.168.0.2
>   up arp -Ds 192.168.0.39 eth0 pub
>
> auto eth1
> iface eth1 inet static
>   address 192.168.0.34
>   netmask 255.255.255.255
>   up route add -host 192.168.0.39 dev eth1
>
> It also have ip_forward activated by sysctl
>
> ------
>
> In dom0, I do the following things:
>
> In /etc/xen/xend-config.sxp I have:
>
> (network-script 'network-bridge bridge=br-inet')
> (vif-script 'vif-bridge bridge=br-inet')
>
>
> I also have a script which brings up br-dmz bridge on dummy0
>
> # brctl show:
> bridge name     bridge id               STP enabled     interfaces
>
> br-dmz          8000.feffffffffff       no              agustina0
>                                                         hades1
>                                                         pdummy0
>
> br-inet         8000.feffffffffff       no              hades0
>                                                         peth0
>                                                         vif0.0
>
> Here is the configuration for hades and agustina:
>
> /etc/xen/auto/hades:
> name="hades"
> memory=128
> kernel="/boot/vmlinuz-2.6.16-xenU"
> vif = [ 'mac=00:16:3e:00:01:01,bridge=br-inet,vifname=hades0',
> 'mac=00:16:3e:00:00:02,bridge=br-dmz,vifname=hades1' ]
> disk=['phy:/dev/xen/hades-OS,hda1,w','phy:/dev/xen/hades-SWAP,hda2,w']
> root="/dev/hda1 ro"
> on_crash="restart"
>
> /etc/xen/auto/agustina:
> name="agustina"
> memory=64
> kernel="/boot/vmlinuz-2.6.16-xenU"
> vif = [ 'mac=00:16:3e:00:00:07,bridge=br-dmz,vifname=agustina0' ]
> disk=['phy:/dev/xen/Agustina-OS,hda1,w','phy:/dev/xen/Agustina-SWAP,hda2,w']
> root="/dev/hda1 ro"
> on_crash="restart"
>
> -------
>
> So.... what is the problem?
> well:
>  - routing is Ok
>  - ping works in all directions
>  - ssh from waste (lan machine) to Dom0 works
>  - ssh from Dom0 to waste works
>  - ssh from waste to hades works
>  - ssh from hades to waste works
>  - ssh from Dom0 to agustina works
>  - ssh from hades to agustina works
>  - ssh from agustina to Dom0 works
>  - ssh from agustina to hades works
>
> but:
>  - ssh from waste to agustina does not work
>  - ssh from agustina to waste does not work
>
> Here are is a tcpdump taken from agustina's eth0:
>
> agustina:~# tcpdump -i eth0 -n host waste
> tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
> listening on eth0, link-type EN10MB (Ethernet), capture size 96 bytes
> 19:45:15.242301 IP waste.4331 > agustina.22: S 30038281:30038281(0)
> win 5840 <mss 1460,sackOK,timestamp 23876432 0,nop,wscale 2>
> 19:45:15.251956 IP agustina.22 > waste.4331: S
> 3550608405:3550608405(0) ack 30038282 win 5792 <mss
> 1460,sackOK,timestamp 867394 23876432,nop, wscale 1>
> 19:45:15.245850 IP waste.4331 > agustina.22: . ack 1 win 1460
> <nop,nop,timestamp 23876783 867394>
> 19:45:15.255867 IP agustina.22 > waste.4331: P 1:42(41) ack 1 win 2896
> <nop,nop,timestamp 867394 23876783>
> 19:45:15.468349 IP agustina.22 > waste.4331: P 1:42(41) ack 1 win 2896
> <nop,nop,timestamp 867417 23876783>
> 19:45:15.888650 IP agustina.22 > waste.4331: P 1:42(41) ack 1 win 2896
> <nop,nop,timestamp 867459 23876783>
> 19:45:16.728328 IP agustina.22 > waste.4331: P 1:42(41) ack 1 win 2896
> <nop,nop,timestamp 867543 23876783>
> 19:45:18.408341 IP agustina.22 > waste.4331: P 1:42(41) ack 1 win 2896
> <nop,nop,timestamp 867711 23876783>
> 19:45:21.768338 IP agustina.22 > waste.4331: P 1:42(41) ack 1 win 2896
> <nop,nop,timestamp 868047 23876783>
> 19:45:28.491449 IP agustina.22 > waste.4331: P 1:42(41) ack 1 win 2896
> <nop,nop,timestamp 868719 23876783>
>
> And that goes and goes until timeout.
>
> Those packets from 'agustina' _are_ received by 'waste' in the same way
> (I verified that with tcpdump too), and then are dropped by 'waste'
> (netfilter conntrack say they are INVALID), so TCP socket is established,
> but there is no communication.
>
> The problem I see there is the tcp window size of agustina's reply,
> which is bigger than waste first ACK packet, or I am wrong?
>
> The strange thing is that agustina does not have any strange
> configuration, and if I connect it to 'br-inet' bridge and change his
> netmask and gateway, it work as expected.
>
> There is no firewall on Dom[0U].
>
> Does any of you have any idea of what could be the problem?
>
> Regards,
> Diego.
>
> PS1: sorry for the large mail.
>
> PS2: I have also tried packages from
> http://packages.debianbase.de/sid/i386/xen3,
>      with xen-3.0.1 and kernel 2.6.12, and have the same results.
>

_______________________________________________
Xen-users mailing list
Xen-users@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-users