[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [Xen-users] Solved: Re: Communication problem with virtual DMZ



Diego Alvarez wrote:
Once again I was mistaken, the real problema was an incorrect TCP
Checksum (discovered with tcpdump), so the solution was to use
"ethtool -K eth0 tx off" on agustina.
I learned this from the list archives.

Folks,

In recent days, quite a few fixes have gone in to xen-unstable
and backported to 3.0-testing to fix the offload problems.

In particular, if you were operating in a VLAN or IPSec
environment, this is believed fixed.

It would really help us if those of you who were having
problems and needed to use ethtool to turn offload off,
could try latest bits without doing that.

If you are still experiencing problems and turning offload
off still is a workaround, please let us know!

thanks,
Nivedita


On 4/27/06, Diego Alvarez <arcane.lord@xxxxxxxxx> wrote:

Hi all,
I am running Xen 3.0.2-2 (taken from XenSource) with Linux kernel
2.6.16 (taken from Debian Sid), I compiled Xen and 2 kernels
(dom0 and domU).

Here is the ascii-art of my setup:

 ------------                        -------------
 |    LAN   |------------------------|   waste   | 192.168.0.94/24
 ------------                        -------------
      |
·····························
·      |                    ·
·      |        Dom0        ·
·      |                    ·
·  ---------                ·    ································
·  | peth0 |                ·    ·                              ·
·  ---------                ·    ·         DomU hades           ·
·      |                    ·    ·         (Firewall)           ·
·      |                    ·    ·                              ·
· -----------   ----------  ·    ·    --------                  ·
· | br-inet |---| hades0 |============| eth0 |  192.168.0.34/24 ·
· -----------   |  (vif) |  ·    ·    --------                  ·
·      |        ----------  ·    ·                              ·
·      |                    ·    ·    --------                  ·
· -----------               ·    ·    | eth1 |  192.168.0.34/32 ·
· | vif0.0  |               ·    ·    --------                  ·
· -----------               ·    ·       ||                     ·
·     ||                    ·    ········||······················
·     ||                    ·            ||
·     ||                    ·············||···········
·     ||                                 ||          ·
·  --------                              ||          ·
·  | eth0 | 192.168.0.22/24          ----------      ·
·  --------                          | hades1 |      ·
·                                    |  (vif) |      ·
·                                    ----------      ·
·                                        |           ·
·                -----------         ----------      ·
·                | pdummy0 |---------| br-dmz |      ·
·                -----------         ----------      ·
·                                        |           ·
·                                   -------------    ·
·                                   | agustina0 |    ·
·                                   |   (vif)   |    ·
·                                   -------------    ·
·                                        ||          ·
·                                        ||          ·
·········································||···········
                                        ||
                     ·-·-·-·-·-·-·-·-·-·||·-·-·-·-·-·-·-·-·
                     ·                  ||                ·
                     |        ··········||········        |
                     ·        ·         ||       ·        ·
                     |        ·     --------     ·        |
                     ·        ·     | eth0 |     ·        ·
                     |        ·     --------     ·        |
                     ·        · 192.168.0.39/32  ·        ·
                     |        ·                  ·        |
                     ·        ·   DomU agustina  ·        ·
                     |        ·    (DMZ Server)  ·        |
                     ·        ·                  ·        ·
                     |        ····················        |
                     ·                                    ·
                     |             Virtual DMZ            |
                     ·                                    ·
                     ·-·-·-·-·-·-·-·-·-·-·-·-·-·-·-·-·-·-·-

--------

Network configuration for Dom0:

auto eth0
iface eth0 inet static
       address 192.168.0.22
       netmask 255.255.255.0
       gateway 192.168.0.2

auto dummy0
iface dummy0 inet static
       address 10.1.1.1
       netmask 255.255.255.255
       up ifconfig dummy0 0.0.0.0 up

--------

Network configuration for DomU agustina (DMZ Server):

auto eth0
iface eth0 inet static
       address 192.168.0.39
       netmask 255.255.255.255
       up route add -host 192.168.0.34 dev eth0
       up route add default gw 192.168.0.34 dev eth0

-------

Network configuration for DomU hades (Firewall):

auto eth0
iface eth0 inet static
 address 192.168.0.34
 netmask 255.255.255.0
 gateway 192.168.0.2
 up arp -Ds 192.168.0.39 eth0 pub

auto eth1
iface eth1 inet static
 address 192.168.0.34
 netmask 255.255.255.255
 up route add -host 192.168.0.39 dev eth1

It also have ip_forward activated by sysctl

------

In dom0, I do the following things:

In /etc/xen/xend-config.sxp I have:

(network-script 'network-bridge bridge=br-inet')
(vif-script 'vif-bridge bridge=br-inet')


I also have a script which brings up br-dmz bridge on dummy0

# brctl show:
bridge name     bridge id               STP enabled     interfaces

br-dmz          8000.feffffffffff       no              agustina0
                                                       hades1
                                                       pdummy0

br-inet         8000.feffffffffff       no              hades0
                                                       peth0
                                                       vif0.0

Here is the configuration for hades and agustina:

/etc/xen/auto/hades:
name="hades"
memory=128
kernel="/boot/vmlinuz-2.6.16-xenU"
vif = [ 'mac=00:16:3e:00:01:01,bridge=br-inet,vifname=hades0',
'mac=00:16:3e:00:00:02,bridge=br-dmz,vifname=hades1' ]
disk=['phy:/dev/xen/hades-OS,hda1,w','phy:/dev/xen/hades-SWAP,hda2,w']
root="/dev/hda1 ro"
on_crash="restart"

/etc/xen/auto/agustina:
name="agustina"
memory=64
kernel="/boot/vmlinuz-2.6.16-xenU"
vif = [ 'mac=00:16:3e:00:00:07,bridge=br-dmz,vifname=agustina0' ]
disk=['phy:/dev/xen/Agustina-OS,hda1,w','phy:/dev/xen/Agustina-SWAP,hda2,w']
root="/dev/hda1 ro"
on_crash="restart"

-------

So.... what is the problem?
well:
- routing is Ok
- ping works in all directions
- ssh from waste (lan machine) to Dom0 works
- ssh from Dom0 to waste works
- ssh from waste to hades works
- ssh from hades to waste works
- ssh from Dom0 to agustina works
- ssh from hades to agustina works
- ssh from agustina to Dom0 works
- ssh from agustina to hades works

but:
- ssh from waste to agustina does not work
- ssh from agustina to waste does not work

Here are is a tcpdump taken from agustina's eth0:

agustina:~# tcpdump -i eth0 -n host waste
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on eth0, link-type EN10MB (Ethernet), capture size 96 bytes
19:45:15.242301 IP waste.4331 > agustina.22: S 30038281:30038281(0)
win 5840 <mss 1460,sackOK,timestamp 23876432 0,nop,wscale 2>
19:45:15.251956 IP agustina.22 > waste.4331: S
3550608405:3550608405(0) ack 30038282 win 5792 <mss
1460,sackOK,timestamp 867394 23876432,nop, wscale 1>
19:45:15.245850 IP waste.4331 > agustina.22: . ack 1 win 1460
<nop,nop,timestamp 23876783 867394>
19:45:15.255867 IP agustina.22 > waste.4331: P 1:42(41) ack 1 win 2896
<nop,nop,timestamp 867394 23876783>
19:45:15.468349 IP agustina.22 > waste.4331: P 1:42(41) ack 1 win 2896
<nop,nop,timestamp 867417 23876783>
19:45:15.888650 IP agustina.22 > waste.4331: P 1:42(41) ack 1 win 2896
<nop,nop,timestamp 867459 23876783>
19:45:16.728328 IP agustina.22 > waste.4331: P 1:42(41) ack 1 win 2896
<nop,nop,timestamp 867543 23876783>
19:45:18.408341 IP agustina.22 > waste.4331: P 1:42(41) ack 1 win 2896
<nop,nop,timestamp 867711 23876783>
19:45:21.768338 IP agustina.22 > waste.4331: P 1:42(41) ack 1 win 2896
<nop,nop,timestamp 868047 23876783>
19:45:28.491449 IP agustina.22 > waste.4331: P 1:42(41) ack 1 win 2896
<nop,nop,timestamp 868719 23876783>

And that goes and goes until timeout.

Those packets from 'agustina' _are_ received by 'waste' in the same way
(I verified that with tcpdump too), and then are dropped by 'waste'
(netfilter conntrack say they are INVALID), so TCP socket is established,
but there is no communication.

The problem I see there is the tcp window size of agustina's reply,
which is bigger than waste first ACK packet, or I am wrong?

The strange thing is that agustina does not have any strange
configuration, and if I connect it to 'br-inet' bridge and change his
netmask and gateway, it work as expected.

There is no firewall on Dom[0U].

Does any of you have any idea of what could be the problem?

Regards,
Diego.

PS1: sorry for the large mail.

PS2: I have also tried packages from
http://packages.debianbase.de/sid/i386/xen3,
    with xen-3.0.1 and kernel 2.6.12, and have the same results.



_______________________________________________
Xen-users mailing list
Xen-users@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-users


_______________________________________________
Xen-users mailing list
Xen-users@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-users


 


Rackspace

Lists.xenproject.org is hosted with RackSpace, monitoring our
servers 24x7x365 and backed by RackSpace's Fanatical Support®.