[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [Xen-users] How to setup Xen for 3 bridges environment?

To: "Daniel Goertzen" <goertzen@xxxxxxxx>
From: "Hong @ gmail" <whtsang22@xxxxxxxxx>
Date: Sun, 29 Jan 2006 00:44:28 +0800
Cc: Xen-users <xen-users@xxxxxxxxxxxxxxxxxxx>
Delivery-date: Sun, 29 Jan 2006 14:44:45 +0000
Domainkey-signature: a=rsa-sha1; q=dns; c=nofws; s=beta; d=gmail.com; h=received:message-id:from:to:cc:references:subject:date:mime-version:content-type:content-transfer-encoding:x-priority:x-msmail-priority:x-mailer:x-mimeole; b=kAcfHd2J1LhwdekRS7mwwtYO5/NwchK2G9S4XJ7MLQfkfpAJkJdnhiSDpXYK7Nwt86b9+WnXJY5xvQN5LSjtC8x5XQoPSiVBvn1wGZ4IWEiAKs14hodXxrWmUMJ9uoAUn4atTX3Uz+jtXNDOWIL9hBNdiuSSmXEDUsUjSbwvMYU=
List-id: Xen user discussion <xen-users.lists.xensource.com>

Hi Daniel,

Really thanks for your hints and it is really the checksum errors.

When I ssh from PC ( 192.168.21.100) in Wan to domU in xbrdmz0, it haschecksum errors. It causes me not able to use ssh.

-------------------------------------------------------------
[root@jssrv01 conf]# tcpdump -v -n host 192.168.21.15
tcpdump: WARNING: eth0: no IPv4 address assigned

tcpdump: listening on eth0, link-type EN10MB (Ethernet), capture size 96bytes00:13:01.797379 IP (tos 0x0, ttl 128, id 25066, offset 0, flags [DF], proto6, length: 48) 192.168.21.100.1521 > 192.168.21.15.ssh: S [tcp s

um ok] 4291786245:4291786245(0) win 65535 <mss 1460,nop,nop,sackOK>

00:13:01.798189 IP (tos 0x0, ttl 63, id 0, offset 0, flags [DF], proto 6,length: 48) 192.168.21.15.ssh > 192.168.21.100.1521: S [tcp sum ok] 2137667989:2137667989(0) ack 4291786246 win 5840 <mss1460,nop,nop,sackOK>00:13:01.798364 IP (tos 0x0, ttl 128, id 25067, offset 0, flags [DF], proto6, length: 40) 192.168.21.100.1521 > 192.168.21.15.ssh: . [tcp s

um ok] ack 1 win 65535

00:13:01.823132 IP (tos 0x0, ttl 63, id 48214, offset 0, flags [DF], proto6, length: 60) 192.168.21.15.ssh > 192.168.21.100.1521: P [bad t

cp cksum 3a1 (->5a1)!] 1:21(20) ack 1 win 5840

00:13:04.816419 IP (tos 0x0, ttl 63, id 48216, offset 0, flags [DF], proto6, length: 60) 192.168.21.15.ssh > 192.168.21.100.1521: P [bad t

cp cksum 3a1 (->5a1)!] 1:21(20) ack 1 win 5840

00:13:08.706572 IP (tos 0x0, ttl 63, id 27133, offset 0, flags [DF], proto6, length: 60) 192.168.21.15.ssh > 192.168.21.100.1520: P [bad t

cp cksum b307 (->b507)!] 2129139235:2129139255(20) ack 731365070 win 5840

00:13:10.816417 IP (tos 0x0, ttl 63, id 48218, offset 0, flags [DF], proto6, length: 60) 192.168.21.15.ssh > 192.168.21.100.1521: P [bad t

cp cksum 3a1 (->5a1)!] 1:21(20) ack 1 win 5840

00:13:22.816419 IP (tos 0x0, ttl 63, id 48220, offset 0, flags [DF], proto6, length: 60) 192.168.21.15.ssh > 192.168.21.100.1521: P [bad tcp cksum 3a1(->5a1)!] 1:21(20) ack 1 win 584000:13:32.706490 IP (tos 0x0, ttl 63, id 27135, offset 0, flags [DF], proto6, length: 60) 192.168.21.15.ssh > 192.168.21.100.1520: P [bad tcp cksumb307 (->b507)!] 0:20(20) ack 1 win 584000:13:46.816434 IP (tos 0x0, ttl 63, id 48222, offset 0, flags [DF], proto6, length: 60) 192.168.21.15.ssh > 192.168.21.100.1521: P [bad tcp cksum 3a1(->5a1)!] 1:21(20) ack 1 win 5840

However, as said before, if I don't create the xbrwan0 and use eth0 instead.I can ssh from PC in Wan to domU.

The tcpdump shows as follows: (PC and domU talks perfectly to each other).
------------------------------------------
[root@jssrv01 conf]# tcpdump -v -n host 192.168.21.15

tcpdump: listening on eth0, link-type EN10MB (Ethernet), capture size 96bytes00:24:39.154125 IP (tos 0x0, ttl 128, id 29657, offset 0, flags [DF], proto6, length: 48) 192.168.21.100.support > 192.168.21.15.ssh: S [tcp sum ok]3072641878:3072641878(0) win 65535 <mss 1460,nop,nop,sackOK>00:24:39.154804 IP (tos 0x0, ttl 63, id 0, offset 0, flags [DF], proto 6,length: 48) 192.168.21.15.ssh > 192.168.21.100.support: S [tcp sum ok]2802605415:2802605415(0) ack 3072641879 win 5840 <mss 1460,nop,nop,sackOK>00:24:39.155017 IP (tos 0x0, ttl 128, id 29658, offset 0, flags [DF], proto6, length: 40) 192.168.21.100.support > 192.168.21.15.ssh: . [tcp sum ok]ack 1 win 6553500:24:39.195572 IP (tos 0x0, ttl 63, id 57410, offset 0, flags [DF], proto6, length: 60) 192.168.21.15.ssh > 192.168.21.100.support: P [bad tcp cksumabf2 (->ad7e)!] 1:21(20) ack 1 win 584000:24:39.200501 IP (tos 0x0, ttl 128, id 29661, offset 0, flags [DF], proto6, length: 68) 192.168.21.100.support > 192.168.21.15.ssh: P [tcp sum ok]1:29(28) ack 21 win 6551500:24:39.201187 IP (tos 0x0, ttl 63, id 57412, offset 0, flags [DF], proto6, length: 40) 192.168.21.15.ssh > 192.168.21.100.support: . [tcp sum ok]ack 29 win 584000:24:39.205163 IP (tos 0x0, ttl 63, id 57414, offset 0, flags [DF], proto6, length: 680) 192.168.21.15.ssh > 192.168.21.100.support: P 21:661(640)ack 29 win 584000:24:39.210997 IP (tos 0x0, ttl 128, id 29665, offset 0, flags [DF], proto6, length: 528) 192.168.21.100.support > 192.168.21.15.ssh: P 29:517(488)ack 661 win 6487500:24:39.211064 IP (tos 0x0, ttl 128, id 29666, offset 0, flags [DF], proto6, length: 56) 192.168.21.100.support > 192.168.21.15.ssh: P [tcp sum ok]517:533(16) ack 661 win 6487500:24:39.249606 IP (tos 0x0, ttl 63, id 57416, offset 0, flags [DF], proto6, length: 40) 192.168.21.15.ssh > 192.168.21.100.support: . [tcp sum ok]ack 533 win 584000:24:39.258471 IP (tos 0x0, ttl 63, id 57418, offset 0, flags [DF], proto6, length: 320) 192.168.21.15.ssh > 192.168.21.100.support: P 661:941(280)ack 533 win 584000:24:39.372868 IP (tos 0x0, ttl 128, id 29670, offset 0, flags [DF], proto6, length: 40) 192.168.21.100.support > 192.168.21.15.ssh: . [tcp sum ok]ack 941 win 6459500:24:39.551000 IP (tos 0x0, ttl 128, id 29672, offset 0, flags [DF], proto6, length: 312) 192.168.21.100.support > 192.168.21.15.ssh: P 533:805(272)ack 941 win 64595

Though the problem is located, I still don't know how to fix it...( I willsearch the list for any findings..)

But....Just wonder Daniel has any further hints for me.... ;->

As I said before, I am new to Xen. This configuration is just an outcome oftrial and error.I remember I initially use the default Xen configuration and has someproblems (I cannot remember is it exactly the same as this one or not).Then, I change the configuration bit by bit and finally down this situation.I am happy to stick with Xen default. As Fernando gives me some reference, Iwill also check if it suit my target configuration.


Hong...

----- Original Message -----From: "Daniel Goertzen" <goertzen@xxxxxxxx>

To: "Hong @ gmail" <whtsang22@xxxxxxxxx>
Cc: "Xen-users" <xen-users@xxxxxxxxxxxxxxxxxxx>
Sent: Saturday, January 28, 2006 10:59 PM
Subject: Re: [Xen-users] How to setup Xen for 3 bridges environment?

Hong, I don't grasp the full complexity of what you're trying to do, but Inote that you are bypassing the vif0.x/veth0.x interfaces in dom0 whichbypasses some special xen networking magic. Instead of assigning ipaddresses directly to xbrwan0, add vif0.0 to xbrwan0 and assign theaddresses to veth0. Likewise for the other bridges.


Search the list tcp checksum errors for more info on this.

Cheers,
Dan.

Hong @ gmail wrote:

Hi Fernando Maior,

Thanks for your information. I have a quick look on the Wiki document,but find that it is a bit different from my current setup.In my setup, I create 3 bridges in dom0 and dom0 also act as thefirewall.

Following is the summary of my configuration:

1) I by-pass the Xen bridge.

- comment out the '(network-script network-bridge)' and '(vif-scriptvif-bridge)' in 'xend-config.sxp'

  - comment out the 'handle_iptable' in vif-bridge.

2) Create my own 3 bridges as follows:
  #
  # xbrwan0 (Wan bridge)
  #
  ip link set eth0 down
  ip addr flush eth0
  brctl addbr xbrwan0
  brctl stp xbrwan0 off
  brctl setfd xbrwan0 0

ip addr add 192.168.21.11/24 dev xbrwan0 brd + (the xbrwan0 bridgeip)ip addr add 192.168.21.15/24 dev xbrwan0 brd + (NAT ip to192.168.22.15)ip addr add 192.168.21.16/24 dev xbrwan0 brd + (NAT ip to192.168.22.16)

  sleep 2
  brctl addif xbrwan0 eth0
  ifconfig eth0 0.0.0.0 up
  ip link set xbrwan0 up
  #
  # xbrdmz0 (Dmz bridge)
  #
  brctl addbr xbrdmz0
  brctl stp xbrdmz0 off
  brctl setfd xbrdmz0 0

ifconfig xbrdmz0 192.168.22.11 netmask 255.255.255.0 up (the Dmz bridgeip)

  #
  # xbrlan0 (Lan bridge)
  #
  brctl addbr xbrlan0
  brctl stp xbrlan0 off
  brctl setfd xbrlan0 0

ifconfig xbrlan0 192.168.23.11 netmask 255.255.255.0 up (the Lan bridgeip)


3) Start two domU with ip 15 and 16
  xm create domU01  (ip 192.168.21.15)
  xm create domU02  (ip 192.168.21.16)

4) Create the NAT tables.
  #
  iptables -X
  iptables -F
  iptables -X -t nat
  iptables -F -t nat

iptables -t nat -A PREROUTING -i xbrwan0 -d 192.168.21.15 -j DNAT --to192.168.22.15iptables -t nat -A POSTROUTING -o xbrwan0 -s 192.168.22.15 -j SNAT --to192.168.21.15iptables -t nat -A PREROUTING -i xbrwan0 -d 192.168.21.16 -j DNAT --to192.168.22.16iptables -t nat -A POSTROUTING -o xbrwan0 -s 192.168.22.16 -j SNAT --to192.168.21.16

  sysctl -w net.ipv4.ip_forward=1

In above setup, I ssh from Pc in Wan to domU01 and domU02.

If I didn't create the Wan bridge, instead using the eth0, I can ssh fromPC in Wan to domU01 and domU02.


=========================================================================
Following is the ifconfig and brctl show result.
[root@jssrv01 conf]# ifconfig
eth0      Link encap:Ethernet  HWaddr 00:0D:0B:A7:2F:88
         UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
         RX packets:4710 errors:0 dropped:0 overruns:0 frame:0
         TX packets:4760 errors:0 dropped:0 overruns:0 carrier:0
         collisions:0 txqueuelen:1000
         RX bytes:717399 (700.5 KiB)  TX bytes:469146 (458.1 KiB)
         Interrupt:5 Base address:0xe800

lo        Link encap:Local Loopback
         inet addr:127.0.0.1  Mask:255.0.0.0
         UP LOOPBACK RUNNING  MTU:16436  Metric:1
         RX packets:0 errors:0 dropped:0 overruns:0 frame:0
         TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
         collisions:0 txqueuelen:0
         RX bytes:0 (0.0 b)  TX bytes:0 (0.0 b)

vif1.0    Link encap:Ethernet  HWaddr FE:FF:FF:FF:FF:FF
         UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
         RX packets:33 errors:0 dropped:0 overruns:0 frame:0
         TX packets:451 errors:0 dropped:22 overruns:0 carrier:0
         collisions:0 txqueuelen:0
         RX bytes:2418 (2.3 KiB)  TX bytes:27310 (26.6 KiB)

vif2.0    Link encap:Ethernet  HWaddr FE:FF:FF:FF:FF:FF
         UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
         RX packets:53 errors:0 dropped:0 overruns:0 frame:0
         TX packets:434 errors:0 dropped:27 overruns:0 carrier:0
         collisions:0 txqueuelen:0
         RX bytes:3740 (3.6 KiB)  TX bytes:25916 (25.3 KiB)

xbrdmz0   Link encap:Ethernet  HWaddr FE:FF:FF:FF:FF:FF

inet addr:192.168.22.11 Bcast:192.168.22.255Mask:255.255.255.0

         UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
         RX packets:86 errors:0 dropped:0 overruns:0 frame:0
         TX packets:32 errors:0 dropped:0 overruns:0 carrier:0
         collisions:0 txqueuelen:0
         RX bytes:4954 (4.8 KiB)  TX bytes:2208 (2.1 KiB)

xbrlan0   Link encap:Ethernet  HWaddr 00:00:00:00:00:00

inet addr:192.168.23.11 Bcast:192.168.23.255Mask:255.255.255.0

         UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
         RX packets:0 errors:0 dropped:0 overruns:0 frame:0
         TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
         collisions:0 txqueuelen:0
         RX bytes:0 (0.0 b)  TX bytes:0 (0.0 b)

xbrwan0   Link encap:Ethernet  HWaddr 00:0D:0B:A7:2F:88

inet addr:192.168.21.11 Bcast:192.168.21.255Mask:255.255.255.0

         UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
         RX packets:4709 errors:0 dropped:0 overruns:0 frame:0
         TX packets:4049 errors:0 dropped:0 overruns:0 carrier:0
         collisions:0 txqueuelen:0
         RX bytes:646686 (631.5 KiB)  TX bytes:416226 (406.4 KiB)

and brctl show result:
[root@jssrv01 conf]# brctl show
bridge name     bridge id               STP enabled     interfaces
xbrdmz0         8000.feffffffffff       no                     vif1.0

vif2.0
xbrlan0         8000.feffffffffff       no                     vif3.0
xbrwan0         8000.000d0ba72f88       no             eth0

Thanks

----- Original Message ----- From: "Fernando Maior"<fernando.souto.maior@xxxxxxxxx>To: "Hong @ gmail" <whtsang22@xxxxxxxxx>; "Xen-users"<xen-users@xxxxxxxxxxxxxxxxxxx>

Sent: Friday, January 27, 2006 7:28 PM
Subject: Re: [Xen-users] How to setup Xen for 3 bridges environment?


Hong,

Did you have a look at Xen wiki documentation? You may
compare http://wiki.xensource.com/xenwiki/XenNetworkingSuse
to your experience and try something like that.

On 1/26/06, Hong @ gmail <whtsang22@xxxxxxxxx> wrote:

Hello,

I am new to Xen and is trying a confugration that simulates a firewall
environment.
My target environment is to setup three bridges: one for Wan (xen-br0),one
for DMZ (xbrdmz) and one for Lan (xbrlan)
The three subnet are as follows:
Wan: 192.168.21.0/24
Dmz: 192.168.22.0/24
Lan:   192.168.23.0/24

I use xen3.0 and FC4 for both dom0 and domU.
My machine currently has one Nic, eth0, and I ensalve it into the Wanbridge
xen-br0.

Dom0 has the IP address 192.168.21.11. I have two domU in DMZ with IP
192.168.22.15, 192.168.22.16).
I use NAT 192.168.21.15 -> 192.168.22.15 and 192.168.21.16 ->192.168.22.16
so that the PC from Wan can access the PC.
Most of the things work fine. I can ping dom0 and the two domU and vice
versa. I can ssh from dom0 and domU and vice versa and I can ssh from PCon
Wan to dom0.
The only problem is that I cannot ssh from PC on Wan to domU.
I have tried another setup. If I don't use the Wan bridge ( xen-br0 )andjust use the eth0 and the Dmz bridge (xbrdmz), everything worksperfectly.
(I can ssh from PC on Wan to domU also).
However, I still want to have the Wan bridge cause I can add some domUin
Wan subnet (so that I can say, add some IDS domU to Wan bridge).
I have searched the mailing list and find a similar case is:
http://lists.xensource.com/archives/html/xen-users/2005-06/msg00669.html

I have tried the NOTRACK option but still can help in my case.

Just wonder anyone has setup similar environment?
Thanks alot.


_______________________________________________
Xen-users mailing list
Xen-users@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-users



--
Bye,
Fernando Maior
LPIC/1(31908)
LinuxCounter(391325)

_______________________________________________
Xen-users mailing list
Xen-users@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-users



_______________________________________________
Xen-users mailing list
Xen-users@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-users

Follow-Ups:
- Re: [Xen-users] How to setup Xen for 3 bridges environment?
  - From: Daniel Goertzen

References:
- [Xen-users] How to setup Xen for 3 bridges environment?
  - From: Hong @ gmail
- Re: [Xen-users] How to setup Xen for 3 bridges environment?
  - From: Fernando Maior
- Re: [Xen-users] How to setup Xen for 3 bridges environment?
  - From: Hong @ gmail
- Re: [Xen-users] How to setup Xen for 3 bridges environment?
  - From: Daniel Goertzen

Prev by Date: Re: [Xen-users] Debian packages for xen3 + kernel images for dom0/U
Next by Date: Re: [Xen-users] Xen and iSCSI
Previous by thread: Re: [Xen-users] How to setup Xen for 3 bridges environment?
Next by thread: Re: [Xen-users] How to setup Xen for 3 bridges environment?
Index(es):
- Date
- Thread

Lists.xenproject.org is hosted with RackSpace, monitoring our
servers 24x7x365 and backed by RackSpace's Fanatical Support®.