[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

[Xen-users] Advanced Networking

  • To: <xen-users@xxxxxxxxxxxxxxxxxxx>
  • From: "Steffen Heil" <lists@xxxxxxxxxxxxxxx>
  • Date: Fri, 10 Mar 2006 14:04:23 +0100
  • Delivery-date: Fri, 10 Mar 2006 13:05:08 +0000
  • List-id: Xen user discussion <xen-users.lists.xensource.com>
  • Thread-index: AcZEQyaVXt33Llj4RBahCeSbbEhiFA==


I have a fairly complex networking issue with xen, which I need to get
My biggest problem obviously does not arise from xen directly, but more from
linux networking, but I assume someone here might have had the same

So, here we go:

Given hardware:
I have 1 server with 4 public IPs which need to have the same MAC.
Say, $mac for $ip1, $ip2, $ip3, $ip4. All IPs have subnet /24, with $ip3 and
$ip4 residing in the same subnet, and $ip1 and $ip2 in different subnets

Required installations:
I want to run dom0 with as few as possible services for domain admin only.
I need to run two domUs for ISP services.
I want to keep one IP for testing purposes.
I need to run one domU for backuppc.
I propably want to run one domU for monitoring of the other domUs and some
external systems.

My first step was to get dom0 running. Done at $ip1.
Then I created the backuppc domU (was most important). Done at $ip2.

I use network/vif-bridge for that purpose and I got the problem, that my
provider only accepts ONE MAC.
I solved this using ebtables:

server02:~# ebtables -t nat -L
Bridge table: nat

Bridge chain: PREROUTING, entries: 6, policy: ACCEPT
-p IPv4 -d $mac -i peth0 --ip-dst $2 -j dnat --to-dst 0:16:3e:0:16:1
--dnat-target ACCEPT
-p ARP -d $mac -i peth0 --arp-ip-dst $2 -j dnat --to-dst 0:16:3e:0:16:1
--dnat-target ACCEPT
-p IPv4 -d $mac -i peth0 --ip-dst $3 -j dnat --to-dst 0:16:3e:0:16:2
--dnat-target ACCEPT
-p ARP -d $mac -i peth0 --arp-ip-dst $3 -j dnat --to-dst 0:16:3e:0:16:2
--dnat-target ACCEPT
-p IPv4 -d $mac -i peth0 --ip-dst $4 -j dnat --to-dst 0:16:3e:0:16:3
--dnat-target ACCEPT
-p ARP -d $mac -i peth0 --arp-ip-dst $4 -j dnat --to-dst 0:16:3e:0:16:3
--dnat-target ACCEPT

Bridge chain: OUTPUT, entries: 0, policy: ACCEPT

Bridge chain: POSTROUTING, entries: 6, policy: ACCEPT
-p IPv4 -s 0:16:3e:0:16:1 -o peth0 --ip-src $2 -j snat --to-src $mac
--snat-target ACCEPT
-p ARP -s 0:16:3e:0:16:1 -o peth0 --arp-ip-src $2 -j snat --to-src $mac
--snat-target ACCEPT
-p IPv4 -s 0:16:3e:0:16:2 -o peth0 --ip-src $3 -j snat --to-src $mac
--snat-target ACCEPT
-p ARP -s 0:16:3e:0:16:2 -o peth0 --arp-ip-src $3 -j snat --to-src $mac
--snat-target ACCEPT
-p IPv4 -s 0:16:3e:0:16:3 -o peth0 --ip-src $4 -j snat --to-src $mac
--snat-target ACCEPT
-p ARP -s 0:16:3e:0:16:3 -o peth0 --arp-ip-src $4 -j snat --to-src $mac
--snat-target ACCEPT

This is basically SNAT at MAC-level for $ip2, $ip3 and $ip4.
$ip1 is the ip of the dom0, which inherits it's mac from the real hardware,
which already has $mac.

That works. I can access backuppc, it can access other systems.
I see that there is still a problem with access between dom0 and backuppc,
since because they are in different subnets, every information is sent over
the providers router, which makes few sense for two systems runnung on the
same hardware. But I will come back on this later. I will propably add
direct routes.

Everything until here is (except for MAC-NAT) xen-default. xen creates the
xenbr0 bridge, moves eth0 there (renamed to peth0) and connects a virtual
eth0 in dom0.

Now I finally get to my problem.

At dom0 I set up another bridge xenintbr and give it the ip
Then I set up another domU called test and give it

This works. I can ping it and access services from dom0.

BUT I cannot get ip forwarding working.

I tried:
- iptables -t nat -A POSTROUTING -o eth0 -s -j SNAT
--to-source $ip1
- iptables -t nat -A POSTROUTING -o eth0 -j MASQUERADE
- iptables -t nat -A POSTROUTING -o peth0 -s -j SNAT
--to-source $ip1
- iptables -t nat -A POSTROUTING -o peth0 -j MASQUERADE

None of these work.
If the test-domU sends DNS-requests, they ARE translated to source=$ip1 and
send out. The DNS-Server replies to $ip1, which I can see tcpdumping peth0.
And those packets dest-mac is really $mac, which means they should be
delivered to dom0, which should do NAT and forward them to test-domU.
However, that does not work. The DNS-response never reaches dom0, they do
not even apper on tcpdumping eth0, neighter correctly nor incorrectly. They
simple don't.

So, over to you.
Anyone any guess?


Attachment: smime.p7s
Description: S/MIME cryptographic signature

Xen-users mailing list



Lists.xenproject.org is hosted with RackSpace, monitoring our
servers 24x7x365 and backed by RackSpace's Fanatical Support®.