[Xen-users] Advanced Networking

["Steffen Heil" <lists@xxxxxxxxxxxxxxx>]

Hi

I have a fairly complex networking issue with xen, which I need to get
runnning.
My biggest problem obviously does not arise from xen directly, but more from
linux networking, but I assume someone here might have had the same
problem(s).

So, here we go:

Given hardware:
I have 1 server with 4 public IPs which need to have the same MAC.
Say, $mac for $ip1, $ip2, $ip3, $ip4. All IPs have subnet /24, with $ip3 and
$ip4 residing in the same subnet, and $ip1 and $ip2 in different subnets
each.

Required installations:
I want to run dom0 with as few as possible services for domain admin only.
I need to run two domUs for ISP services.
I want to keep one IP for testing purposes.
I need to run one domU for backuppc.
I propably want to run one domU for monitoring of the other domUs and some
external systems.

My first step was to get dom0 running. Done at $ip1.
Then I created the backuppc domU (was most important). Done at $ip2.

I use network/vif-bridge for that purpose and I got the problem, that my
provider only accepts ONE MAC.
I solved this using ebtables:

server02:~# ebtables -t nat -L
Bridge table: nat

Bridge chain: PREROUTING, entries: 6, policy: ACCEPT
-p IPv4 -d $mac -i peth0 --ip-dst $2 -j dnat --to-dst 0:16:3e:0:16:1
--dnat-target ACCEPT
-p ARP -d $mac -i peth0 --arp-ip-dst $2 -j dnat --to-dst 0:16:3e:0:16:1
--dnat-target ACCEPT
-p IPv4 -d $mac -i peth0 --ip-dst $3 -j dnat --to-dst 0:16:3e:0:16:2
--dnat-target ACCEPT
-p ARP -d $mac -i peth0 --arp-ip-dst $3 -j dnat --to-dst 0:16:3e:0:16:2
--dnat-target ACCEPT
-p IPv4 -d $mac -i peth0 --ip-dst $4 -j dnat --to-dst 0:16:3e:0:16:3
--dnat-target ACCEPT
-p ARP -d $mac -i peth0 --arp-ip-dst $4 -j dnat --to-dst 0:16:3e:0:16:3
--dnat-target ACCEPT

Bridge chain: OUTPUT, entries: 0, policy: ACCEPT

Bridge chain: POSTROUTING, entries: 6, policy: ACCEPT
-p IPv4 -s 0:16:3e:0:16:1 -o peth0 --ip-src $2 -j snat --to-src $mac
--snat-target ACCEPT
-p ARP -s 0:16:3e:0:16:1 -o peth0 --arp-ip-src $2 -j snat --to-src $mac
--snat-target ACCEPT
-p IPv4 -s 0:16:3e:0:16:2 -o peth0 --ip-src $3 -j snat --to-src $mac
--snat-target ACCEPT
-p ARP -s 0:16:3e:0:16:2 -o peth0 --arp-ip-src $3 -j snat --to-src $mac
--snat-target ACCEPT
-p IPv4 -s 0:16:3e:0:16:3 -o peth0 --ip-src $4 -j snat --to-src $mac
--snat-target ACCEPT
-p ARP -s 0:16:3e:0:16:3 -o peth0 --arp-ip-src $4 -j snat --to-src $mac
--snat-target ACCEPT

This is basically SNAT at MAC-level for $ip2, $ip3 and $ip4.
$ip1 is the ip of the dom0, which inherits it's mac from the real hardware,
which already has $mac.

That works. I can access backuppc, it can access other systems.
I see that there is still a problem with access between dom0 and backuppc,
since because they are in different subnets, every information is sent over
the providers router, which makes few sense for two systems runnung on the
same hardware. But I will come back on this later. I will propably add
direct routes.

Everything until here is (except for MAC-NAT) xen-default. xen creates the
xenbr0 bridge, moves eth0 there (renamed to peth0) and connects a virtual
eth0 in dom0.

Now I finally get to my problem.

At dom0 I set up another bridge xenintbr and give it the ip 192.168.1.1/24.
Then I set up another domU called test and give it 192.168.1.2.

This works. I can ping it and access services from dom0.

BUT I cannot get ip forwarding working.

I tried:
- iptables -t nat -A POSTROUTING -o eth0 -s 192.168.1.0/24 -j SNAT
--to-source $ip1
- iptables -t nat -A POSTROUTING -o eth0 -j MASQUERADE
- iptables -t nat -A POSTROUTING -o peth0 -s 192.168.1.0/24 -j SNAT
--to-source $ip1
- iptables -t nat -A POSTROUTING -o peth0 -j MASQUERADE

None of these work.
If the test-domU sends DNS-requests, they ARE translated to source=$ip1 and
send out. The DNS-Server replies to $ip1, which I can see tcpdumping peth0.
And those packets dest-mac is really $mac, which means they should be
delivered to dom0, which should do NAT and forward them to test-domU.
However, that does not work. The DNS-response never reaches dom0, they do
not even apper on tcpdumping eth0, neighter correctly nor incorrectly. They
simple don't.

So, over to you.
Anyone any guess?

(Please, as my mailing list subscription seems not to be completed yet, send
answers as direct copy to me. Thanks.)

Regards,
  Steffen

Attachment: smime.p7s
Description: S/MIME cryptographic signature

_______________________________________________
Xen-users mailing list
Xen-users@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-users