Re: [Xen-users] Complex Network Interface

[list user <xktnniuymlla@xxxxxxxxxxxxxx>]

Jürgen Ladstätter wrote:
> Hi all, 
>
> my xenserver has two domUs. One scans for Viruses and Spam (Spamassasin, 
> ClamAV) and the other one handles our mail relay which is used very 
> frequently.
> The mailserver will get a “real” ip and the scanserver should get a 
> local ip.
> To scan mails, the mailserver would need a virtual interface with a 
> local ip too. But how can I achieve this? I know how I would do that 
> normally, but xen uses some freakin kind of bridging and that stuff :-/
> Maybe you can help me with that lil problem too: I only can use dhcp to 
> get an ip, otherwise I cant resolve any dns name, I cant ping any 
> outstanding ip and so on.
Hello, Jürgen,

I don't understand whether you *must* use dhcp or whether you *can't* 
get static IPs working, so I'll give a static IP setup.  If that works 
it should not be too difficult to change to dhcp.
There are many different configurations possible.  Here's one.  It has
not been tested for accuracy but I use a similar approach in one of my
xen setups.

Assumptions:
    Fedora Core 4
    xen-3.0.2-2
    two ethernet cards*
    bridged networking
    static IPs

** to further isolate the antivirus subnet you could use a real nic
for the public connection and a virtual nic for the private network:
the virtual nic would have no physical connector.  Only way in would
be via the mail exchanger domain.

######## Domain0

/etc/sysconfig/network-scripts/ifcfg-eth0 contains
DEVICE=eth0
BOOTPROTO=static
TYPE=ethernet
ONBOOT=yes

/etc/sysconfig/network-scripts/ifcfg-eth1 contains
DEVICE=eth1
BOOTPROTO=static
TYPE=ethernet
ONBOOT=yes

Note that they have no IPs.  This helps isolate Domain0.  This will
require you to connect from Domain0 to your domUs with xm console.

Since you will have two subnets you'll need two bridges.  One is for
public acces to the mail exchanger.  The other is for the private
network between the mail exchanger and antivirus box.

Default xen config creates only one so we need a custom script.

Edit /etc/xen/xend-config.sxp.

Find  (network-script network-bridge)
Change to (network-script my-network-bridge)
Comment out any other line that begins with "(network-script".

Create file /etc/xen/scripts/my-network-bridge containing:

#!/bin/bash
PATH=/etc/xen/scripts:$PATH
network-bridge start bridge=MX netdev=eth0 vifnum=0
network-bridge start bridge=AV netdev=eth1 vifnum=1

Make it executable.

Start xend and execute "brctl show".  You should see this:

MX       8000.feffffffffff       no        peth0
                                           vif0.0
AV       8000.feffffffffff       no        peth1
                                           vif0.1

If so, you now have your two bridges.

######## domU setup

The domUs are assigned to whichever bridges they will need access
in their own config files

In the mail exchanger's domU config file add:

vif = [ 'mac=00:16:3e:11:11:01, bridge=MX' ,
        'mac=00:16:3e:22:22:01, bridge=AV' ]

In the antivirus domU config file add:

vif = [ 'mac=00:16:3e:22:22:02, bridge=AV' ]

# *** pick whatever MACs you desire
# *** 00:16:3e:xx:xx:xx are registered to xen and are good choices

######## Mailserver

Remember to set up your firewall to keep outsiders at bay.

Since you will not be allowing outsiders to connect to the
antivirus domain directly, you should not need ip forwarding.

/etc/sysconfig/network-scripts/ifcfg-eth0 contains
DEVICE=eth0
BOOTPROTO=static
TYPE=ethernet
ONBOOT=yes
# provide your PUBLIC netinfo here
NETWORK=a.b.c.0
IPADDR=a.b.c.d
BROADCAST=a.b.c.255
NETMASK=255.255.255.0

/etc/sysconfig/network-scripts/ifcfg-eth1 contains
DEVICE=eth1
BOOTPROTO=static
TYPE=ethernet
ONBOOT=yes
# provide your PRIVATE netinfo here
NETWORK=e.f.g.0
IPADDR=e.f.g.h
BROADCAST=e.f.g.255
NETMASK=255.255.255.0

######## Antivirus

/etc/sysconfig/network-scripts/ifcfg-eth0 contains
DEVICE=eth0
BOOTPROTO=static
TYPE=ethernet
ONBOOT=yes
# provide your PRIVATE netinfo here
NETWORK=e.f.g.0
IPADDR=e.f.g.i
BROADCAST=e.f.g.255
NETMASK=255.255.255.0

######## Possible problems

Domain0 must be allowed to bridge the traffic on the network adapters
so you may need to run iptables with --physdev-in/out rules.  (google is 
your friend ;)
The domUs may have network checksumming problems so you should run the
following line in the domUs after you bring them up, maybe from rc.local.

  domU-mx
    ethtool -K eth0 tx off
    ethtool -K eth1 tx off

  domU-av
    ethtool -K eth0 tx off

For a mail server you probably should have a decent time standard at
your disposal.  If you choose to use ntpd make sure to give the domU
an independent wallclock.

  echo 1 > /proc/sys/xen/independent_wallclock

Caveat: this is off the top of my head and probably contains errors. 
This setup is peculiar to fedora/redhat/centos but should be more or 
less adaptable to other distro's.
Good luck,
Mike Wright

_______________________________________________
Xen-users mailing list
Xen-users@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-users