Re: [Xen-users] Force traffic out one interface

[Felix Kuperjans <felix@xxxxxxxxxxxxxxxxxx>]

Hi Jonathan,

I think it would be better to do that in two steps:

Firstly, get rid of your bridge and get a running configuration with
routing.
Secondly, you can try to add those policy routing rules, to enforce what
you really want. But policy routing is a highly complex and advanced
technique.

I'll explain the first step to you know, but I need to setup the second
step myself, I can't do that spontaneously. By the way, you'll need
policy routing built into your dom0 kernel or at least as a module (it's
under advanced routing in the configuration) and iproute2 (should be
pulled in as one of Xen's dependencies).

To change to routing, you only need to use another network and
vif-script, but I usually write an own one (you'll need that for those
policy routing rules anyway).
Important is, that IP forwarding (and if you like, IPv6 forwarding) is
enabled.

Do that in your network script:

# Enbale IP forwarding:
echo 1 >/proc/sys/net/ipv4/ip_forward

# Enable forwarding for all devices:
for dev in $(ls /proc/sys/net/ipv4/conf/) ; do
    echo 1 >"/proc/sys/net/ipv4/conf/${dev}/forwarding"
    echo 1 >"/proc/sys/net/ipv6/conf/${dev}/forwarding"
done

# Disable forwarding for lo:
echo 0 >/proc/sys/net/ipv4/conf/lo/forwarding
echo 0 >/proc/sys/net/ipv6/conf/lo/forwarding

Note, that there are multiple ways to handle routing + ARP:
- The default Xen way: use proxy_arp to distribute your full ARP table
to your clients. Pros: Behaviour is more similar to bridging or LANs.
Contra: proxy_arp is considered unsafe, may reveal internal network
topology and increases the vulnerability to ARP-based attacks. In
addition, it is not IPv6 compatible (there is no neighbour_proxy).
- My personal way: Use static ARP and force all traffic (even 'local'
traffic from one domU to another), to be routed via the dom0 as a
gateway. Pros: Everything must be routed, and is subject to domain-0's
control. Contra: A little bit harder to setup the domUs if you've got
local area IP addresses and you'll need your own vif-scripts.
- There are surely other ways, but I'll not cover that in detail here.

Your vif-script should look something like that:

# Include common script:
. "$(dirname "${0}")/vif-common.sh"

# Include vif configuration:
. "/etc/xen/network/${vif}.sh"

# Switch the command:
case "${command}" in
"online")
    # Setup Tun/Tap device:
    ip link set "${vif}" up
    echo 1 >"/proc/sys/net/ipv4/conf/${vif}/forwarding"
    echo 1 >"/proc/sys/net/ipv6/conf/${vif}/forwarding"
    ip addr add "${gateway_ip}/32" dev "${vif}"

    # Setup the routes:
    for addr in ${ip} ; do
        ip route add "${addr}/32" dev "${vif}" src "${main_ip}"
        ip neigh replace "${addr}" lladdr "${hw_addr}" nud permanent dev
"${vif}"
    done
    ;;
"offline")
    # Delete the routes:
    for addr in ${ip} ; do
        ip route del "${addr}/32" dev "${vif}"
        ip neigh del "${addr}" dev "${vif}"
    done
    ;;
esac

# Log success:
log debug "Successful vif-myroute ${command} for ${vif}."
if [ "${command}" = "online" ] ; then
    success
fi

This will require some things (which will make things easier later):
- An own configuration per vif in /etc/xen/network, which stores these
variables:
    gateway_ip: The address of the gateway of the ip addresses assigned
to the domU.
    hw_addr: The mac address you assign to the vif (you must set static
ones).
- To find this file, use unique names for the vifs, e.g. vif-c1 (c for
customer) or the names of the domU (can be set by the vifname=...
parameter).
- You need to specify the ip address of the domU in the
vif-configuration or the configuration file above.

I hope you are familiar with routing, things will get more complex with
policy routing and firewalling.

An example setup for the domU:

mac address: 00:16:3E:01:02:03
ip address (take a real address here): 123.45.67.89
gateway address (probably specified by the provider, or just take lower
address in an as small as possible subnet, that you do *not* really
assign to any domUs): 123.45.67.81 (would be gateway of /20 network)

There are better way to specify dummy gateway ip addresses, but this
makes domU configuration harder (configuration of domU's OS).

Try out this setup and tell me about your problems / successes. The DomU
OS would need to configure its ip address statically with the gateway
you specified.

Regards,
Felix

Am 13.06.2010 18:20, schrieb Jonathan Tripathy:
> Hi Felix,
>
> Excellent plan! We are getting closer! I should really put all these
> wonderful tips from everyone here on a blog or something..
>
> Back to the plan..
>
> The only difficulty I see here is that the DomUs will be using public
> IP address, and the firewall (between the Internet and Dom0) will be a
> "filtering bridge" in its own right. However maybe that doesn't matter.
>
> Would you maybe be able to give me some example of the actual rules
> that I could use? This would be very much appreciated, and if I saw
> the rules I could work out if my firewall setup is a problem.
>
> It would be nice if my ISP just gave my firewall's WAN interface a
> single address, and allowed multiple public subnets to be routed via
> my firewall (so my firewall would act like a router, not a bridge),
> however I don't think this is the case. I think all that I will get is
> just an ethernet cable connected to a switch..
>
> Thanks
>
> On 13/06/10 17:10, Felix Kuperjans wrote:
>> Hi Jonathan,
>>
>> I read your mail and those you posted in different previous threads and
>> I think that you should probably consider *not* using a bridge and using
>> pure routing instead:
>> - Do you really need bridge-only features (especially broadcasts from
>> domU to domU or broadcasts trespassing the dom0)? If I understand your
>> plans correctly, you want all your domUs to be isolated with their own
>> IP address and only communicating via a dedicated firewall. This way,
>> you would not need broadcasts between clients (this is only interesting
>> if you want to use LAN services between your domUs, because broadcasts
>> are not sent across the internet anyway).
>> - AFAIK, routing is more secure and faster than bridging, but somewhat
>> harder to setup.
>> - You could do what you posted below with routing. It might work with
>> bridging, too, but I don't know a good way to do that with a bridge.
>>
>> With routing, you would need policy routing because of this elementary
>> problem:
>>
>> You have (to make things easier to explain, in this example only 2) two
>> DomUs (let's say, 1.0.0.1 on vif-1.0 and 1.0.0.2 on vif-2.0), the
>> Domain-0 and a dedicated firewall between the Dom0 and the internet.
>> If 1.0.0.1 wants to reach any server on the internet (or vice versa), it
>> will trespass the firewall by default.
>> But if 1.0.0.1 wants to send (e.g. an e-mail) to 1.0.0.2 or (more
>> dangerous) wants to attack 1.0.0.2, they would only communicate via the
>> Domain-0 (without the firewall).
>>
>> The problem is:
>>
>> If you route 1.0.0.2 to vif-2.0 under all circumstances, it will bypass
>> the firewall if 1.0.0.1 sent the package.
>> If you route everything except 1.0.0.1 via eth0, you wont be able to
>> reach 1.0.0.2 any way.
>>
>> The solution is:
>>
>> You need to do policy routing.
>> If a package originates from the internet an should be sent to 1.0.0.2,
>> it must be routed to vif-2.0. But if it originates from 1.0.0.1, it must
>> be routed to eth0, so that it is sent to the firewall.
>> The firewall will then process the package and return it to the server,
>> which now must route the package to vif-2.0.
>>
>> So it will take two policy routes:
>> route 1.0.0.2 via vif-2.0, if it is from eth0
>> route 1.0.0.2 via eth0, if it is not from eth0
>>
>> I don't think that those routes would work with a bridge, so consider
>> using routing.
>>
>> Felix
>>
>> Am 13.06.2010 17:45, schrieb Jonathan Tripathy:
>>   
>>> Hi Everyone,
>>>
>>> Does anyone know any rules that I could use (using iptable, ebtables,
>>> or otherwise) that could force all traffic coming from a guest to go
>>> out via a particular interface? I wish to stop "inter-guest"
>>> communication, without going via my firewall first.
>>>
>>> Thanks
>>>
>>> _______________________________________________
>>> Xen-users mailing list
>>> Xen-users@xxxxxxxxxxxxxxxxxxx
>>> http://lists.xensource.com/xen-users
>>>
>>>      
>> _______________________________________________
>> Xen-users mailing list
>> Xen-users@xxxxxxxxxxxxxxxxxxx
>> http://lists.xensource.com/xen-users
>>    
>
> _______________________________________________
> Xen-users mailing list
> Xen-users@xxxxxxxxxxxxxxxxxxx
> http://lists.xensource.com/xen-users
>

_______________________________________________
Xen-users mailing list
Xen-users@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-users