[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [Xen-users] firewall xenU woes (help would be appreciated)



Hi Adam,
I've added some of my configs and thoughts below:

Adam Tworkowski wrote:
> Hi Marcus,
> 
> I have the dummy module loaded on dom0 and several dummy interfaces
> which show up with ifconfig.  They are configured as follows:
> 
> auto dummy0
> iface dummy0 inet static
>       address 192.168.254.1
>       netmask 255.255.255.248
>       network 192.168.254.0
>       broadcast 192.168.254.7
>       gateway 192.168.254.6
>       post-up brctl addbr xen-br0 || true
>       post-up brctl addif xen-br0 dummy0 || true
>       post-up brctl delif xen-br0 dummy0 || true
> 
> Additional bridges configured in same manner.
> 
> Without intervention dom0 boots with 4 dummy interfaces
> (dummy0,dummy1,etc.) and xen-br0.  This is apparent from ifconfig.
> The additional bridges, however, do not load w/o intervention.  I have
> to use the brctl command as per the above /etc/network/interfaces.
> 
> In /etc/xen/scripts/network I have the bridges mapped to the dummy
> devices as so:
> 
> bridge=${bridge:-xen-br0}
> netdev=${netdev:-dummy0}
> bridge=${bridge:-xen-br1}
> netdev=${netdev:-dummy1}
> ...
> 

As xen brings up xen-br0 automatically, I haven't used post hooks for
dummy0:
/etc/network/interfaces includes:
        auto dummy0
        iface dummy0 inet static
                address         192.168.254.1
                netmask         255.255.255.248
                network         192.168.254.0
                broadcast       192.168.254.7
                gateway         192.168.254.6
                dns-nameservers 192.168.254.6

        auto dummy1
        iface dummy1 inet static
                address         192.168.254.9
                netmask         255.255.255.248
                network         192.168.254.8
                broadcast       192.168.254.15
                post-up         brctl addbr br1 || true
                post-up         brctl addif br1 dummy1 || true
                post-up         ifconfig br1 192.168.254.9/29
                post-down       brctl delif br1 dummy1

        auto dummy2
        iface dummy2 inet static
                address         192.168.254.17
                netmask         255.255.255.248
                network         192.168.254.16
                broadcast       192.168.254.23
                post-up         brctl addbr br2 || true
                post-up         brctl addif br2 dummy2 || true
                post-up         ifconfig br2 192.168.254.17/29
                post-down       brctl delif br2 dummy2

        ... and so on

/etc/xen/scripts/network includes:
        bridge=${bridge:-xen-br0}
        netdev=${netdev:-dummy0}
        antispoof=${antispoof:-yes}

I'm not sure if multiple bridge & netdev statements work here,
but nice if it does.

BTW:
The post hook configuring the ip of the bridge is wrong, as this
method only works if the bridge already has an ip.
It should read:
        post-up ifconfig br2 192.168.254.17 netmask 255.255.255.248
Also, I don't think it's necessary to assign ip to the dummy, but
I'll look into that later.

> It is worth noting that the routing table shows something like the
> following (having only brought up xen-br0 and xen-br1):
> 
> 192.168.254.16 0.0.0.0       255.255.255.248 dummy2
> 192.168.254.24 0.0.0.0       255.255.255.248 dummy3
> 192.168.254.0  0.0.0.0       255.255.255.248 xen-br0
> 192.168.254.8  0.0.0.0       255.255.255.248 xen-br1
> 0.0.0.0        192.168.254.6 0.0.0.0         xen-br0
> 
> Does this look sound?
> 
yep,
# route -n
Kernel IP routing table
Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
192.168.254.64  0.0.0.0         255.255.255.248 U     0      0        0 dummy6
192.168.254.72  0.0.0.0         255.255.255.248 U     0      0        0 dummy7
192.168.254.16  0.0.0.0         255.255.255.248 U     0      0        0 dummy2
192.168.254.24  0.0.0.0         255.255.255.248 U     0      0        0 dummy3
192.168.254.0   0.0.0.0         255.255.255.248 U     0      0        0 xen-br0
192.168.254.8   0.0.0.0         255.255.255.248 U     0      0        0 dummy1
192.168.254.96  0.0.0.0         255.255.255.240 U     0      0        0 dummy9
192.168.254.80  0.0.0.0         255.255.255.240 U     0      0        0 dummy8
192.168.254.48  0.0.0.0         255.255.255.240 U     0      0        0 dummy5
192.168.254.32  0.0.0.0         255.255.255.240 U     0      0        0 dummy4
192.168.253.0   0.0.0.0         255.255.255.0   U     0      0        0 dummy10
192.168.254.0   0.0.0.0         255.255.255.0   U     0      0        0 br1
192.168.254.0   0.0.0.0         255.255.255.0   U     0      0        0 br2
192.168.254.0   0.0.0.0         255.255.255.0   U     0      0        0 br3
192.168.254.0   0.0.0.0         255.255.255.0   U     0      0        0 br4
192.168.254.0   0.0.0.0         255.255.255.0   U     0      0        0 br5
0.0.0.0         192.168.254.6   0.0.0.0         UG    0      0        0 xen-br0

# ifconfig | egrep "Link|addr"
br1       Link encap:Ethernet  HWaddr 62:3F:39:21:9D:DB
          inet addr:192.168.254.9  Bcast:192.168.254.255  Mask:255.255.255.0
br2       Link encap:Ethernet  HWaddr 82:D1:F6:2A:55:B6
          inet addr:192.168.254.17  Bcast:192.168.254.255  Mask:255.255.255.0
br3       Link encap:Ethernet  HWaddr AE:DB:D9:29:16:B8
          inet addr:192.168.254.25  Bcast:192.168.254.255  Mask:255.255.255.0
br4       Link encap:Ethernet  HWaddr 62:50:E4:EF:72:A1
          inet addr:192.168.254.33  Bcast:192.168.254.255  Mask:255.255.255.0
br5       Link encap:Ethernet  HWaddr 9E:4A:BC:41:68:A8
          inet addr:192.168.254.49  Bcast:192.168.254.255  Mask:255.255.255.0
dummy0    Link encap:Ethernet  HWaddr FA:99:8D:02:7D:FE
          inet addr:192.168.254.1  Bcast:192.168.254.7  Mask:255.255.255.248
dummy1    Link encap:Ethernet  HWaddr 62:3F:39:21:9D:DB
          inet addr:192.168.254.9  Bcast:192.168.254.15  Mask:255.255.255.248
dummy2    Link encap:Ethernet  HWaddr 82:D1:F6:2A:55:B6
          inet addr:192.168.254.17  Bcast:192.168.254.23  Mask:255.255.255.248
dummy3    Link encap:Ethernet  HWaddr AE:DB:D9:29:16:B8
          inet addr:192.168.254.25  Bcast:192.168.254.31  Mask:255.255.255.248
dummy4    Link encap:Ethernet  HWaddr 62:50:E4:EF:72:A1
          inet addr:192.168.254.33  Bcast:192.168.254.47  Mask:255.255.255.240
dummy5    Link encap:Ethernet  HWaddr 9E:4A:BC:41:68:A8
          inet addr:192.168.254.49  Bcast:192.168.254.63  Mask:255.255.255.240
dummy6    Link encap:Ethernet  HWaddr 06:EB:9E:DF:25:2F
          inet addr:192.168.254.65  Bcast:192.168.254.71  Mask:255.255.255.248
dummy7    Link encap:Ethernet  HWaddr 6E:92:58:A4:1D:C2
          inet addr:192.168.254.73  Bcast:192.168.254.79  Mask:255.255.255.248
dummy8    Link encap:Ethernet  HWaddr 32:4E:B6:5D:1B:12
          inet addr:192.168.254.81  Bcast:192.168.254.95  Mask:255.255.255.240
dummy9    Link encap:Ethernet  HWaddr 06:1E:0C:F4:5C:09
          inet addr:192.168.254.97  Bcast:192.168.254.111  Mask:255.255.255.240
dummy10   Link encap:Ethernet  HWaddr E6:5B:DC:A9:F2:54
          inet addr:192.168.253.1  Bcast:192.168.253.255  Mask:255.255.255.0
lo        Link encap:Local Loopback
          inet addr:127.0.0.1  Mask:255.0.0.0
vif1.0    Link encap:Ethernet  HWaddr FE:FF:FF:FF:FF:FF
vif1.1    Link encap:Ethernet  HWaddr FE:FF:FF:FF:FF:FF
vif1.2    Link encap:Ethernet  HWaddr FE:FF:FF:FF:FF:FF
vif1.3    Link encap:Ethernet  HWaddr FE:FF:FF:FF:FF:FF
vif1.4    Link encap:Ethernet  HWaddr FE:FF:FF:FF:FF:FF
vif1.5    Link encap:Ethernet  HWaddr FE:FF:FF:FF:FF:FF
vif1.6    Link encap:Ethernet  HWaddr FE:FF:FF:FF:FF:FF
vif1.7    Link encap:Ethernet  HWaddr FE:FF:FF:FF:FF:FF
vif1.8    Link encap:Ethernet  HWaddr FE:FF:FF:FF:FF:FF
vif1.9    Link encap:Ethernet  HWaddr FE:FF:FF:FF:FF:FF
vif1.10   Link encap:Ethernet  HWaddr FE:FF:FF:FF:FF:FF
vif2.0    Link encap:Ethernet  HWaddr FE:FF:FF:FF:FF:FF
vif4.0    Link encap:Ethernet  HWaddr FE:FF:FF:FF:FF:FF
vif5.0    Link encap:Ethernet  HWaddr FE:FF:FF:FF:FF:FF
xen-br0   Link encap:Ethernet  HWaddr FA:99:8D:02:7D:FE
          inet addr:192.168.254.1  Bcast:192.168.254.7  Mask:255.255.255.255

# brctl show
bridge name     bridge id               STP enabled     interfaces
br10            8000.e65bdca9f254       no              dummy10
                                                        vif1.0
br1             8000.623f39219ddb       no              dummy1
                                                        vif1.2
br2             8000.82d1f62a55b6       no              dummy2
                                                        vif1.3
br3             8000.aedbd92916b8       no              dummy3
                                                        vif1.4
br4             8000.6250e4ef72a1       no              dummy4
                                                        vif1.5
br5             8000.9e4abc4168a8       no              dummy5
                                                        vif1.6
                                                        vif2.0
                                                        vif4.0
br6             8000.06eb9edf252f       no              dummy6
                                                        vif1.7
br7             8000.6e9258a41dc2       no              dummy7
                                                        vif1.8
br8             8000.324eb65d1b12       no              dummy8
                                                        vif1.9
br9             8000.061e0cf45c09       no              dummy9
                                                        vif1.10
xen-br0         8000.fa998d027dfe       no              dummy0
                                                        vif1.1
                                                        vif5.0



> Also note that /proc/net/dev only shows network activity for dummy0 and
> xen-br0 -- none for xen-br1 even though I have pinged the associated
> address and attempt to have domU1 use xen-br1 (vif = 'backend=fw,
> brige=xen-br1' ].
> 
fw doesn't need to export the interface AFAIK,  as domU1 can have the
bridge exported to it directly from dom0.
eg.

/etc/xen/Firewall includes:
        name = "Firewall"
        pci = ['00,0b,0', '00,0a,0' ]
        nics=11
        vif = [ 'mac=aa:00:00:00:22:01, bridge=br10',
                'mac=aa:00:00:25:40:01, bridge=xen-br0',
                'mac=aa:00:00:25:40:09, bridge=br1',
                'mac=aa:00:00:25:40:17, bridge=br2',
                'mac=aa:00:00:25:40:25, bridge=br3',
                'mac=aa:00:00:25:40:33, bridge=br4',
                'mac=aa:00:00:25:40:49, bridge=br5',
                'mac=aa:00:00:25:40:45, bridge=br6',
                'mac=aa:00:00:25:40:73, bridge=br7',
                'mac=aa:00:00:25:40:81, bridge=br8',
                'mac=aa:00:00:25:40:97, bridge=br9' ]

In the Firewall, the PCI NICs become eth0/1, and the bridges are eth2...

An example domU:
/etc/xen/Mail includes:
        name = "MailServer"
        vif = [ 'mac=AA:00:00:25:40:51, bridge=br5' ]

You can tell I've gone overboard with bridges. I'll be reducing this
number with the latest design, but they're great to have lying around
for testing :) . That many interfaces causes shorewall to think for
30-60 secs when recreating iptables rules !!!

Marcus.

> What else do you need from me to help troubleshoot?  
> 
> Adam  
> 
> On Mon, 2005-08-29 at 02:24 -0400, Marcus Brown wrote:
> 
>>Hi Adam,
>>
>>Perhaps you could try loading the dummy modules in dom0 instead?
>>Then attach your dom0 bridges to the dummy interfaces by replacing
>>the netdev argument for "eth0" in /etc/xen/scripts/network with "dummy",
>>and create auto scripts for the dummy interfaces that create bridges.
>>
>>Sorry to be so brief ... just caught me before going to bed.
>>    (Trying to work on a quote :) )
>>If you get stuck perhaps you could provide more detail?
>>
>>Regards,
>>
>>Marcus.
>>
>>
>>Adam Tworkowski wrote:
>>
>>
>>>Hi,
>>>
>>>Could someone please help me out with networking my firewall on xenU
>>>configuration.  I have combed the list archives for posts on similar
>>>configurations and getting bridging working properly but I am stuck and
>>>generally confused.  I am trying to achieve the scenario below (ascii
>>>borrowed from previous thread).
>>>
>>>              +------------------------+
>>>              | physical machine, dom0 |
>>>              |   +---------------+    |
>>>-- Internet -------+ Firewall domU +--------- Intranet
>>>              |   +------+--------+    |
>>>              |          |             |
>>>              |          |   DMZ       |
>>>              |    +-----+------+      |
>>>              |    |            |      |
>>>              | +--+--+      +--+--+   |
>>>              | |domU1|      |domU2|   |
>>>              | +-----+      +-----+   |
>>>              +------------------------+
>>>
>>>So far:
>>>
>>>- physical ethernet device (will be two later) hidden from dom0 (no issue) 
>>>- domU (fw) using ethernet device -- has access to internet on eth0 (no 
>>>issue)
>>>
>>>- domU1 and domU2 can only see domU when using bridge=xen-br0 although it 
>>>appears that only on of the two can be active as if both are pinging domU 
>>>there is much
>>>packet.  Using vif = ['mac=xx.xx..., bridge=xen-br0'"].
>>>
>>>Should domU1 and domU2 should be using xen-br0 at all or should this only be 
>>>for domU and the former be using xen-br1 and xen-br2 respectively.
>>>
>>>- I have attempted to have domU1 use vif = ['backend=fw'] without success
>>>
>>>- I have also attempted to created xen-br1,xen-br2 and have the non-fw domUs 
>>>use these: vif = ['bridge=xen-br1']
>>>
>>>- Having searched the list, I have attempted to create dummy0, dummy1,etc. 
>>>on the domU as suggested.  
>>>
>>>
>>>I am hoping that someone can lay out what they have done to get this 
>>>working.  My suspicion is that my bridging is not working correctly on dom0.
>>>
>>>Question:  what devices should be showing from 'ifconfig' on respective 
>>>nodes (dom0, domU, domU1...)?  
>>>
>>>xen0 shows multiple vifs, xen-br0-4.
>>>xenU shows eth0 - should it show vifs?
>>>xenU1,2 shows eth0.  Should it show more? vifs?  
>>>
>>>Maybe someone can share the output of their ifconfigs to compare.
>>>
>>>All my instance are running on Debian Sarge.
>>>
>>>Thanks in advance!
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>_______________________________________________
>>>Xen-users mailing list
>>>Xen-users@xxxxxxxxxxxxxxxxxxx
>>>http://lists.xensource.com/xen-users
>>> 
>>>
>>
>>
>>
> 
> 
> 
> _______________________________________________
> Xen-users mailing list
> Xen-users@xxxxxxxxxxxxxxxxxxx
> http://lists.xensource.com/xen-users


_______________________________________________
Xen-users mailing list
Xen-users@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-users


 


Rackspace

Lists.xenproject.org is hosted with RackSpace, monitoring our
servers 24x7x365 and backed by RackSpace's Fanatical Support®.