[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

[Xen-users] XEN Virtualization & Isolation of Guest using ebtables



Hello,

The basic details of my host and XEN configuration and network configuration
all appear below. 

Whilst I have come this far, a lot of it has been intuitive so I now have
plenty of questions. Firstly I would like some help and clarification with
regard to ebtables and especially how it works in a Virtualized environment
on Novell SUSE 10 SP1

(1) ebtables Installation

ebtables was not installed with my original SUSE distribution and is not
listed by YAST as as installable package. So I downloaded the rpm from the
ebtables homepage and did something like this as root:

rpm -i -vv <ebtables rpm> 

This produced a lot of info but I did not notice any significant errors.
However, there is clearly a mismatch between my SUSE file structure and the
rpm, because executing the package created an "init.d" directory within the
existing "/etc/init.d" directory and placed ebtables script inside that. So
I manually corrected this by moving ebtables to the higher init.d directory 
(/etc/init.d) and rebooted.

When I tried to use ebtables 

ebtables -L

        I received a message something  like this:

  error while loading shared libraries: cannot open shared object file
<filename.so>: No such file or directory 


I did some research and the only readily available fix that I could find was
to append this line to /etc/ld.so.conf:

/usr/lib/ebtables

Then I ran "ldconfig" and re-booted, and now ebtables works fine.

This brings me to my first question:    Q1: Is this is an acceptable way to
have installed ebtables, or should I have made it work differently, and if
so, what are the specific techniques on this SUSE platform?

It has not escaped my attention that iptables does work on this platform
without requiring its own "pointer" in /etc/ld.so.conf

Furthermore, if we compare the file structure currently supporting iptables
vs ebtables:

The iptables shared objects would appear to reside in their own subdirectory
underneath /usr/lib    i.e.  /usr/lib/iptables,  
and this holds true for the shared objects for ebtables which are in
/usr/lib/ebtables

There is a big difference however in that the iptables executable seems to
be located in 

/usr/sbin 

whereas the ebtables executable is located in:

/usr/local/sbin

This does mean that the relative path from the executable to the .so files
is different for iptables than it is for ebtables (although I do not know if
that matters.)

Q2: On my platform, how does iptables know where to find its shared object
files without any entry in ld.so.conf and should ebtables work using the
same mechanism?

I did try moving the ebtables executable from /usr/local/sbin but the result
was to break it, i.e. when I ran, for example, "ebtables -L"   I receive an
error message stating that the executable could not be found (which I
suppose is reasonable...lol)

(2)  ebtables script

The init.d script that shipped with the ebtables rpm package  is broken as
soon as it is installed on my SUSE platform, so I had to make some
modifications to it.

I commented out these 2 lines: 

# source /etc/init.d
# source /etc/sysconfig/network

because they seemed to be causing problems. I think on my SUSE platform the
relevant interpreter does not understand the "source" directive.  

Q3: Is there an alternative to the "source" directive on my platform?  What
is going on here and what is "source" supposed to do: Is it similar to a
c-style "include" directive?

I made a couple of other minor amendments to the script (including the
elimination of "success" and "failure" functions which do not appear to
exist on SUSE) and now it runs really well :)   But I could still do with
some pointers on writing these kind of scripts: can someone point me to some
good web resources?


(3)  ebtables for XEN

You can see from the information dump below that my XEN network
configuration is a little complex (that is another story and another post,
but anyway...) 

What I am attempting to do with this network setup is to ensure that no
packet can travel to or from the Windows 2003 guest domain "interweb" (ID 3)
to Domain-0  unless it does so via the physical ethernet interface.

I have ensured that interweb(3) is on its own bridge along with the physical
interface, eth2:

xenbr2          8000.dab561626236       no              vif0.2
                                                        peth2
                                                        tap2
                                                        vif3.0

Q4:  If ebtables and iptables were both left open and un-configured, would
the above configuration by itself be sufficient to isolate interweb(3) in
the way I want?
I am assuming not, because: 
 (a) packets might be routed from xenbr1 to xenbr2, or be routed from xenbr0
to xenbr2?
 (b) packets might be forwarded by xenbr2 from  vif0.2 (Domain0) to vif3.0
without ever passing over a physical interface (?)
Q5:  Are (a) and (b) reasonable assumptions?

By using ebtables I am attempting to eliminate problem (b) as described
above:

Bridge table: filter

Bridge chain: INPUT, entries: 1, policy: ACCEPT
-d 0:16:3e:70:fd:a6 -i vif0.2 -j DROP
Bridge chain: FORWARD, entries: 2, policy: ACCEPT
-d 0:16:3e:70:fd:a6 -i vif0.2 -j DROP
-s 0:16:3e:70:fd:a6 -o vif0.2 -j DROP
Bridge chain: OUTPUT, entries: 0, policy: ACCEPT

i.e. I want this configuration to prevent any packets being forwarded by
xenbr2 from its Domain-0 interface to its interweb(3) interface. 
Note here that '0:16:3e:70:fd:a6' is the mac Address that is actually seen
by the Windows 2003 guest interweb(3)
 Q6: Have I got it right?

Finally, I was slightly confused by  the ebtables documentation referring to
an -i option and a --logical-in option;   Q7: Can someone help by explaining
in this context the difference between a logical interface and an interface,
particularly as it applies to my "virtual" environment?


Most grateful for any help, advice and/or tips on any or all of the above.

With thanks,

Robert


coredbhost1:~ # cat /proc/version
Linux version 2.6.16.46-0.12-xenpae (geeko@buildhost) (gcc version 4.1.2
20070115 (prerelease) (SUSE Linux)) #1 SMP Thu May 17 14:00:09 UTC 2007
coredbhost1:~ # xm list
Name                                      ID   Mem VCPUs      State  
Time(s)
Domain-0                                   0   861     2     r-----   1397.2
coredb1                                    1  4096     2     -b----    965.3
devdb1                                     2  2048     2     -b----    530.4
interweb1                                  3  1024     1     -b----    333.3
coredbhost1:~ # xm network-list 0
coredbhost1:~ # xm network-list 1
Idx BE     MAC Addr.     handle state evt-ch tx-/rx-ring-ref BE-path
0   0  ??                   0     1      -1    -1   /-1     
/local/domain/0/backend/vif/1/0
coredbhost1:~ # xm network-list 2
Idx BE     MAC Addr.     handle state evt-ch tx-/rx-ring-ref BE-path
0   0  ??                   0     1      -1    -1   /-1     
/local/domain/0/backend/vif/2/0
coredbhost1:~ # xm network-list 3
Idx BE     MAC Addr.     handle state evt-ch tx-/rx-ring-ref BE-path
0   0  ??                   0     1      -1    -1   /-1     
/local/domain/0/backend/vif/3/0
coredbhost1:~ # ifconfig
eth0      Link encap:Ethernet  HWaddr 00:1C:C4:6C:C9:0E
          inet addr:192.168.55.177  Bcast:192.168.55.255  Mask:255.255.255.0
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:71467 errors:0 dropped:0 overruns:0 frame:0
          TX packets:15107 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0
          RX bytes:6197655 (5.9 Mb)  TX bytes:64344156 (61.3 Mb)

eth1      Link encap:Ethernet  HWaddr 00:1C:C4:6C:C9:10
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:45816 errors:0 dropped:0 overruns:0 frame:0
          TX packets:2 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0
          RX bytes:4646668 (4.4 Mb)  TX bytes:64 (64.0 b)

eth2      Link encap:Ethernet  HWaddr 00:1F:29:F0:0E:F6
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:7 errors:0 dropped:0 overruns:0 frame:0
          TX packets:2 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0
          RX bytes:341 (341.0 b)  TX bytes:64 (64.0 b)

lo        Link encap:Local Loopback
          inet addr:127.0.0.1  Mask:255.0.0.0
          UP LOOPBACK RUNNING  MTU:16436  Metric:1
          RX packets:47370 errors:0 dropped:0 overruns:0 frame:0
          TX packets:47370 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0
          RX bytes:43573556 (41.5 Mb)  TX bytes:43573556 (41.5 Mb)

peth0     Link encap:Ethernet  HWaddr FE:FF:FF:FF:FF:FF
          UP BROADCAST RUNNING NOARP  MTU:1500  Metric:1
          RX packets:46416 errors:0 dropped:0 overruns:0 frame:0
          TX packets:76830 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:5007330 (4.7 Mb)  TX bytes:68162860 (65.0 Mb)
          Interrupt:17 Memory:f8000000-f8012100

peth1     Link encap:Ethernet  HWaddr FE:FF:FF:FF:FF:FF
          UP BROADCAST RUNNING NOARP  MTU:1500  Metric:1
          RX packets:109051 errors:0 dropped:0 overruns:0 frame:0
          TX packets:128330 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:84573995 (80.6 Mb)  TX bytes:83178082 (79.3 Mb)
          Interrupt:18 Memory:f6000000-f6012100

peth2     Link encap:Ethernet  HWaddr FE:FF:FF:FF:FF:FF
          UP BROADCAST RUNNING NOARP  MTU:1500  Metric:1
          RX packets:422 errors:0 dropped:0 overruns:0 frame:0
          TX packets:414 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:81827 (79.9 Kb)  TX bytes:113415 (110.7 Kb)
          Interrupt:16 Memory:fc000000-fc012100

tap0      Link encap:Ethernet  HWaddr CA:26:5E:9D:E5:88
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:79385 errors:0 dropped:0 overruns:0 frame:0
          TX packets:115175 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:500
          RX bytes:62736755 (59.8 Mb)  TX bytes:67369983 (64.2 Mb)

tap1      Link encap:Ethernet  HWaddr 7A:66:FA:C8:F5:75
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:23892 errors:0 dropped:0 overruns:0 frame:0
          TX packets:64905 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:500
          RX bytes:18439045 (17.5 Mb)  TX bytes:22805533 (21.7 Mb)

tap2      Link encap:Ethernet  HWaddr DA:B5:61:62:62:36
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:408 errors:0 dropped:0 overruns:0 frame:0
          TX packets:427 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:500
          RX bytes:111399 (108.7 Kb)  TX bytes:79635 (77.7 Kb)

vif0.0    Link encap:Ethernet  HWaddr FE:FF:FF:FF:FF:FF
          UP BROADCAST RUNNING NOARP  MTU:1500  Metric:1
          RX packets:15107 errors:0 dropped:0 overruns:0 frame:0
          TX packets:71467 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0
          RX bytes:64344156 (61.3 Mb)  TX bytes:6197655 (5.9 Mb)

vif0.1    Link encap:Ethernet  HWaddr FE:FF:FF:FF:FF:FF
          UP BROADCAST RUNNING NOARP  MTU:1500  Metric:1
          RX packets:2 errors:0 dropped:0 overruns:0 frame:0
          TX packets:45816 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0
          RX bytes:64 (64.0 b)  TX bytes:4646668 (4.4 Mb)

vif0.2    Link encap:Ethernet  HWaddr FE:FF:FF:FF:FF:FF
          UP BROADCAST RUNNING NOARP  MTU:1500  Metric:1
          RX packets:2 errors:0 dropped:0 overruns:0 frame:0
          TX packets:7 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0
          RX bytes:64 (64.0 b)  TX bytes:341 (341.0 b)

vif1.0    Link encap:Ethernet  HWaddr FE:FF:FF:FF:FF:FF
          UP BROADCAST NOARP  MTU:1500  Metric:1
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
          TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:32
          RX bytes:0 (0.0 b)  TX bytes:0 (0.0 b)

vif2.0    Link encap:Ethernet  HWaddr FE:FF:FF:FF:FF:FF
          UP BROADCAST NOARP  MTU:1500  Metric:1
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
          TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:32
          RX bytes:0 (0.0 b)  TX bytes:0 (0.0 b)

vif3.0    Link encap:Ethernet  HWaddr FE:FF:FF:FF:FF:FF
          UP BROADCAST NOARP  MTU:1500  Metric:1
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
          TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:32
          RX bytes:0 (0.0 b)  TX bytes:0 (0.0 b)

xenbr0    Link encap:Ethernet  HWaddr FE:FF:FF:FF:FF:FF
          UP BROADCAST RUNNING NOARP  MTU:1500  Metric:1
          RX packets:20524 errors:0 dropped:0 overruns:0 frame:0
          TX packets:2 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0
          RX bytes:2834679 (2.7 Mb)  TX bytes:64 (64.0 b)

xenbr1    Link encap:Ethernet  HWaddr 7A:66:FA:C8:F5:75
          UP BROADCAST RUNNING NOARP  MTU:1500  Metric:1
          RX packets:20521 errors:0 dropped:0 overruns:0 frame:0
          TX packets:2 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0
          RX bytes:2835269 (2.7 Mb)  TX bytes:64 (64.0 b)

xenbr2    Link encap:Ethernet  HWaddr DA:B5:61:62:62:36
          UP BROADCAST RUNNING NOARP  MTU:1500  Metric:1
          RX packets:14 errors:0 dropped:0 overruns:0 frame:0
          TX packets:2 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0
          RX bytes:588 (588.0 b)  TX bytes:64 (64.0 b)

coredbhost1:~ # brctl show
bridge name     bridge id               STP enabled     interfaces
xenbr0          8000.feffffffffff       yes             vif0.0
                                                        peth0
xenbr1          8000.7a66fac8f575       yes             vif0.1
                                                        peth1
                                                        vif1.0
                                                        tap0
                                                        tap1
                                                        vif2.0
xenbr2          8000.dab561626236       no              vif0.2
                                                        peth2
                                                        tap2
                                                        vif3.0
coredbhost1:~ # brctl showmacs xenbr0
port no mac addr                is local?       ageing timer
  2     00:01:e7:4d:a7:80       no                49.37
  2     00:02:a5:28:5e:6d       no                13.09
  2     00:02:a5:58:04:63       no                72.72
  2     00:02:a5:cd:3f:b6       no                 0.67
  2     00:07:4d:28:11:c8       no                 4.04
  2     00:0b:cd:ee:b9:ff       no                18.24
  2     00:0c:29:b9:ea:86       no                 5.30
  2     00:12:79:da:73:ae       no               120.89
  2     00:14:c2:62:be:65       no                36.50
  2     00:17:a4:fb:50:42       no                65.16
  2     00:1a:4b:2b:84:19       no               260.30
  2     00:1b:78:18:0a:ed       no               285.96
  2     00:1b:78:1c:8d:3f       no                67.75
  2     00:1c:7e:c1:4c:21       no                22.25
  2     00:1c:7e:e8:97:1d       no               161.80
  1     00:1c:c4:6c:c9:0e       no                 0.00
  2     00:1c:c4:d4:e8:11       no               270.64
  2     00:1c:c4:d4:e9:a1       no                95.86
  2     00:1c:c4:d4:ea:93       no                47.20
  2     00:40:ca:69:43:ec       no                90.56
  2     00:40:ca:85:84:7e       no                 9.78
  2     00:40:ca:85:84:84       no                 0.00
  2     00:40:ca:85:84:b1       no               160.35
  2     00:40:ca:85:8b:4d       no                83.95
  2     00:40:ca:85:8b:7b       no                 7.54
  2     00:40:ca:85:8c:1b       no               179.27
  2     00:40:ca:85:8c:3d       no               212.32
  2     00:40:ca:85:8c:47       no                63.81
  2     00:40:ca:85:8c:6b       no               177.12
  2     00:40:ca:85:8c:7d       no               201.07
  2     00:40:ca:85:8c:85       no               256.02
  2     00:80:77:ca:c6:67       no               112.77
  2     00:c0:b7:69:d5:3f       no                15.72
  1     fe:ff:ff:ff:ff:ff       yes                0.00
coredbhost1:~ # brctl showmacs xenbr1
port no mac addr                is local?       ageing timer
  2     00:01:e7:4d:a7:80       no                53.57
  2     00:02:a5:28:5e:6d       no                17.28
  2     00:02:a5:58:04:63       no                76.91
  2     00:02:a5:cd:3f:b6       no                 1.86
  2     00:07:4d:28:11:c8       no                 8.24
  2     00:0b:cd:ee:b9:ff       no                 1.23
  2     00:0c:29:b9:ea:86       no                 9.49
  2     00:12:79:da:73:ae       no               125.08
  2     00:14:c2:62:be:65       no                40.69
  4     00:16:3e:4c:7f:a6       no                 5.59
  2     00:17:a4:fb:50:42       no                69.35
  2     00:1a:4b:2b:84:19       no               264.49
  2     00:1b:78:18:0a:ed       no               290.15
  2     00:1b:78:1c:8d:3f       no                71.94
  2     00:1c:7e:c1:4c:21       no                26.44
  2     00:1c:7e:e8:97:1d       no               165.99
  2     00:1c:c4:6c:c9:0e       no               116.96
  2     00:1c:c4:d4:e8:11       no               274.84
  2     00:1c:c4:d4:e9:a1       no               100.05
  2     00:1c:c4:d4:ea:93       no                51.39
  2     00:40:ca:69:43:ec       no                94.76
  2     00:40:ca:85:84:7e       no                13.98
  2     00:40:ca:85:84:84       no                34.70
  2     00:40:ca:85:84:b1       no                 0.46
  2     00:40:ca:85:8b:4d       no                88.14
  2     00:40:ca:85:8b:7b       no                11.73
  2     00:40:ca:85:8c:1b       no               183.46
  2     00:40:ca:85:8c:3c       no                 2.39
  2     00:40:ca:85:8c:3d       no               216.52
  2     00:40:ca:85:8c:47       no                68.01
  2     00:40:ca:85:8c:6b       no               181.31
  2     00:40:ca:85:8c:7d       no                 5.59
  2     00:40:ca:85:8c:85       no               260.22
  2     00:80:77:ca:c6:67       no               116.96
  2     00:c0:b7:69:d5:3f       no                19.91
  5     7a:66:fa:c8:f5:75       yes                0.00
  4     ca:26:5e:9d:e5:88       yes                0.00
  1     fe:ff:ff:ff:ff:ff       yes                0.00
coredbhost1:~ # brctl showmacs xenbr2
port no mac addr                is local?       ageing timer
  2     00:10:db:83:62:55       no               119.61
  3     00:16:3e:70:fd:a6       no               119.82
  3     da:b5:61:62:62:36       yes                0.00
  1     fe:ff:ff:ff:ff:ff       yes                0.00

coredbhost1:~ # ebtables -L
Bridge table: filter

Bridge chain: INPUT, entries: 1, policy: ACCEPT
-d 0:16:3e:70:fd:a6 -i vif0.2 -j DROP

Bridge chain: FORWARD, entries: 2, policy: ACCEPT
-d 0:16:3e:70:fd:a6 -i vif0.2 -j DROP
-s 0:16:3e:70:fd:a6 -o vif0.2 -j DROP

Bridge chain: OUTPUT, entries: 0, policy: ACCEPT
coredbhost1:~ #

-- 
View this message in context: 
http://www.nabble.com/XEN-Virtualization---Isolation-of-Guest-using-ebtables-tp20660647p20660647.html
Sent from the Xen - User mailing list archive at Nabble.com.


_______________________________________________
Xen-users mailing list
Xen-users@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-users


 


Rackspace

Lists.xenproject.org is hosted with RackSpace, monitoring our
servers 24x7x365 and backed by RackSpace's Fanatical Support®.