Re: [Xen-users] Still confused about bridging (I think)

["Andrew Lyon" <andrew.lyon@xxxxxxxxx>]

On Mon, Sep 22, 2008 at 10:47 PM, David Dyer-Bennet <dd-b@xxxxxxxx> wrote:
>
> On Mon, September 22, 2008 11:24, Dustin Henning wrote:
>> David,
>>       You appear to have an awful lot of interfaces I'm not familiar with.
>
> At this point I'm suspecting that CentOS (should be same as RHEL) 5.2 is
> considerably complicating my experience here.  Their documentation isn't
> answering any of these questions for me, though.
>
>> Here's what I can tell you in that regard:
>> -vifX.Y is the default naming convention for domU interfaces where X is
>> the
>> domU and Y is the eth# in the domU
>
> So vif 1.2 is the "other end" of probably eth2 in domain 1?
>
>> -tapX is an interface for an HVM domU, but X isn't related to the domU and
>> I
>> don't know how to tell which tap interface goes where
>
> Luckily I have only one HVM domain at the moment, so there's no ambiguity
> about what it connects to.
>
>> -xenbrX is the bridge on older Xen systems, usually a fake ethX is created
>> on these as well, and I thought it needed to be added to xenbr0 along with
>> the other interfaces, but I believe someone disagreed with me regarding
>> that
>> on a previous thread where another user couldn't get bridging working
>
> I think I created xenbr0, by including it in a "bridge=xenbr0" in the vif
> lien in /etc/xen/vl01.
>
>> -ethX is the bridge on newer Xen systems, and it has an IP assigned
>> directly
>> instead of having a separate fake eth0
>
> eth1 is, I believe, my physical second ethernet interface in this case.
> It connects to a private vlan used within this set of systems (I'm working
> towards a set of servers behind a load-balancer).
>
>> -pethX is the physical Ethernet interface as renamed by the Xen network
>> script
>
> Yep, peth0 was eth0 originally.  It connects to the corporate LAN.
>
>> -I know of no reason why you would need to use multiple physical
>> interfaces
>> and bridges for multiple networks/subnetworks unless they are physically
>> separated or perhaps separated by VLANs, this is assuming the NICs are
>> properly configured on the domUs and nothing is interfering with traffic
>> on
>> the bridges or physical connection
>
> Interesting point there, in that eth1 is connected to a different vlan
> than eth0.  With the switch handling it, I don't think this will be
> visible in the packets and hence won't propagate inside the server; but
> I've never worked with vlans in either form before.
>
>> -vif0.X - I don't believe I have ever seen this before, as I have always
>> seen dom0 use ethX one way or another, and yours appears to be doing that
>> based on your ip addr list output
>> -vethX - same as vif0.X
>
> I have a vague memory that those are created in pairs for me by something
> in the startup scripts. i don't think I'm using them for anything.

http://wiki.xensource.com/xenwiki/XenNetworking explains that 7 pairs
of "connected virtual ethernet interfaces" are created but I really do
not understand why that is necessary? and on my system I only see 4:

 ip link list | grep vif
2: vif0.0: <BROADCAST,MULTICAST> mtu 1500 qdisc noop
4: vif0.1: <BROADCAST,MULTICAST> mtu 1500 qdisc noop
6: vif0.2: <BROADCAST,MULTICAST> mtu 1500 qdisc noop
8: vif0.3: <BROADCAST,MULTICAST> mtu 1500 qdisc noop

I have one HVM running: 1
8: vif6.0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast qlen 32

And there is one bridge called eth0 which has the physical and vif6.0 in it:

 brctl show
bridge name     bridge id               STP enabled     interfaces
eth0            8000.003048c39d98       no              peth0
                                                        tap0
                                                        vif6.0

I need to create a second bridge with eth1, and a third bridge which
needs to have a virtual interface on the dom0.

I plan to connect several windows hvm's that have been migrated from
hardware to the third bridge so that they can talk to each other and
get internet access through dom0, but they must NOT be able to talk to
the lan on physical eth0 as bringing up duplicate domain controllers
and servers would not be good at all,I guess that is what the
"connected virtual ethernet interfaces" are for but I am unsure about
how to create the 2 extra bridges, once the 3rd bridge is there I
guess I need to configure a ip on the dom0 virtual interface that is
connected to the bridge and setup some iptables rules to allow
internet but block access to the local lan.

I need to test a microsoft exchange 2003 > 2007 migration before I do
it on the real servers, the upgrade has gone wrong once already and
with a small window of time to do it I need to be sure it will be
successful.

I see no xenbr0 at all, so what do I need to put in the config files
to use the 2nd or 3rd bridge?

I am running Xen 3.2.1 on Gentoo with kernel 2.6.25.15, any help would
be appreciated.

Andy

>
>> -It might be helpful to post the dom0 output from iptables -nL and route
>
> [root@prcapp02 xen]# iptables -nL
> Chain INPUT (policy ACCEPT)
> target     prot opt source               destination
> ACCEPT     udp  --  0.0.0.0/0            0.0.0.0/0           udp dpt:53
> ACCEPT     tcp  --  0.0.0.0/0            0.0.0.0/0           tcp dpt:53
> ACCEPT     udp  --  0.0.0.0/0            0.0.0.0/0           udp dpt:67
> ACCEPT     tcp  --  0.0.0.0/0            0.0.0.0/0           tcp dpt:67
>
> Chain FORWARD (policy ACCEPT)
> target     prot opt source               destination
> ACCEPT     all  --  0.0.0.0/0            192.168.122.0/24    state
> RELATED,ESTABLISHED
> ACCEPT     all  --  192.168.122.0/24     0.0.0.0/0
> ACCEPT     all  --  0.0.0.0/0            0.0.0.0/0
> REJECT     all  --  0.0.0.0/0            0.0.0.0/0           reject-with
> icmp-port-unreachable
> REJECT     all  --  0.0.0.0/0            0.0.0.0/0           reject-with
> icmp-port-unreachable
> ACCEPT     all  --  0.0.0.0/0            0.0.0.0/0           PHYSDEV match
> --physdev-in vif1.0
>
> Chain OUTPUT (policy ACCEPT)
> target     prot opt source               destination
>
>
> [root@prcapp02 xen]# ip route show
> 192.168.1.0/24 dev eth0  proto kernel  scope link  src 192.168.1.14
> 169.254.0.0/16 dev eth0  scope link
> 172.17.0.0/16 dev eth1  proto kernel  scope link  src 172.17.0.1
> default via 192.168.1.254 dev eth0
>
>
>> -You might need to check out iptables and routes on your domUs
>
> iptables is empty in the domU I'm working with.
>
> The routing situation is screwy, and may be the underlying problem.  I
> can't get it to configure with the IP and routes I want.
>
> In my head, with the bridge setup (this domU plugged into a bridge that
> has everything else including both ethernet interfaces plugged into it) I
> think of my domU as being on a lan segment with everybody else.  That
> should be simple (except it'll be a multi-network LAN segment), but it
> doesn't seem to be.
>
> I worked on bridge and routing code for a while in the 80s and 90s, so
> it's possible that my vague belief that I understand something about how
> this works may be twisting around and biting me on the ass, too.
>
>
>> -Your physical switch could be limiting the port to one MAC or something
>> like this, I have seen issues with Cisco switches doing so in the past on
>> this list
>
> Interesting.  I'll have to think about how I could test this, because I
> believe the switch is from Cisco.
>
>> Also, are you running SELinux?  I haven't read about any issues with Xen
>> networking caused by it, but it's probably worth investigating.
>
> It's set to disabled currently (apparently a necessary step to getting LVS
> working).
>
> --
> David Dyer-Bennet, dd-b@xxxxxxxx; http://dd-b.net/
> Snapshots: http://dd-b.net/dd-b/SnapshotAlbum/data/
> Photos: http://dd-b.net/photography/gallery/
> Dragaera: http://dragaera.info
>
>
> _______________________________________________
> Xen-users mailing list
> Xen-users@xxxxxxxxxxxxxxxxxxx
> http://lists.xensource.com/xen-users
>

_______________________________________________
Xen-users mailing list
Xen-users@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-users