Re: [Xen-devel] BUG: [?] host-only networking under HVM is broken with custom script

["Ray Barnes" <tical.net@xxxxxxxxx>]

Thanks Kier.  My script does get executed but apparently it's being
done too late in the game for the addition of the bridge to be
effective.  To solve for this, based on your comments, I borrowed some
code from one of the other scripts and stuck it in qemu-ifup:

    if [ ! -e "/sys/class/net/${bridge}/bridge" ]; then
        brctl addbr ${bridge}
        brctl stp ${bridge} off
        brctl setfd ${bridge} 0
    fi

This works perfectly, only with the added side-effect that tap0 ends
up on the bridge too.  I'd imagine this will cause problems once I get
more than one HVM up under the same scenario on the same node, so I
delete tap0 from the bridge at the bottom of qemu-ifup.  As an aside,
most people (according to what I've read) seem to take it for granted
that you should bring up the bridge outside of Xen (i.e. in the OS's
network scripts).  Obviously that's a flawed methodology that doesn't
scale well.  Surely there's a cleaner way of doing this?  Any plan to
code something like this into a future release?

-Ray


On Mon, Jun 2, 2008 at 2:55 AM, Keir Fraser <keir.fraser@xxxxxxxxxxxxx> wrote:
> I doubt your script is getting executed. HVM guests have a script run by
> qemu-dm: the default location of that is /etc/xen/qemu-ifup and I think xend
> never overrides that. You probably need to do some command-line splicing in
> tools/python/xen/xend/image.py -- we already extract the 'bridge' parameter
> and pass that along to qemu-dm, but the same is not done for the 'script'
> parameter.
>
>  -- Keir
>
> On 2/6/08 07:37, "Ray Barnes" <tical.net@xxxxxxxxx> wrote:
>
>> Resending this to xen-devel since no one rendered a response on xen-users
>>
>> ---
>>
>> I'm bumping into a problem which looks like a bug.  Perhaps it's for
>> lack of knowledge (and _documentation_ of HVM).  If I do the
>> following:
>>
>> brctl addbr bmette31
>> ifconfig bmette31 1.2.3.249 netmask 255.255.255.248
>> xm create bmette31
>>
>> Where "bmette31" has a config file that has a vif entry like: vif = [
>> 'type=ioemu, mac=00:16:3e:00:00:07, bridge=bmette31']  this works
>> *just fine*.  I'm able to route through the host to the HVM domain
>> which is set to 1.2.3.250.  However, whenever I use a script which is
>> loosely based on a vif-bridge script that I got from /etc/xen/scripts
>> running Xen 3.2.1 compiled from source, it does not work.
>> Specifically, I can reach 1.2.3.249 from the outside but not
>> 1.2.3.250.  As far as I can tell, the problem is that whenever 'brctl
>> addbr bmette31' is invoked from outside the script, networking between
>> the host and the HVM guest works fine, but when invoked from within
>> the script, networking between the host and the HVM guest does not
>> work.  Note that while the domain is running (and using the script
>> below), 'brctl show' indicates that I have a bridge called bmette31
>> and that 'ifconfig bmette31' shows the right IP.  So just to recap -
>> if I comment out 'brctl addbr bmette31' from the script below and run
>> it manually then start the HVM guest, networking is fine, but if I run
>> that from within the script it does not work.
>>
>> One thing I notice immediately is that when I create the bridge from
>> the command-line, the bridge starts out with all zeros in the MAC
>> address, and once the custom script runs upon starting the domU, it
>> gets a MAC address like so:
>>
>> bridge name     bridge id               STP enabled     interfaces
>> bmette31                8000.8a10ff068f3f       no              vif61.0
>>
>> That's the working scenario as seen by 'brctl show'.  The non-working
>> scenario (invoking 'brctl addbr bmette31' from within the script) look
>> like this:
>>
>> bridge name     bridge id               STP enabled     interfaces
>> bmette31                8000.feffffffffff       no              vif62.0
>>
>> FYI, I only care about this because I'm trying to make my domUs
>> extremely portable (relying only on a config file and a vif script),
>> i.e. for live migration.  Staticly creating hundreds (thousands?) of
>> bridges across all nodes isn't an option.  Any ideas?
>>
>>
>> -Ray
>>
>>
>> #!/bin/bash
>> #
>> #vif bridge script for HVMs
>> #invoke like this:
>> #
>> #vif = [ 'type=ioemu, mac=00:16:3e:00:00:07, bridge=bmette31,
>> script=vif-bmette31' ]
>>
>> /usr/sbin/brctl addbr bmette31
>>
>> dir=$(dirname "$0")
>> . "$dir/vif-common.sh"
>>
>> bridge=${bridge:-}
>> bridge=$(xenstore_read_default "$XENBUS_PATH/bridge" "$bridge")
>>
>>
>> RET=0
>> ip link show $bridge 1>/dev/null 2>&1 || RET=1
>> if [ "$RET" -eq 1 ]
>> then
>> #       do_without_error brctl addbr "$bridge"
>>     fatal "Could not find bridge device $bridge"
>> fi
>>
>> case "$command" in
>>     online)
>>         setup_bridge_port "$vif"
>>         add_to_bridge "$bridge" "$vif"
>>         sleep 2
>>         ifconfig "$bridge" 1.2.3.249 netmask 255.255.255.248
>>         ;;
>>
>>     offline)
>>         do_without_error ifconfig "$bridge" down
>>         do_without_error brctl delbr "$bridge"
>>         ;;
>> esac
>>
>>
>> log debug "Successful vif-bridge $command for $vif, bridge $bridge."
>> if [ "$command" == "online" ]
>> then
>>   success
>> fi
>>
>> _______________________________________________
>> Xen-devel mailing list
>> Xen-devel@xxxxxxxxxxxxxxxxxxx
>> http://lists.xensource.com/xen-devel
>
>
>

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-devel