Re: [Xen-devel] reboot driver domain, vifX.Y = NO-CARRIER?

[Wei Liu <wei.liu2@xxxxxxxxxx>]

On Fri, Apr 27, 2018 at 04:14:16PM +0000, Jason Cooper wrote:
> On Fri, Apr 27, 2018 at 04:52:57PM +0100, Andrew Cooper wrote:
> > On 27/04/18 16:35, Jason Cooper wrote:
> > > On Fri, Apr 27, 2018 at 04:11:39PM +0100, Andrew Cooper wrote:
> > >> On 27/04/18 16:03, Jason Cooper wrote:
> > >>> The problem occurs when I reboot a driver domain.  Regardless of the
> > >>> type of guest attached to it, I'm unable to re-establish connectivity
> > >>> between the driver domain and the re-attached guest.  e.g. I reboot
> > >>> GW/FW, then re-attach VM1, VM2 and the rest.  No matter how I do it, I
> > >>> get:
> > >>>
> > >>> $ ip link
> > >>> ...
> > >>> 11: vif20.1: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc mq 
> > >>> master br10 qlen 32
> > >>>     link/ether fe:ff:ff:ff:ff:ff brd ff:ff:ff:ff:ff:ff
> > >>>
> > >>> In the driver domain.  At this point, absolutely no packets flow between
> > >>> the two VMs.  Not even ARP.  The only solution, so far, is to 
> > >>> unnecessarily
> > >>> reboot the PV guests.  After that, networking is fine.
> > >>>
> > >>> Any thoughts?
> > >> The underlying problem is that the frontend/backend setup in xenstore
> > >> encodes the domid in path, and changing that isn't transparent to the
> > >> guest at all.
> > > Oh joy.  Would seem to make more send to use the domain name or the
> > > uuid...
> > 
> > domids are also used in the grant and event hypercall interfaces with Xen.
> > 
> > There is no way this horse is being put back in its stable...
> 
> :-(
> 
> > >> The best idea we came up with was to reboot the driver domain and reuse
> > >> its old domid, at which point all the xenstore paths would remain
> > >> valid.  There is support in Xen for explicitly choosing the domid of a
> > >> domain, but I don't think that it is wired up sensibly in xl.
> > > hmmm, yes.  It's not wired up at all afaict.  Mind giving me a hint on
> > > how to reuse the domid?
> > 
> > xc_domain_create() takes a domid value by pointer.  Passing a value
> > other than zero will cause Xen to use that domid, rather than by
> > searching for the next free domid.
> > 
> > diff --git a/tools/libxl/libxl_create.c b/tools/libxl/libxl_create.c
> > index b5e27a7..7866092 100644
> > --- a/tools/libxl/libxl_create.c
> > +++ b/tools/libxl/libxl_create.c
> > @@ -583,6 +583,7 @@ int libxl__domain_make(libxl__gc *gc,
> > libxl_domain_config *d_config,
> >              goto out;
> >          }
> >  
> > +        *domid = atoi(getenv("OVERRIDE_DOMID") ?: "0");
> >          ret = xc_domain_create(ctx->xch, info->ssidref, handle, flags, 
> > domid,
> >                                 &xc_config);
> >          if (ret < 0) {
> > 
> > This gross hack may get you somewhere (Entirely untested).
> 
> Gah!  Yep, that's just what I needed, thanks!  I don't suppose a patch
> series adding a 'domid' field to the domain config file would be
> rejected outright?  That would allow callers of xl to use key=value for
> reboot scripts like mine, and also allow for a static domid setup of the
> driver domains if folks want that.

Seems a bit  hacky to me. You also need to reserve a set of domids
before hand?

Wei.

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxxx
https://lists.xenproject.org/mailman/listinfo/xen-devel