Re: Network driver domain broken

[Roger Pau Monné <roger.pau@xxxxxxxxxx>]

On Thu, Mar 03, 2022 at 01:08:31PM -0500, Jason Andryuk wrote:
> On Thu, Mar 3, 2022 at 11:34 AM Roger Pau Monné <roger.pau@xxxxxxxxxx> wrote:
> >
> > On Thu, Mar 03, 2022 at 05:01:23PM +0100, Andrea Stevanato wrote:
> > > On 03/03/2022 15:54, Andrea Stevanato wrote:
> > > > Hi all,
> > > >
> > > > according to the conversation that I had with royger, aa67b97ed34  
> > > > broke the driver domain support.
> > > >
> > > > What I'm trying to do is to setup networking between guests using 
> > > > driver domain. Therefore, the guest (driver) has been started with the 
> > > > following cfg.
> > > >
> > > > name    = "guest0"
> > > > kernel  = "/media/sd-mmcblk0p1/Image"
> > > > ramdisk = "/media/sd-mmcblk0p1/rootfs.cpio.gz"
> > > > extra   = "console=hvc0 rdinit=/sbin/init root=/dev/ram0"
> > > > memory  = 1024 vcpus   = 2
> > > > driver_domain = 1
> > > >
> > > > On guest0 I created the bridge, assigned a static IP and started the 
> > > > udhcpd on xenbr0 interface.
> > > > While the second guest has been started with the following cfg:
> > > >
> > > > name    = "guest1"
> > > > kernel  = "/media/sd-mmcblk0p1/Image"
> > > > ramdisk = "/media/sd-mmcblk0p1/rootfs.cpio.gz"
> > > > extra   = "console=hvc0 rdinit=/sbin/init root=/dev/ram0"
> > > > memory  = 1024 vcpus   = 2
> > > > vcpus   = 2
> > > > vif = [ 'bridge=xenbr0, backend=guest0' ]
> > > >
> > > > Follows the result of strace xl devd:
> > > >
> > > > # strace xl devd
> > > > execve("/usr/sbin/xl", ["xl", "devd"], 0xffffdf0420c8 /* 13 vars */) = 0
> 
> > > > ioctl(5, _IOC(_IOC_NONE, 0x50, 0, 0x30), 0xffffe6e41b40) = -1 EPERM 
> > > > (Operation not permitted)
> > > > write(2, "libxl: ", 7libxl: )                  = 7
> > > > write(2, "error: ", 7error: )                  = 7
> > > > write(2, "libxl_utils.c:820:libxl_cpu_bitm"..., 
> > > > 87libxl_utils.c:820:libxl_cpu_bitmap_alloc: failed to retrieve the 
> > > > maximum number of cpus) = 87
> > > > write(2, "\n", 1
> > > > )                       = 1
> > > > clone(child_stack=NULL, 
> > > > flags=CLONE_CHILD_CLEARTID|CLONE_CHILD_SETTID|SIGCHLD, 
> > > > child_tidptr=0xffff9ee7a0e0) = 814
> > > > wait4(814, [{WIFEXITED(s) && WEXITSTATUS(s) == 0}], 0, NULL) = 814
> > > > --- SIGCHLD {si_signo=SIGCHLD, si_code=CLD_EXITED, si_pid=814, 
> > > > si_uid=0, si_status=0, si_utime=2, si_stime=2} ---
> 
> xl devd is daemonizing, but strace is only following the first
> process.  Use `strace xl devd -F` to prevent the daemonizing (or
> `strace -f xl devd` to follow children).

Or as a first step try to see what kind of messages you get from `xl
devd -F` when trying to attach a device using the driver domain.

> > > > close(6)                                = 0
> > > > close(5)                                = 0
> > > > munmap(0xffff9f45f000, 4096)            = 0
> > > > close(7)                                = 0
> > > > close(10)                               = 0
> > > > close(9)                                = 0
> > > > close(8)                                = 0
> > > > close(11)                               = 0
> > > > close(3)                                = 0
> > > > close(4)                                = 0
> > > > exit_group(0)                           = ?
> > > > +++ exited with 0 +++
> > > >
> > > > royger told me that it is a BUG and not an issue with my setup. 
> > > > Therefore here I am.
> >
> > Just a bit more context: AFAICT the calls to libxl_cpu_bitmap_alloc in
> > parse_global_config will prevent xl from being usable on anything
> > different than the control domain (due to sysctl only available to
> > privileged domains). This is an issue for 'xl devd', as it won't
> > start anymore.
> 
> These look non-fatal at first glance?

Indeed. I was too quick reading the trace and assumed `xl devd` exited
due to the errors, but those are non fatal, the process just
daemonized.

Thanks, Roger.