Re: [Xen-users] Wrong version of xen-utils after building xen from source causes xen to exit with error

[Brett Stahlman <brettstahlman@xxxxxxxxx>]

On Thu, Sep 15, 2016 at 9:07 AM, George Dunlap <george.dunlap@xxxxxxxxxx> wrote:
> On 15/09/16 14:59, Brett Stahlman wrote:
>> On Thu, Sep 15, 2016 at 5:53 AM, George Dunlap <george.dunlap@xxxxxxxxxx> 
>> wrote:
>>> On 14/09/16 15:40, Brett Stahlman wrote:
>>>> On Wed, Sep 14, 2016 at 5:32 AM, George Dunlap <george.dunlap@xxxxxxxxxx> 
>>>> wrote:
>>>>> On Thu, Sep 8, 2016 at 8:43 PM, Brett Stahlman <brettstahlman@xxxxxxxxx> 
>>>>> wrote:
>>>>>> ** Update **
>>>>>> Following recommendations on a page dealing with the installation of
>>>>>> multiple versions of Xen side-by-side, I re-ran configure like this...
>>>>>>
>>>>>> configure --prefix=/home/bstahlman/install/xen \
>>>>>> --sysconfdir=/home/bstahlman/install/xen/etc \
>>>>>> --enable-rpath
>>>>>>
>>>>>> ...and was then able to run xl by specifying the path to the sbin/xl
>>>>>> under the directory specified with --prefix.
>>>>>>
>>>>>> Aside: I also had to add this line to /etc/fstab...
>>>>>>
>>>>>> none /proc/xen xenfs defaults 0 0
>>>>>>
>>>>>> ...and I have to start the xencommons service manually like so:
>>>>>>
>>>>>> sudo ~/install/xen/etc/init.d/xencommons start
>>>>>>
>>>>>> But now I'm able to run xen-create-image to create an ubuntu guest:
>>>>>> sudo xen-create-image --hostname ubuntu-test --dhcp --dir
>>>>>> ~/xen/images/ --size=4G --role=udev
>>>>>>
>>>>>> (Note that I had to use the script installed by package manager, as I
>>>>>> don't see a version of xen-create-image under my custom install
>>>>>> directory). Strangely, xen-create-image gives what looks like an
>>>>>> error...
>>>>>>
>>>>>> ERROR: Can't find version 4.8 of xen utils, bailing out!
>>>>>>
>>>>>> ...but the command appears to work, and I'm able to start the guest by
>>>>>> running xl on the resulting config file:
>>>>>> sudo ./install/xen/sbin/xl create /etc/xen/ubuntu-test.cfg -c
>>>>>>
>>>>>> The problem occurs when I attempt to create an HVM guest:
>>>>>> $ sudo ~/install/xen/sbin/xl create ~/cfg/xen/xl-ubuntu-hvmloader.cfg -c
>>>>>> Parsing config from /home/bstahlman/cfg/xen/xl-ubuntu-hvmloader.cfg
>>>>>> libxl: error: libxl_create.c:562:libxl__domain_make: domain creation
>>>>>> fail: Invalid argument
>>>>>> libxl: error: libxl_create.c:904:initiate_domain_create: cannot make 
>>>>>> domain: -3
>>>>>>
>>>>>> I initially assumed the error must be due to the options in the config
>>>>>> file (pasted at the end), but when I ran the xl create command in gdb,
>>>>>> I discovered that the option parsing was successful: the error occurs
>>>>>> because the ioctl() call in the following function returns -1:
>>>>>
>>>>> ioctl returns -1 on failure and then sets the global variable 'errno'.
>>>>> If you see above, libxl itself gives you the appropriate
>>>>> interpretation of errno -- "Invalid argument".
>>>>>
>>>>> Given that your config file doesn't seem to have anything strange in
>>>>> it, I would suspect that you're still having some sort of issue where
>>>>> you're running the wrong set of tools (or perhaps the wrong library)
>>>>> for the Xen version that you're running.
>>>>>
>>>>> As an aside, another useful trick if you want to build from source is
>>>>> to use "make debball".  That will create a .deb package you can
>>>>> install and remove easily.  The key thing is this debball is NOT A
>>>>> COMPLETE PACKAGE -- you still need to run ldconfig and everything
>>>>> manually, just as you would if you'd run "make install".  It's just a
>>>>> lot easier to remove and/or update than "make install", since dpkg
>>>>> keeps track of all the files for you.
>>>>
>>>> Hmm... I never ran ldconfig after the `make install', so perhaps that
>>>> could have been the issue?
>>>
>>> Potentially; but normally if that's the issue then running the resulting
>>> binary will give you linking errors.
>>>
>>> If you had previously built the full Debian package with Xen 4.8, it's
>>> *possible* that you had the old libraries in your ld.so.cache, so that
>>> stuff worked even without running ldconfig.
>>>
>>> You could try running "ldd /path/to/your/new/xl" to verify which version
>>> of the library you're actually using when running it.
>>>
>>>> Just yesterday, I was able to build the package obtained with `apt-get
>>>> source' using the following command:
>>>>
>>>> sudo dpkg-buildpackage -us -uc
>>>>
>>>> But this is a bit tedious because each time I want to tweak a file
>>>> (e.g., to put in some debug printf's), I have to run the debian
>>>> "quilt" utility to make a patch. I would prefer to just modify source
>>>> and rebuild. Wasn't aware of the "debball" target. Also, didn't
>>>> realize it was necessary to run ldconfig after installing a package
>>>> with dpkg. What's the standard way to run ldconfig in this case? If I
>>>> built package without using configure to change default paths, which
>>>> directory(s) need to be passed to ldconfig (or are the defaults
>>>> sufficient)?
>>>
>>> Fully-functional packages do the ldconfig for you (along with dependency
>>> tracking, starting system services, &c).  The output of
>>> dpkg-buildpackage will be a fully-functional package with all those bits
>>> handled already.
>>>
>>> The reason the "debball" makefile target is called that is because it's
>>> essentially just a tarball which uses dpkg for file-tracking, not a
>>> fully functional package.  We don't want to be burdened with doing all
>>> the work of maintaining a package -- our downstreams (Debian, Fedora,
>>> &c) are doing a good job of that and there's no need to duplicate their
>>> work.  We just want an easy way to install and remove the files while
>>> doing development, hence "debball" (and the corresponding "rpmball").
>>>
>>>> When I built with dkpg-buildpackage, I got a handful of .debs: e.g.,
>>>> xen-hypervisor, xenstore, xen-utils, xen-system. Would it be best to
>>>> run `dpkg -i' on these files individually, or drop them all in a
>>>> directory and run dpkg on that? (I haven't used dpkg standalone much,
>>>> and want to be sure the dependencies are handled properly.)
>>>
>>> dpkg will handle dependencies in the sense that it will refuse to
>>> install something if there are dependencies that aren't there.  So
>>> normally installing xen-*.deb is the best thing.
>>>
>>> But the output of "make debball" has *no* dependency tracking -- it is
>>> the exact equivalent of "tar xfz [blah]", but easier to remove.  :-)
>>
>> I think I understand. So if I use `make debball' instead of
>> dkpg-buildpackage, I would need to do something like...
>>     sudo dpkg -i xen-*.deb
>>     sudo ldconfig
>> ...and then start services manually after I've booted into Xen: e.g.,
>>     sudo /etc/init.d/xencommons start
>> (I'm assuming that the .deb's produced by the debball package will put
>> things in the normal system directories unless I override with
>> configure: hence, no need to specify directories to ldconfig.)
>
> "make debball" takes exactly what's in xen.git/dist/install (i.e.,
> installing it should be equivalent to doing "make install").  The
> xen.git build process will put binaries and libraries in /usr/local
> unless you specify something else via configure --prefix=.  (The stuff
> in /etc and /boot will always go to /etc and /boot).

Understood. So ldconfig with no args should be sufficient for me, as
/usr/local/lib is in my /etc/ld.so.conf.

Thanks for all your help.
Brett S.

>
>  -George

_______________________________________________
Xen-users mailing list
Xen-users@xxxxxxxxxxxxx
https://lists.xen.org/xen-users