[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [Xen-devel] Hypervisor build can't work without `python` (Was: Re: Xen commit 9b0bc91b3 possibly removed too much info from README)


  • To: Andrew Cooper <andrew.cooper3@xxxxxxxxxx>, Wei Liu <wei.liu2@xxxxxxxxxx>
  • From: Juergen Gross <jgross@xxxxxxxx>
  • Date: Wed, 17 Apr 2019 14:44:21 +0200
  • Autocrypt: addr=jgross@xxxxxxxx; prefer-encrypt=mutual; keydata= mQENBFOMcBYBCACgGjqjoGvbEouQZw/ToiBg9W98AlM2QHV+iNHsEs7kxWhKMjrioyspZKOB ycWxw3ie3j9uvg9EOB3aN4xiTv4qbnGiTr3oJhkB1gsb6ToJQZ8uxGq2kaV2KL9650I1SJve dYm8Of8Zd621lSmoKOwlNClALZNew72NjJLEzTalU1OdT7/i1TXkH09XSSI8mEQ/ouNcMvIJ NwQpd369y9bfIhWUiVXEK7MlRgUG6MvIj6Y3Am/BBLUVbDa4+gmzDC9ezlZkTZG2t14zWPvx XP3FAp2pkW0xqG7/377qptDmrk42GlSKN4z76ELnLxussxc7I2hx18NUcbP8+uty4bMxABEB AAG0H0p1ZXJnZW4gR3Jvc3MgPGpncm9zc0BzdXNlLmNvbT6JATkEEwECACMFAlOMcK8CGwMH CwkIBwMCAQYVCAIJCgsEFgIDAQIeAQIXgAAKCRCw3p3WKL8TL8eZB/9G0juS/kDY9LhEXseh mE9U+iA1VsLhgDqVbsOtZ/S14LRFHczNd/Lqkn7souCSoyWsBs3/wO+OjPvxf7m+Ef+sMtr0 G5lCWEWa9wa0IXx5HRPW/ScL+e4AVUbL7rurYMfwCzco+7TfjhMEOkC+va5gzi1KrErgNRHH kg3PhlnRY0Udyqx++UYkAsN4TQuEhNN32MvN0Np3WlBJOgKcuXpIElmMM5f1BBzJSKBkW0Jc Wy3h2Wy912vHKpPV/Xv7ZwVJ27v7KcuZcErtptDevAljxJtE7aJG6WiBzm+v9EswyWxwMCIO RoVBYuiocc51872tRGywc03xaQydB+9R7BHPuQENBFOMcBYBCADLMfoA44MwGOB9YT1V4KCy vAfd7E0BTfaAurbG+Olacciz3yd09QOmejFZC6AnoykydyvTFLAWYcSCdISMr88COmmCbJzn sHAogjexXiif6ANUUlHpjxlHCCcELmZUzomNDnEOTxZFeWMTFF9Rf2k2F0Tl4E5kmsNGgtSa aMO0rNZoOEiD/7UfPP3dfh8JCQ1VtUUsQtT1sxos8Eb/HmriJhnaTZ7Hp3jtgTVkV0ybpgFg w6WMaRkrBh17mV0z2ajjmabB7SJxcouSkR0hcpNl4oM74d2/VqoW4BxxxOD1FcNCObCELfIS auZx+XT6s+CE7Qi/c44ibBMR7hyjdzWbABEBAAGJAR8EGAECAAkFAlOMcBYCGwwACgkQsN6d 1ii/Ey9D+Af/WFr3q+bg/8v5tCknCtn92d5lyYTBNt7xgWzDZX8G6/pngzKyWfedArllp0Pn fgIXtMNV+3t8Li1Tg843EXkP7+2+CQ98MB8XvvPLYAfW8nNDV85TyVgWlldNcgdv7nn1Sq8g HwB2BHdIAkYce3hEoDQXt/mKlgEGsLpzJcnLKimtPXQQy9TxUaLBe9PInPd+Ohix0XOlY+Uk QFEx50Ki3rSDl2Zt2tnkNYKUCvTJq7jvOlaPd6d/W0tZqpyy7KVay+K4aMobDsodB3dvEAs6 ScCnh03dDAFgIq5nsB11j3KPKdVoPlfucX2c7kGNH+LUMbzqV6beIENfNexkOfxHf4kBrQQY AQgAIBYhBIUSZ3Lo9gSUpdCX97DendYovxMvBQJa3fDQAhsCAIEJELDendYovxMvdiAEGRYI AB0WIQRTLbB6QfY48x44uB6AXGG7T9hjvgUCWt3w0AAKCRCAXGG7T9hjvk2LAP99B/9FenK/ 1lfifxQmsoOrjbZtzCS6OKxPqOLHaY47BgEAqKKn36YAPpbk09d2GTVetoQJwiylx/Z9/mQI CUbQMg1pNQf9EjA1bNcMbnzJCgt0P9Q9wWCLwZa01SnQWFz8Z4HEaKldie+5bHBL5CzVBrLv 81tqX+/j95llpazzCXZW2sdNL3r8gXqrajSox7LR2rYDGdltAhQuISd2BHrbkQVEWD4hs7iV 1KQHe2uwXbKlguKPhk5ubZxqwsg/uIHw0qZDk+d0vxjTtO2JD5Jv/CeDgaBX4Emgp0NYs8IC UIyKXBtnzwiNv4cX9qKlz2Gyq9b+GdcLYZqMlIBjdCz0yJvgeb3WPNsCOanvbjelDhskx9gd 6YUUFFqgsLtrKpCNyy203a58g2WosU9k9H+LcheS37Ph2vMVTISMszW9W8gyORSgmw==
  • Cc: Kevin Buckley <kevin.buckley.ecs.vuw.ac.nz@xxxxxxxxx>, xen-devel@xxxxxxxxxxxxxxxxxxxx, Julien Grall <julien.grall@xxxxxxx>, Stefano Stabellini <sstabellini@xxxxxxxxxx>, Jan Beulich <JBeulich@xxxxxxxx>
  • Delivery-date: Wed, 17 Apr 2019 12:44:43 +0000
  • List-id: Xen developer discussion <xen-devel.lists.xenproject.org>
  • Openpgp: preference=signencrypt

On 17/04/2019 13:53, Andrew Cooper wrote:
> On 17/04/2019 12:43, Juergen Gross wrote:
>> On 17/04/2019 13:26, Wei Liu wrote:
>>> On Wed, Apr 17, 2019 at 12:23:29PM +0100, Andrew Cooper wrote:
>>>> On 17/04/2019 12:16, Wei Liu wrote:
>>>>> On Wed, Apr 17, 2019 at 12:15:04PM +0100, Andrew Cooper wrote:
>>>>>> On 17/04/2019 12:03, Wei Liu wrote:
>>>>>>> On Wed, Apr 17, 2019 at 11:58:49AM +0100, Andrew Cooper wrote:
>>>>>>>> On 17/04/2019 11:57, Wei Liu wrote:
>>>>>>>>> On Wed, Apr 17, 2019 at 11:44:36AM +0100, Andrew Cooper wrote:
>>>>>>>>>> On 17/04/2019 11:41, Wei Liu wrote:
>>>>>>>>>>> On Wed, Apr 17, 2019 at 10:56:57AM +0100, Wei Liu wrote:
>>>>>>>>>>>>> Here's what i did having pulled the master at commit cb70a26
>>>>>>>>>>>>>
>>>>>>>>>>>>> tar xf /path/to/xen-cb70a26.tar.gz
>>>>>>>>>>>>>
>>>>>>>>>>>>> cd xen-master/
>>>>>>>>>>>>>
>>>>>>>>>>>>> PYTHON=/usr/bin/python3 ./configure --prefix=/usr   \
>>>>>>>>>>>>>    --disable-seabios   \
>>>>>>>>>>>>>    --disable-qemu-traditional  \
>>>>>>>>>>>>>    --disable-rombios     \
>>>>>>>>>>>>>    --disable-stubdom 2>&1 | tee ../config.log
>>>>>>>>>>>>>
>>>>>>>>>>>>> make EFI_DIR=/usr/lib/efi world 2>&1 | tee ../make_world_3.out
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> ...
>>>>>>>>>>>>> gcc -Wp,-MD,tools/kconfig/.zconf.tab.o.d    -D_GNU_SOURCE
>>>>>>>>>>>>> -D_DEFAULT_SOURCE -DCURSES_LOC="<ncurses.h>" -DNCURSES_WIDECHAR=1
>>>>>>>>>>>>> -DLOCALE  -Itools/kconfig -c -o tools/kconfig/zconf.tab.o
>>>>>>>>>>>>> tools/kconfig/zconf.tab.c
>>>>>>>>>>>>> gcc  -o tools/kconfig/conf tools/kconfig/conf.o 
>>>>>>>>>>>>> tools/kconfig/zconf.tab.o
>>>>>>>>>>>>> tools/kconfig/conf -s --silentoldconfig Kconfig
>>>>>>>>>>>>> make[3]: Leaving directory '/usr/src/xen/xen-master/xen'
>>>>>>>>>>>>> make -f Rules.mk _install
>>>>>>>>>>>>> make[3]: Entering directory '/usr/src/xen/xen-master/xen'
>>>>>>>>>>>>> make -C tools
>>>>>>>>>>>>> make[4]: Entering directory '/usr/src/xen/xen-master/xen/tools'
>>>>>>>>>>>>> make symbols
>>>>>>>>>>>>> make[5]: Entering directory '/usr/src/xen/xen-master/xen/tools'
>>>>>>>>>>>>> gcc -Wall -Werror -Wstrict-prototypes -O2 -fomit-frame-pointer
>>>>>>>>>>>>> -fno-strict-aliasing -Wdeclaration-after-statement -o symbols
>>>>>>>>>>>>> symbols.c
>>>>>>>>>>>>> make[5]: Leaving directory '/usr/src/xen/xen-master/xen/tools'
>>>>>>>>>>>>> make[4]: Leaving directory '/usr/src/xen/xen-master/xen/tools'
>>>>>>>>>>>>> make -f /usr/src/xen/xen-master/xen/Rules.mk include/xen/compile.h
>>>>>>>>>>>>> make[4]: Entering directory '/usr/src/xen/xen-master/xen'
>>>>>>>>>>>>>  Xen 4.13-unstable
>>>>>>>>>>>>> /bin/sh: python: command not found
>>>>>>>>>>>>> make[4]: *** [Makefile:169: include/xen/compile.h] Error 127
>>>>>>>>>>>> Here. The PYTHON invocation is from the hypervisor build, not the 
>>>>>>>>>>>> tools
>>>>>>>>>>>> build.
>>>>>>>>>>>>
>>>>>>>>>>>> Thanks for the log.  I will have a closer look.
>>>>>>>>>>>>
>>>>>>>>>>>>> make[4]: Leaving directory '/usr/src/xen/xen-master/xen'
>>>>>>>>>>>>> make[3]: *** [Makefile:137: /usr/src/xen/xen-master/xen/xen] 
>>>>>>>>>>>>> Error 2
>>>>>>>>>>>>> make[3]: Leaving directory '/usr/src/xen/xen-master/xen'
>>>>>>>>>>>>> make[2]: *** [Makefile:45: install] Error 2
>>>>>>>>>>>>> make[2]: Leaving directory '/usr/src/xen/xen-master/xen'
>>>>>>>>>>>>> make[1]: *** [Makefile:123: install-xen] Error 2
>>>>>>>>>>>>> make[1]: Leaving directory '/usr/src/xen/xen-master'
>>>>>>>>>>>>> make: *** [Makefile:165: world] Error 2
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> Note the
>>>>>>>>>>>>>
>>>>>>>>>>>>> /bin/sh: python: command not found
>>>>>>>>>>>>>
>>>>>>>>>>> The issue here is on LFS there is no /usr/bin/python. So the 
>>>>>>>>>>> hypervisor
>>>>>>>>>>> build is broken because xen/include/compile.h invokes
>>>>>>>>>>>
>>>>>>>>>>>  @$(PYTHON) tools/fig-to-oct.py < .banner >> $@.new
>>>>>>>>>>>
>>>>>>>>>>> There are other appearances of PYTHON in hypervisor build, which are
>>>>>>>>>>> equally broken.
>>>>>>>>>>>
>>>>>>>>>>> We had / have a policy to avoid making hypervisor build dependent on
>>>>>>>>>>> configure. I have CC'ed more people on their opinions.
>>>>>>>>>> Where has $(PYTHON) got lost?  It was specified correctly to begin 
>>>>>>>>>> with.
>>>>>>>>> Where is it specified?
>>>>>>>> Oh - I see now.
>>>>>>>>
>>>>>>>> It is specified for ./configure, but not for make.
>>>>>>> Right.
>>>>>>>
>>>>>>> It works for tools because tools build includes config/Tools.mk which
>>>>>>> contains the correct python path. Tools.mk is generated by configure.
>>>>>>>  
>>>>>>> Obviously hypervisor build shouldn't include Tools.mk. Furthermore, the
>>>>>>> policy of "hv shouldn't depend on configure" makes this route a no-go.
>>>>>>>
>>>>>>> Either we change the policy, or we document the deficiency, or we come
>>>>>>> up with some clever trick to probe.
>>>>>> Given the shebang in all of our scripts, would `env python` be
>>>>>> sufficiently clever trickery, or is that a can of worms we'd prefer not
>>>>>> to open?
>>>>> I already tried env python -- it didn't work. I think it searches PATHS
>>>>> just like a shell would.
>>>> So this particular use of python is completely trivial.  It is an octal
>>>> conversion on the banner string, to feed into a string for XEN_BANNER
>>>>
>>>> It exists because I removed a fork of figlet[1] from Xen's source tree
>>>> with this script, but its probably just as easy to do with awk or
>>>> something similar.
>>>>
>>>> ~Andrew
>>>>
>>>> [1] Yes - Xen did use to carry a fork of figlet which differed solely by
>>>> being able to output an octal-escaped string.  No - I have no words
>>>> which can possibly be used to defend this...
>>>
>>> But there are other less trivial ones. See xen/include/Makefile. XSM
>>> also relies on it to generate policy. What do you want to do with those?
>> Shouldn't _some_ settings of configure end up in the hypervisor build,
>> too? Like CC, AWK, SHELL, HOSTCC?
>>
>> Maybe we need a rather small basic_tools.mk containing those settings
>> (with a sane default, of course)?
> 
> In the past, there has been a vehement objection to requiring
> ./configure before building Xen, and I still agree with this point of
> view.  Noone should need to run ./configure to be able to build a
> freestanding item with no userspace connection.
> 
> The problem Xen has is that we've got two vastly different projects in
> the same source repo, and `make` runes which encompass both.
> 
> Could we go with some compromise where the results of configure are used
> if they are available, but for people not wanting to run configure, if
> they've got non-standard standard tools, they need to pass them to make
> as well?  This is consistent with how cross compiling works.

That should be fairly easy.

Let Config.mk have:

-include config/basic_tools.mk

at the very beginning, with config/basic_tools.mk being created via
configure and containing the basic tool variables:

CC := gcc
AWK := awk
PYTHON := python3
...

As Config.mk will set those variables with defualt values in case they
are empty all is fine then.


Juergen

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxxx
https://lists.xenproject.org/mailman/listinfo/xen-devel

 


Rackspace

Lists.xenproject.org is hosted with RackSpace, monitoring our
servers 24x7x365 and backed by RackSpace's Fanatical Support®.