[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [Xen-devel] [PATCH RFC-for-4.12] tools/xen-foreign: Update python scripts to be Py3 compatible


  • To: George Dunlap <george.dunlap@xxxxxxxxxx>, Wei Liu <wei.liu2@xxxxxxxxxx>
  • From: Juergen Gross <jgross@xxxxxxxx>
  • Date: Tue, 5 Mar 2019 13:36:30 +0100
  • Autocrypt: addr=jgross@xxxxxxxx; prefer-encrypt=mutual; keydata= xsBNBFOMcBYBCACgGjqjoGvbEouQZw/ToiBg9W98AlM2QHV+iNHsEs7kxWhKMjrioyspZKOB ycWxw3ie3j9uvg9EOB3aN4xiTv4qbnGiTr3oJhkB1gsb6ToJQZ8uxGq2kaV2KL9650I1SJve dYm8Of8Zd621lSmoKOwlNClALZNew72NjJLEzTalU1OdT7/i1TXkH09XSSI8mEQ/ouNcMvIJ NwQpd369y9bfIhWUiVXEK7MlRgUG6MvIj6Y3Am/BBLUVbDa4+gmzDC9ezlZkTZG2t14zWPvx XP3FAp2pkW0xqG7/377qptDmrk42GlSKN4z76ELnLxussxc7I2hx18NUcbP8+uty4bMxABEB AAHNHkp1ZXJnZW4gR3Jvc3MgPGpncm9zc0BzdXNlLmRlPsLAeQQTAQIAIwUCU4xw6wIbAwcL CQgHAwIBBhUIAgkKCwQWAgMBAh4BAheAAAoJELDendYovxMvi4UH/Ri+OXlObzqMANruTd4N zmVBAZgx1VW6jLc8JZjQuJPSsd/a+bNr3BZeLV6lu4Pf1Yl2Log129EX1KWYiFFvPbIiq5M5 kOXTO8Eas4CaScCvAZ9jCMQCgK3pFqYgirwTgfwnPtxFxO/F3ZcS8jovza5khkSKL9JGq8Nk czDTruQ/oy0WUHdUr9uwEfiD9yPFOGqp4S6cISuzBMvaAiC5YGdUGXuPZKXLpnGSjkZswUzY d9BVSitRL5ldsQCg6GhDoEAeIhUC4SQnT9SOWkoDOSFRXZ+7+WIBGLiWMd+yKDdRG5RyP/8f 3tgGiB6cyuYfPDRGsELGjUaTUq3H2xZgIPfOwE0EU4xwFgEIAMsx+gDjgzAY4H1hPVXgoLK8 B93sTQFN9oC6tsb46VpxyLPfJ3T1A6Z6MVkLoCejKTJ3K9MUsBZhxIJ0hIyvzwI6aYJsnOew cCiCN7FeKJ/oA1RSUemPGUcIJwQuZlTOiY0OcQ5PFkV5YxMUX1F/aTYXROXgTmSaw0aC1Jpo w7Ss1mg4SIP/tR88/d1+HwkJDVW1RSxC1PWzGizwRv8eauImGdpNnseneO2BNWRXTJumAWDD pYxpGSsGHXuZXTPZqOOZpsHtInFyi5KRHSFyk2Xigzvh3b9WqhbgHHHE4PUVw0I5sIQt8hJq 5nH5dPqz4ITtCL9zjiJsExHuHKN3NZsAEQEAAcLAXwQYAQIACQUCU4xwFgIbDAAKCRCw3p3W KL8TL0P4B/9YWver5uD/y/m0KScK2f3Z3mXJhME23vGBbMNlfwbr+meDMrJZ950CuWWnQ+d+ Ahe0w1X7e3wuLVODzjcReQ/v7b4JD3wwHxe+88tgB9byc0NXzlPJWBaWV01yB2/uefVKryAf AHYEd0gCRhx7eESgNBe3+YqWAQawunMlycsqKa09dBDL1PFRosF708ic9346GLHRc6Vj5SRA UTHnQqLetIOXZm3a2eQ1gpQK9MmruO86Vo93p39bS1mqnLLspVrL4rhoyhsOyh0Hd28QCzpJ wKeHTd0MAWAirmewHXWPco8p1Wg+V+5xfZzuQY0f4tQxvOpXpt4gQ1817GQ5/Ed/wsDtBBgB CAAgFiEEhRJncuj2BJSl0Jf3sN6d1ii/Ey8FAlrd8NACGwIAgQkQsN6d1ii/Ey92IAQZFggA HRYhBFMtsHpB9jjzHji4HoBcYbtP2GO+BQJa3fDQAAoJEIBcYbtP2GO+TYsA/30H/0V6cr/W V+J/FCayg6uNtm3MJLo4rE+o4sdpjjsGAQCooqffpgA+luTT13YZNV62hAnCLKXH9n3+ZAgJ RtAyDWk1B/0SMDVs1wxufMkKC3Q/1D3BYIvBlrTVKdBYXPxngcRoqV2J77lscEvkLNUGsu/z W2pf7+P3mWWlrPMJdlbax00vevyBeqtqNKjHstHatgMZ2W0CFC4hJ3YEetuRBURYPiGzuJXU pAd7a7BdsqWC4o+GTm5tnGrCyD+4gfDSpkOT53S/GNO07YkPkm/8J4OBoFfgSaCnQ1izwgJQ jIpcG2fPCI2/hxf2oqXPYbKr1v4Z1wthmoyUgGN0LPTIm+B5vdY82wI5qe9uN6UOGyTH2B3p hRQUWqCwu2sqkI3LLbTdrnyDZaixT2T0f4tyF5Lfs+Ha8xVMhIyzNb1byDI5FKCb
  • Cc: Stefano Stabellini <sstabellini@xxxxxxxxxx>, Konrad Rzeszutek Wilk <konrad.wilk@xxxxxxxxxx>, George Dunlap <George.Dunlap@xxxxxxxxxxxxx>, Andrew Cooper <andrew.cooper3@xxxxxxxxxx>, Tim Deegan <tim@xxxxxxx>, Xen-devel <xen-devel@xxxxxxxxxxxxx>, Julien Grall <julien.grall@xxxxxxx>, Jan Beulich <JBeulich@xxxxxxxx>, Ian Jackson <ian.jackson@xxxxxxxxxx>
  • Delivery-date: Tue, 05 Mar 2019 12:36:40 +0000
  • List-id: Xen developer discussion <xen-devel.lists.xenproject.org>
  • Openpgp: preference=signencrypt

On 05/03/2019 13:33, George Dunlap wrote:
> On 3/5/19 12:31 PM, George Dunlap wrote:
>> On 3/5/19 12:30 PM, Juergen Gross wrote:
>>> On 05/03/2019 13:19, George Dunlap wrote:
>>>> On 3/5/19 12:04 PM, Wei Liu wrote:
>>>>> On Tue, Mar 05, 2019 at 12:41:53PM +0100, Juergen Gross wrote:
>>>>>> On 04/03/2019 20:00, George Dunlap wrote:
>>>>>>> On 3/4/19 6:31 PM, Andrew Cooper wrote:
>>>>>>>> The issues are:
>>>>>>>>  * dict.has_key() was completely removed in Py3
>>>>>>>>  * dict.keys() is an iterable rather than list in Py3, so .sort() 
>>>>>>>> doesn't work.
>>>>>>>>  * list.sort(cmp=) was deprecated in Py2.4 and removed in Py3.  
>>>>>>>> Replace it
>>>>>>>>    with a key= sort instead.
>>>>>>>>
>>>>>>>> This is all compatible with Py2.4 and later, which is when the sorted()
>>>>>>>> builtin was introduced.  Tested with Py2.7 and Py3.4
>>>>>>>
>>>>>>> Did you test ./configure with python3?
>>>>>>>
>>>>>>> Loads of the python-related runes in the configure are also
>>>>>>> python2-only, resulting in (at least for me) the wrong library paths
>>>>>>> being generated, so the following fails:
>>>>>>>
>>>>>>>     PYTHON=/usr/bin/python3 ./configure
>>>>>>>
>>>>>>> With the following message:
>>>>>>>
>>>>>>>   File "<string>", line 1
>>>>>>>     import distutils.sysconfig;     print
>>>>>>> distutils.sysconfig.get_config_var("VERSION")
>>>>>>>                                                   ^
>>>>>>> SyntaxError: invalid syntax
>>>>>>> checking for python3-config... 
>>>>>>> /home/gdunlap/anaconda3/bin/python3-config
>>>>>>> checking Python.h usability... yes
>>>>>>> checking Python.h presence... yes
>>>>>>> checking for Python.h... yes
>>>>>>> checking for PyArg_ParseTuple in -lpython... no
>>>>>>> configure: error: Unable to find a suitable python development library
>>>>>>> configure: error: ./configure failed for tools
>>>>>>>
>>>>>>> Note the error with the VERSION above; that results in looking for
>>>>>>> `-lpython` rather than `-lpython3.7`, which results in the following
>>>>>>> line being executed:
>>>>>>>
>>>>>>> configure:7468: checking for PyArg_ParseTuple in -lpython
>>>>>>> configure:7493: gcc -o conftest -g -O2 -g -O2
>>>>>>> -I/persistent/local/home/dunlapg/anaconda3/include/python3.7m
>>>>>>> -I/persistent/local/home/dunlapg/anaconda3/include/python3.7m
>>>>>>> -Wno-unused-result -Wsign-compare -march=nocona -mtune=haswell
>>>>>>> -ftree-vectorize -fPIC -fstack-protector-strong -fno-plt -O3 -pipe
>>>>>>> -fdebug-prefix-map==/usr/local/src/conda/-
>>>>>>> -fdebug-prefix-map==/usr/local/src/conda-prefix -fuse-linker-plugin
>>>>>>> -ffat-lto-objects -flto-partition=none -flto -flto -fuse-linker-plugin
>>>>>>> -ffat-lto-objects -flto-partition=none -g -DNDEBUG -fwrapv -O3 -Wall
>>>>>>> -L/persistent/local/home/dunlapg/anaconda3/lib/python3.7/config-3.7m-x86_64-linux-gnu
>>>>>>> -L/persistent/local/home/dunlapg/anaconda3/lib -lpython3.7m -lpthread
>>>>>>> -ldl  -lutil -lrt -lm  -Xlinker -export-dynamic conftest.c -lpython
>>>>>>> -llzma  -lpython3.7m -lpthread -ldl  -lutil -lrt -lm  -llzma  >&5
>>>>>>> /usr/bin/ld: cannot find -lpython
>>>>>>> collect2: error: ld returned 1 exit status
>>>>>>>
>>>>>>> The following fixes it for me:
>>>>>>>
>>>>>>> diff --git a/tools/configure b/tools/configure
>>>>>>> index acc857510e..d7e21db959 100755
>>>>>>> --- a/tools/configure
>>>>>>> +++ b/tools/configure
>>>>>>> @@ -6953,7 +6953,7 @@ if echo "$PYTHON" | grep -q "^/"; then :
>>>>>>>      PYTHON=`basename $PYTHONPATH`
>>>>>>>
>>>>>>>  elif test -z "$PYTHON"; then :
>>>>>>> -  PYTHON="python"
>>>>>>> +  PYTHON="python2"
>>>>>>>  else
>>>>>>>    as_fn_error $? "PYTHON specified, but is not an absolute path"
>>>>>>> "$LINENO" 5
>>>>>>>  fi
>>>>>>> diff --git a/tools/configure.ac b/tools/configure.ac
>>>>>>> index 1499344ce6..adf70aeb33 100644
>>>>>>> --- a/tools/configure.ac
>>>>>>> +++ b/tools/configure.ac
>>>>>>> @@ -355,7 +355,7 @@ esac
>>>>>>>  AS_IF([echo "$PYTHON" | grep -q "^/"], [
>>>>>>>      PYTHONPATH=$PYTHON
>>>>>>>      PYTHON=`basename $PYTHONPATH`
>>>>>>> -],[test -z "$PYTHON"], [PYTHON="python"],
>>>>>>> +],[test -z "$PYTHON"], [PYTHON="python2"],
>>>>>>>  [AC_MSG_ERROR([PYTHON specified, but is not an absolute path])])
>>>>>>>  AX_PATH_PROG_OR_FAIL([PYTHONPATH], [$PYTHON])
>>>>>>>  AX_CHECK_PYTHON_VERSION([2], [3])
>>>>>>>
>>>>>>> ..but:
>>>>>>> 1) I'm not sure if we can guarantee that `python2` will exist on all
>>>>>>> systems we support,
>>>>>>> 2) We probably want to support python3 at some point anyway.
>>>>>>>
>>>>>>> On the other hand, Arch has python -> python3, but (apparently) has
>>>>>>> since 2011 [1]; so there's an argument to be made that this isn't
>>>>>>> something that needs to be solved for 4.12.
>>>>>>
>>>>>> Which raises the question whether Andrew's patch should be included
>>>>>> in 4.12. I'm tempted to refuse my Rab as I don't see why we need to
>>>>>> be python3 compliant in the build system while configure isn't.
>>>>>>
>>>>>> Thoughts?
>>>>>
>>>>> I plan to have a closer look at python 3 compatibility issue today. But
>>>>> the outcome is unlikely to be simple -- most likely there will be a few
>>>>> patches. I don't think they will be suitable for 4.12.
>>>>>
>>>>> We can document the python 2 requirement for 4.12.
>>>>
>>>> Right, but there are two issues:
>>>> 1. Build system requires python2
>>>> 2. Build system assumes that 'python' == 'python2', and fails in strange
>>>> ways when it's not
>>>>
>>>> I agree #1 isn't something to rush before the release.  Any opinions on #2?
>>>>
>>>> Options are:
>>>>
>>>> A. Simply document that if 'python'  isn't 'python2' (such as on Arch
>>>> Linux, or if you've got anaconda python installed), you need to
>>>> explicitly set PYTHON=/path/to/python2
>>>>
>>>> B. Hard-code configure to look for python2 if not specified rather than
>>>> python.
>>>>
>>>> C. Have configure detect when python is python3, and look for python2,
>>>> failing if it's not available.
>>>>
>>>> Evaluation:
>>>>
>>>> Option A isn't a great UI, but the situation has been like this for a
>>>> long time, so it's probably not the worst thing ever to wait for the
>>>> next release to fix things.  Also, serves you right for using such a
>>>> broken distro. :-)
>>>>
>>>> Option B is dead-simple; I just don't know how far back it will be
>>>> compatible.
>>>>
>>>> Option C is the most robust, but more work (though probably not a huge
>>>> amount).
>>>>
>>>> Thoughts?
>>>
>>> I'm hoping to have 4.12 finished rather soon (maybe one further RC?).
>>>
>>> For this reason I'd like to include fixes for regressions only from now
>>> on. This would imply option A.
>>
>> Works for me.
> 
> I'll send a patch to README (which mentions python) saying that we
> require python2; and saying how to build correctly if python points to
> python3.

Thanks.


Juergen


_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxxx
https://lists.xenproject.org/mailman/listinfo/xen-devel

 


Rackspace

Lists.xenproject.org is hosted with RackSpace, monitoring our
servers 24x7x365 and backed by RackSpace's Fanatical Support®.