[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [Xen-devel] [PATCH RFC-for-4.12] tools/xen-foreign: Update python scripts to be Py3 compatible


  • To: George Dunlap <george.dunlap@xxxxxxxxxx>, Wei Liu <wei.liu2@xxxxxxxxxx>
  • From: Juergen Gross <jgross@xxxxxxxx>
  • Date: Tue, 5 Mar 2019 13:30:50 +0100
  • Autocrypt: addr=jgross@xxxxxxxx; prefer-encrypt=mutual; keydata= xsBNBFOMcBYBCACgGjqjoGvbEouQZw/ToiBg9W98AlM2QHV+iNHsEs7kxWhKMjrioyspZKOB ycWxw3ie3j9uvg9EOB3aN4xiTv4qbnGiTr3oJhkB1gsb6ToJQZ8uxGq2kaV2KL9650I1SJve dYm8Of8Zd621lSmoKOwlNClALZNew72NjJLEzTalU1OdT7/i1TXkH09XSSI8mEQ/ouNcMvIJ NwQpd369y9bfIhWUiVXEK7MlRgUG6MvIj6Y3Am/BBLUVbDa4+gmzDC9ezlZkTZG2t14zWPvx XP3FAp2pkW0xqG7/377qptDmrk42GlSKN4z76ELnLxussxc7I2hx18NUcbP8+uty4bMxABEB AAHNHkp1ZXJnZW4gR3Jvc3MgPGpncm9zc0BzdXNlLmRlPsLAeQQTAQIAIwUCU4xw6wIbAwcL CQgHAwIBBhUIAgkKCwQWAgMBAh4BAheAAAoJELDendYovxMvi4UH/Ri+OXlObzqMANruTd4N zmVBAZgx1VW6jLc8JZjQuJPSsd/a+bNr3BZeLV6lu4Pf1Yl2Log129EX1KWYiFFvPbIiq5M5 kOXTO8Eas4CaScCvAZ9jCMQCgK3pFqYgirwTgfwnPtxFxO/F3ZcS8jovza5khkSKL9JGq8Nk czDTruQ/oy0WUHdUr9uwEfiD9yPFOGqp4S6cISuzBMvaAiC5YGdUGXuPZKXLpnGSjkZswUzY d9BVSitRL5ldsQCg6GhDoEAeIhUC4SQnT9SOWkoDOSFRXZ+7+WIBGLiWMd+yKDdRG5RyP/8f 3tgGiB6cyuYfPDRGsELGjUaTUq3H2xZgIPfOwE0EU4xwFgEIAMsx+gDjgzAY4H1hPVXgoLK8 B93sTQFN9oC6tsb46VpxyLPfJ3T1A6Z6MVkLoCejKTJ3K9MUsBZhxIJ0hIyvzwI6aYJsnOew cCiCN7FeKJ/oA1RSUemPGUcIJwQuZlTOiY0OcQ5PFkV5YxMUX1F/aTYXROXgTmSaw0aC1Jpo w7Ss1mg4SIP/tR88/d1+HwkJDVW1RSxC1PWzGizwRv8eauImGdpNnseneO2BNWRXTJumAWDD pYxpGSsGHXuZXTPZqOOZpsHtInFyi5KRHSFyk2Xigzvh3b9WqhbgHHHE4PUVw0I5sIQt8hJq 5nH5dPqz4ITtCL9zjiJsExHuHKN3NZsAEQEAAcLAXwQYAQIACQUCU4xwFgIbDAAKCRCw3p3W KL8TL0P4B/9YWver5uD/y/m0KScK2f3Z3mXJhME23vGBbMNlfwbr+meDMrJZ950CuWWnQ+d+ Ahe0w1X7e3wuLVODzjcReQ/v7b4JD3wwHxe+88tgB9byc0NXzlPJWBaWV01yB2/uefVKryAf AHYEd0gCRhx7eESgNBe3+YqWAQawunMlycsqKa09dBDL1PFRosF708ic9346GLHRc6Vj5SRA UTHnQqLetIOXZm3a2eQ1gpQK9MmruO86Vo93p39bS1mqnLLspVrL4rhoyhsOyh0Hd28QCzpJ wKeHTd0MAWAirmewHXWPco8p1Wg+V+5xfZzuQY0f4tQxvOpXpt4gQ1817GQ5/Ed/wsDtBBgB CAAgFiEEhRJncuj2BJSl0Jf3sN6d1ii/Ey8FAlrd8NACGwIAgQkQsN6d1ii/Ey92IAQZFggA HRYhBFMtsHpB9jjzHji4HoBcYbtP2GO+BQJa3fDQAAoJEIBcYbtP2GO+TYsA/30H/0V6cr/W V+J/FCayg6uNtm3MJLo4rE+o4sdpjjsGAQCooqffpgA+luTT13YZNV62hAnCLKXH9n3+ZAgJ RtAyDWk1B/0SMDVs1wxufMkKC3Q/1D3BYIvBlrTVKdBYXPxngcRoqV2J77lscEvkLNUGsu/z W2pf7+P3mWWlrPMJdlbax00vevyBeqtqNKjHstHatgMZ2W0CFC4hJ3YEetuRBURYPiGzuJXU pAd7a7BdsqWC4o+GTm5tnGrCyD+4gfDSpkOT53S/GNO07YkPkm/8J4OBoFfgSaCnQ1izwgJQ jIpcG2fPCI2/hxf2oqXPYbKr1v4Z1wthmoyUgGN0LPTIm+B5vdY82wI5qe9uN6UOGyTH2B3p hRQUWqCwu2sqkI3LLbTdrnyDZaixT2T0f4tyF5Lfs+Ha8xVMhIyzNb1byDI5FKCb
  • Cc: Stefano Stabellini <sstabellini@xxxxxxxxxx>, Konrad Rzeszutek Wilk <konrad.wilk@xxxxxxxxxx>, George Dunlap <George.Dunlap@xxxxxxxxxxxxx>, Andrew Cooper <andrew.cooper3@xxxxxxxxxx>, Tim Deegan <tim@xxxxxxx>, Xen-devel <xen-devel@xxxxxxxxxxxxx>, Julien Grall <julien.grall@xxxxxxx>, Jan Beulich <JBeulich@xxxxxxxx>, Ian Jackson <ian.jackson@xxxxxxxxxx>
  • Delivery-date: Tue, 05 Mar 2019 12:31:24 +0000
  • List-id: Xen developer discussion <xen-devel.lists.xenproject.org>
  • Openpgp: preference=signencrypt

On 05/03/2019 13:19, George Dunlap wrote:
> On 3/5/19 12:04 PM, Wei Liu wrote:
>> On Tue, Mar 05, 2019 at 12:41:53PM +0100, Juergen Gross wrote:
>>> On 04/03/2019 20:00, George Dunlap wrote:
>>>> On 3/4/19 6:31 PM, Andrew Cooper wrote:
>>>>> The issues are:
>>>>>  * dict.has_key() was completely removed in Py3
>>>>>  * dict.keys() is an iterable rather than list in Py3, so .sort() doesn't 
>>>>> work.
>>>>>  * list.sort(cmp=) was deprecated in Py2.4 and removed in Py3.  Replace it
>>>>>    with a key= sort instead.
>>>>>
>>>>> This is all compatible with Py2.4 and later, which is when the sorted()
>>>>> builtin was introduced.  Tested with Py2.7 and Py3.4
>>>>
>>>> Did you test ./configure with python3?
>>>>
>>>> Loads of the python-related runes in the configure are also
>>>> python2-only, resulting in (at least for me) the wrong library paths
>>>> being generated, so the following fails:
>>>>
>>>>     PYTHON=/usr/bin/python3 ./configure
>>>>
>>>> With the following message:
>>>>
>>>>   File "<string>", line 1
>>>>     import distutils.sysconfig;     print
>>>> distutils.sysconfig.get_config_var("VERSION")
>>>>                                                   ^
>>>> SyntaxError: invalid syntax
>>>> checking for python3-config... /home/gdunlap/anaconda3/bin/python3-config
>>>> checking Python.h usability... yes
>>>> checking Python.h presence... yes
>>>> checking for Python.h... yes
>>>> checking for PyArg_ParseTuple in -lpython... no
>>>> configure: error: Unable to find a suitable python development library
>>>> configure: error: ./configure failed for tools
>>>>
>>>> Note the error with the VERSION above; that results in looking for
>>>> `-lpython` rather than `-lpython3.7`, which results in the following
>>>> line being executed:
>>>>
>>>> configure:7468: checking for PyArg_ParseTuple in -lpython
>>>> configure:7493: gcc -o conftest -g -O2 -g -O2
>>>> -I/persistent/local/home/dunlapg/anaconda3/include/python3.7m
>>>> -I/persistent/local/home/dunlapg/anaconda3/include/python3.7m
>>>> -Wno-unused-result -Wsign-compare -march=nocona -mtune=haswell
>>>> -ftree-vectorize -fPIC -fstack-protector-strong -fno-plt -O3 -pipe
>>>> -fdebug-prefix-map==/usr/local/src/conda/-
>>>> -fdebug-prefix-map==/usr/local/src/conda-prefix -fuse-linker-plugin
>>>> -ffat-lto-objects -flto-partition=none -flto -flto -fuse-linker-plugin
>>>> -ffat-lto-objects -flto-partition=none -g -DNDEBUG -fwrapv -O3 -Wall
>>>> -L/persistent/local/home/dunlapg/anaconda3/lib/python3.7/config-3.7m-x86_64-linux-gnu
>>>> -L/persistent/local/home/dunlapg/anaconda3/lib -lpython3.7m -lpthread
>>>> -ldl  -lutil -lrt -lm  -Xlinker -export-dynamic conftest.c -lpython
>>>> -llzma  -lpython3.7m -lpthread -ldl  -lutil -lrt -lm  -llzma  >&5
>>>> /usr/bin/ld: cannot find -lpython
>>>> collect2: error: ld returned 1 exit status
>>>>
>>>> The following fixes it for me:
>>>>
>>>> diff --git a/tools/configure b/tools/configure
>>>> index acc857510e..d7e21db959 100755
>>>> --- a/tools/configure
>>>> +++ b/tools/configure
>>>> @@ -6953,7 +6953,7 @@ if echo "$PYTHON" | grep -q "^/"; then :
>>>>      PYTHON=`basename $PYTHONPATH`
>>>>
>>>>  elif test -z "$PYTHON"; then :
>>>> -  PYTHON="python"
>>>> +  PYTHON="python2"
>>>>  else
>>>>    as_fn_error $? "PYTHON specified, but is not an absolute path"
>>>> "$LINENO" 5
>>>>  fi
>>>> diff --git a/tools/configure.ac b/tools/configure.ac
>>>> index 1499344ce6..adf70aeb33 100644
>>>> --- a/tools/configure.ac
>>>> +++ b/tools/configure.ac
>>>> @@ -355,7 +355,7 @@ esac
>>>>  AS_IF([echo "$PYTHON" | grep -q "^/"], [
>>>>      PYTHONPATH=$PYTHON
>>>>      PYTHON=`basename $PYTHONPATH`
>>>> -],[test -z "$PYTHON"], [PYTHON="python"],
>>>> +],[test -z "$PYTHON"], [PYTHON="python2"],
>>>>  [AC_MSG_ERROR([PYTHON specified, but is not an absolute path])])
>>>>  AX_PATH_PROG_OR_FAIL([PYTHONPATH], [$PYTHON])
>>>>  AX_CHECK_PYTHON_VERSION([2], [3])
>>>>
>>>> ..but:
>>>> 1) I'm not sure if we can guarantee that `python2` will exist on all
>>>> systems we support,
>>>> 2) We probably want to support python3 at some point anyway.
>>>>
>>>> On the other hand, Arch has python -> python3, but (apparently) has
>>>> since 2011 [1]; so there's an argument to be made that this isn't
>>>> something that needs to be solved for 4.12.
>>>
>>> Which raises the question whether Andrew's patch should be included
>>> in 4.12. I'm tempted to refuse my Rab as I don't see why we need to
>>> be python3 compliant in the build system while configure isn't.
>>>
>>> Thoughts?
>>
>> I plan to have a closer look at python 3 compatibility issue today. But
>> the outcome is unlikely to be simple -- most likely there will be a few
>> patches. I don't think they will be suitable for 4.12.
>>
>> We can document the python 2 requirement for 4.12.
> 
> Right, but there are two issues:
> 1. Build system requires python2
> 2. Build system assumes that 'python' == 'python2', and fails in strange
> ways when it's not
> 
> I agree #1 isn't something to rush before the release.  Any opinions on #2?
> 
> Options are:
> 
> A. Simply document that if 'python'  isn't 'python2' (such as on Arch
> Linux, or if you've got anaconda python installed), you need to
> explicitly set PYTHON=/path/to/python2
> 
> B. Hard-code configure to look for python2 if not specified rather than
> python.
> 
> C. Have configure detect when python is python3, and look for python2,
> failing if it's not available.
> 
> Evaluation:
> 
> Option A isn't a great UI, but the situation has been like this for a
> long time, so it's probably not the worst thing ever to wait for the
> next release to fix things.  Also, serves you right for using such a
> broken distro. :-)
> 
> Option B is dead-simple; I just don't know how far back it will be
> compatible.
> 
> Option C is the most robust, but more work (though probably not a huge
> amount).
> 
> Thoughts?

I'm hoping to have 4.12 finished rather soon (maybe one further RC?).

For this reason I'd like to include fixes for regressions only from now
on. This would imply option A.


Juergen

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxxx
https://lists.xenproject.org/mailman/listinfo/xen-devel

 


Rackspace

Lists.xenproject.org is hosted with RackSpace, monitoring our
servers 24x7x365 and backed by RackSpace's Fanatical Support®.