[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [Xen-devel] [PATCH RFC-for-4.12] tools/xen-foreign: Update python scripts to be Py3 compatible


  • To: Juergen Gross <jgross@xxxxxxxx>, Wei Liu <wei.liu2@xxxxxxxxxx>
  • From: George Dunlap <george.dunlap@xxxxxxxxxx>
  • Date: Tue, 5 Mar 2019 12:33:36 +0000
  • Autocrypt: addr=george.dunlap@xxxxxxxxxx; prefer-encrypt=mutual; keydata= mQINBFPqG+MBEACwPYTQpHepyshcufo0dVmqxDo917iWPslB8lauFxVf4WZtGvQSsKStHJSj 92Qkxp4CH2DwudI8qpVbnWCXsZxodDWac9c3PordLwz5/XL41LevEoM3NWRm5TNgJ3ckPA+J K5OfSK04QtmwSHFP3G/SXDJpGs+oDJgASta2AOl9vPV+t3xG6xyfa2NMGn9wmEvvVMD44Z7R W3RhZPn/NEZ5gaJhIUMgTChGwwWDOX0YPY19vcy5fT4bTIxvoZsLOkLSGoZb/jHIzkAAznug Q7PPeZJ1kXpbW9EHHaUHiCD9C87dMyty0N3TmWfp0VvBCaw32yFtM9jUgB7UVneoZUMUKeHA fgIXhJ7I7JFmw3J0PjGLxCLHf2Q5JOD8jeEXpdxugqF7B/fWYYmyIgwKutiGZeoPhl9c/7RE Bf6f9Qv4AtQoJwtLw6+5pDXsTD5q/GwhPjt7ohF7aQZTMMHhZuS52/izKhDzIufl6uiqUBge 0lqG+/ViLKwCkxHDREuSUTtfjRc9/AoAt2V2HOfgKORSCjFC1eI0+8UMxlfdq2z1AAchinU0 eSkRpX2An3CPEjgGFmu2Je4a/R/Kd6nGU8AFaE8ta0oq5BSFDRYdcKchw4TSxetkG6iUtqOO ZFS7VAdF00eqFJNQpi6IUQryhnrOByw+zSobqlOPUO7XC5fjnwARAQABtCRHZW9yZ2UgVy4g RHVubGFwIDxkdW5sYXBnQHVtaWNoLmVkdT6JAlcEEwEKAEECGwMFCwkIBwMFFQoJCAsFFgID AQACHgECF4ACGQEWIQTXqBy2bTNXPzpOYFimNjwxBZC0bQUCXEowWQUJDCJ7dgAKCRCmNjwx BZC0beKvEACJ75YlJXd7TnNHgFyiCJkm/qPeoQ3sFGSDZuZh7SKcdt9+3V2bFEb0Mii1hQaz 3hRqZb8sYPHJrGP0ljK09k3wf8k3OuNxziLQBJyzvn7WNlE4wBEcy/Ejo9TVBdA4ph5D0YaZ nqdsPmxe/xlTFuSkgu4ep1v9dfVP1TQR0e+JIBa/Ss+cKC5intKm+8JxpOploAHuzaPu0L/X FapzsIXqgT9eIQeBEgO2hge6h9Jov3WeED/vh8kA7f8c6zQ/gs5E7VGALwsiLrhr0LZFcKcw kI3oCCrB/C/wyPZv789Ra8EXbeRSJmTjcnBwHRPjnjwQmetRDD1t+VyrkC6uujT5jmgOBzaj KCqZ8PcMAssOzdzQtKmjUQ2b3ICPs2X13xZ5M5/OVs1W3TG5gkvMh4YoHi4ilFnOk+v3/j7q 65FG6N0JLb94Ndi80HkIOQQ1XVGTyu6bUPaBg3rWK91Csp1682kD/dNVF3FKHrRLmSVtmEQR 5rK0+VGc/FmR6vd4haKGWIRuPxzg+pBR77avIZpU7C7+UXGuZ5CbHwIdY8LojJg2TuUdqaVj yxmEZLOA8rVHipCGrslRNthVbJrGN/pqtKjCClFZHIAYJQ9EGLHXLG9Pj76opfjHij3MpR3o pCGAh6KsCrfrsvjnpDwqSbngGyEVH030irSk4SwIqZ7FwLkBDQRUWmc6AQgAzpc8Ng5Opbrh iZrn69Xr3js28p+b4a+0BOvC48NfrNovZw4eFeKIzmI/t6EkJkSqBIxobWRpBkwGweENsqnd 0qigmsDw4N7J9Xx0h9ARDqiWxX4jr7u9xauI+CRJ1rBNO3VV30QdACwQ4LqhR/WA+IjdhyMH wj3EJGE61NdP/h0zfaLYAbvEg47/TPThFsm4m8Rd6bX7RkrrOgBbL/AOnYOMEivyfZZKX1vv iEemAvLfdk2lZt7Vm6X/fbKbV8tPUuZELzNedJvTTBS3/l1FVz9OUcLDeWhGEdlxqXH0sYWh E9+PXTAfz5JxKH+LMetwEM8DbuOoDIpmIGZKrZ+2fQARAQABiQNbBBgBCgAmAhsCFiEE16gc tm0zVz86TmBYpjY8MQWQtG0FAlxKMJ4FCQnQ/OQBKcBdIAQZAQoABgUCVFpnOgAKCRCyFcen x4Qb7cXrCAC0qQeEWmLa9oEAPa+5U6wvG1t/mi22gZN6uzQXH1faIOoDehr7PPESE6tuR/vI CTTnaSrd4UDPNeqOqVF07YexWD1LDcQG6PnRqC5DIX1RGE3BaSaMl2pFJP8y+chews11yP8G DBbxaIsTcHZI1iVIC9XLhoeegWi84vYc8F4ziADVfowbmbvcVw11gE8tmALCwTeBeZVteXjh 0OELHwrc1/4j4yvENjIXRO+QLIgk43kB57Upr4tP2MEcs0odgPM+Q+oETOJ00xzLgkTnLPim C1FIW2bOZdTj+Uq6ezRS2LKsNmW+PRRvNyA5ojEbA/faxmAjMZtLdSSSeFK8y4SoCRCmNjwx BZC0bevWEACRu+GyQgrdGmorUptniIeO1jQlpTiP5WpVnk9Oe8SiLoXUhXXNj6EtzyLGpYmf kEAbki+S6WAKnzZd3shL58AuMyDxtFNNjNeKJOcl6FL7JPBIIgIp3wR401Ep+/s5pl3Nw8Ii 157f0T7o8CPb54w6S1WsMkU78WzTxIs/1lLblSMcvyz1Jq64g4OqiWI85JfkzPLlloVf1rzy ebIBLrrmjhCE2tL1RONpE/KRVb+Q+PIs5+YcZ+Q1e0vXWA7NhTWFbWx3+N6WW6gaGpbFbopo FkYRpj+2TA5cX5zW148/xU5/ATEb5vdUkFLUFVy5YNUSyeBHuaf6fGmBrDc47rQjAOt1rmyD 56MUBHpLUbvA6NkPezb7T6bQpupyzGRkMUmSwHiLyQNJQhVe+9NiJJvtEE3jol0JVJoQ9WVn FAzPNCgHQyvbsIF3gYkCYKI0w8EhEoH5FHYLoKS6Jg880IY5rXzoAEfPvLXegy6mhYl+mNVN QUBD4h9XtOvcdzR559lZuC0Ksy7Xqw3BMolmKsRO3gWKhXSna3zKl4UuheyZtubVWoNWP/bn vbyiYnLwuiKDfNAinEWERC8nPKlv3PkZw5d3t46F1Dx0TMf16NmP+azsRpnMZyzpY8BL2eur feSGAOB9qjZNyzbo5nEKHldKWCKE7Ye0EPEjECS1gjKDwbkBDQRUWrq9AQgA7aJ0i1pQSmUR 6ZXZD2YEDxia2ByR0uZoTS7N0NYv1OjU8v6p017u0Fco5+Qoju/fZ97ScHhp5xGVAk5kxZBF DT4ovJd0nIeSr3bbWwfNzGx1waztfdzXt6n3MBKr7AhioB1m+vuk31redUdnhbtvN7O40MC+ fgSk5/+jRGxY3IOVPooQKzUO7M51GoOg4wl9ia3H2EzOoGhN2vpTbT8qCcL92ZZZwkBRldoA Wn7c1hEKSTuT3f1VpSmhjnX0J4uvKZ1V2R7rooKJYFBcySC0wa8aTmAtAvLgfcpe+legOtgq DKzLuN45xzEjyjCiI521t8zxNMPJY9FiCPNv0sCkDwARAQABiQI8BBgBCgAmAhsMFiEE16gc tm0zVz86TmBYpjY8MQWQtG0FAlxKNJYFCQnQrVkACgkQpjY8MQWQtG2Xxg//RrRP+PFYuNXt 9C5hec/JoY24TkGPPd2tMC9usWZVImIk7VlHlAeqHeE0lWU0LRGIvOBITbS9izw6fOVQBvCA Fni56S12fKLusWgWhgu03toT9ZGxZ9W22yfw5uThSHQ4y09wRWAIYvhJsKnPGGC2KDxFvtz5 4pYYNe8Icy4bwsxcgbaSFaRh+mYtts6wE9VzyJvyfTqbe8VrvE+3InG5rrlNn51AO6M4Wv20 iFEgYanJXfhicl0WCQrHyTLfdB5p1w+072CL8uryHQVfD0FcDe+J/wl3bmYze+aD1SlPzFoI MaSIXKejC6oh6DAT4rvU8kMAbX90T834Mvbc3jplaWorNJEwjAH/r+v877AI9Vsmptis+rni JwUissjRbcdlkKBisoUZRPmxQeUifxUpqgulZcYwbEC/a49+WvbaYUriaDLHzg9xisijHwD2 yWV8igBeg+cmwnk0mPz8tIVvwi4lICAgXob7HZiaqKnwaDXs4LiS4vdG5s/ElnE3rIc87yru 24n3ypeDZ6f5LkdqL1UNp5/0Aqbr3EiN7/ina4YVyscy9754l944kyHnnMRLVykg0v+kakj0 h0RJ5LbfLAMM8M52KIA3y14g0Fb7kHLcOUMVcgfQ3PrN6chtC+5l6ouDIlSLR3toxH8Aam7E rIFfe2Dk+lD9A9BVd2rfoHA=
  • Cc: Stefano Stabellini <sstabellini@xxxxxxxxxx>, Konrad Rzeszutek Wilk <konrad.wilk@xxxxxxxxxx>, George Dunlap <George.Dunlap@xxxxxxxxxxxxx>, Andrew Cooper <andrew.cooper3@xxxxxxxxxx>, Tim Deegan <tim@xxxxxxx>, Xen-devel <xen-devel@xxxxxxxxxxxxx>, Julien Grall <julien.grall@xxxxxxx>, Jan Beulich <JBeulich@xxxxxxxx>, Ian Jackson <ian.jackson@xxxxxxxxxx>
  • Delivery-date: Tue, 05 Mar 2019 12:33:51 +0000
  • List-id: Xen developer discussion <xen-devel.lists.xenproject.org>
  • Openpgp: preference=signencrypt

On 3/5/19 12:31 PM, George Dunlap wrote:
> On 3/5/19 12:30 PM, Juergen Gross wrote:
>> On 05/03/2019 13:19, George Dunlap wrote:
>>> On 3/5/19 12:04 PM, Wei Liu wrote:
>>>> On Tue, Mar 05, 2019 at 12:41:53PM +0100, Juergen Gross wrote:
>>>>> On 04/03/2019 20:00, George Dunlap wrote:
>>>>>> On 3/4/19 6:31 PM, Andrew Cooper wrote:
>>>>>>> The issues are:
>>>>>>>  * dict.has_key() was completely removed in Py3
>>>>>>>  * dict.keys() is an iterable rather than list in Py3, so .sort() 
>>>>>>> doesn't work.
>>>>>>>  * list.sort(cmp=) was deprecated in Py2.4 and removed in Py3.  Replace 
>>>>>>> it
>>>>>>>    with a key= sort instead.
>>>>>>>
>>>>>>> This is all compatible with Py2.4 and later, which is when the sorted()
>>>>>>> builtin was introduced.  Tested with Py2.7 and Py3.4
>>>>>>
>>>>>> Did you test ./configure with python3?
>>>>>>
>>>>>> Loads of the python-related runes in the configure are also
>>>>>> python2-only, resulting in (at least for me) the wrong library paths
>>>>>> being generated, so the following fails:
>>>>>>
>>>>>>     PYTHON=/usr/bin/python3 ./configure
>>>>>>
>>>>>> With the following message:
>>>>>>
>>>>>>   File "<string>", line 1
>>>>>>     import distutils.sysconfig;     print
>>>>>> distutils.sysconfig.get_config_var("VERSION")
>>>>>>                                                   ^
>>>>>> SyntaxError: invalid syntax
>>>>>> checking for python3-config... /home/gdunlap/anaconda3/bin/python3-config
>>>>>> checking Python.h usability... yes
>>>>>> checking Python.h presence... yes
>>>>>> checking for Python.h... yes
>>>>>> checking for PyArg_ParseTuple in -lpython... no
>>>>>> configure: error: Unable to find a suitable python development library
>>>>>> configure: error: ./configure failed for tools
>>>>>>
>>>>>> Note the error with the VERSION above; that results in looking for
>>>>>> `-lpython` rather than `-lpython3.7`, which results in the following
>>>>>> line being executed:
>>>>>>
>>>>>> configure:7468: checking for PyArg_ParseTuple in -lpython
>>>>>> configure:7493: gcc -o conftest -g -O2 -g -O2
>>>>>> -I/persistent/local/home/dunlapg/anaconda3/include/python3.7m
>>>>>> -I/persistent/local/home/dunlapg/anaconda3/include/python3.7m
>>>>>> -Wno-unused-result -Wsign-compare -march=nocona -mtune=haswell
>>>>>> -ftree-vectorize -fPIC -fstack-protector-strong -fno-plt -O3 -pipe
>>>>>> -fdebug-prefix-map==/usr/local/src/conda/-
>>>>>> -fdebug-prefix-map==/usr/local/src/conda-prefix -fuse-linker-plugin
>>>>>> -ffat-lto-objects -flto-partition=none -flto -flto -fuse-linker-plugin
>>>>>> -ffat-lto-objects -flto-partition=none -g -DNDEBUG -fwrapv -O3 -Wall
>>>>>> -L/persistent/local/home/dunlapg/anaconda3/lib/python3.7/config-3.7m-x86_64-linux-gnu
>>>>>> -L/persistent/local/home/dunlapg/anaconda3/lib -lpython3.7m -lpthread
>>>>>> -ldl  -lutil -lrt -lm  -Xlinker -export-dynamic conftest.c -lpython
>>>>>> -llzma  -lpython3.7m -lpthread -ldl  -lutil -lrt -lm  -llzma  >&5
>>>>>> /usr/bin/ld: cannot find -lpython
>>>>>> collect2: error: ld returned 1 exit status
>>>>>>
>>>>>> The following fixes it for me:
>>>>>>
>>>>>> diff --git a/tools/configure b/tools/configure
>>>>>> index acc857510e..d7e21db959 100755
>>>>>> --- a/tools/configure
>>>>>> +++ b/tools/configure
>>>>>> @@ -6953,7 +6953,7 @@ if echo "$PYTHON" | grep -q "^/"; then :
>>>>>>      PYTHON=`basename $PYTHONPATH`
>>>>>>
>>>>>>  elif test -z "$PYTHON"; then :
>>>>>> -  PYTHON="python"
>>>>>> +  PYTHON="python2"
>>>>>>  else
>>>>>>    as_fn_error $? "PYTHON specified, but is not an absolute path"
>>>>>> "$LINENO" 5
>>>>>>  fi
>>>>>> diff --git a/tools/configure.ac b/tools/configure.ac
>>>>>> index 1499344ce6..adf70aeb33 100644
>>>>>> --- a/tools/configure.ac
>>>>>> +++ b/tools/configure.ac
>>>>>> @@ -355,7 +355,7 @@ esac
>>>>>>  AS_IF([echo "$PYTHON" | grep -q "^/"], [
>>>>>>      PYTHONPATH=$PYTHON
>>>>>>      PYTHON=`basename $PYTHONPATH`
>>>>>> -],[test -z "$PYTHON"], [PYTHON="python"],
>>>>>> +],[test -z "$PYTHON"], [PYTHON="python2"],
>>>>>>  [AC_MSG_ERROR([PYTHON specified, but is not an absolute path])])
>>>>>>  AX_PATH_PROG_OR_FAIL([PYTHONPATH], [$PYTHON])
>>>>>>  AX_CHECK_PYTHON_VERSION([2], [3])
>>>>>>
>>>>>> ..but:
>>>>>> 1) I'm not sure if we can guarantee that `python2` will exist on all
>>>>>> systems we support,
>>>>>> 2) We probably want to support python3 at some point anyway.
>>>>>>
>>>>>> On the other hand, Arch has python -> python3, but (apparently) has
>>>>>> since 2011 [1]; so there's an argument to be made that this isn't
>>>>>> something that needs to be solved for 4.12.
>>>>>
>>>>> Which raises the question whether Andrew's patch should be included
>>>>> in 4.12. I'm tempted to refuse my Rab as I don't see why we need to
>>>>> be python3 compliant in the build system while configure isn't.
>>>>>
>>>>> Thoughts?
>>>>
>>>> I plan to have a closer look at python 3 compatibility issue today. But
>>>> the outcome is unlikely to be simple -- most likely there will be a few
>>>> patches. I don't think they will be suitable for 4.12.
>>>>
>>>> We can document the python 2 requirement for 4.12.
>>>
>>> Right, but there are two issues:
>>> 1. Build system requires python2
>>> 2. Build system assumes that 'python' == 'python2', and fails in strange
>>> ways when it's not
>>>
>>> I agree #1 isn't something to rush before the release.  Any opinions on #2?
>>>
>>> Options are:
>>>
>>> A. Simply document that if 'python'  isn't 'python2' (such as on Arch
>>> Linux, or if you've got anaconda python installed), you need to
>>> explicitly set PYTHON=/path/to/python2
>>>
>>> B. Hard-code configure to look for python2 if not specified rather than
>>> python.
>>>
>>> C. Have configure detect when python is python3, and look for python2,
>>> failing if it's not available.
>>>
>>> Evaluation:
>>>
>>> Option A isn't a great UI, but the situation has been like this for a
>>> long time, so it's probably not the worst thing ever to wait for the
>>> next release to fix things.  Also, serves you right for using such a
>>> broken distro. :-)
>>>
>>> Option B is dead-simple; I just don't know how far back it will be
>>> compatible.
>>>
>>> Option C is the most robust, but more work (though probably not a huge
>>> amount).
>>>
>>> Thoughts?
>>
>> I'm hoping to have 4.12 finished rather soon (maybe one further RC?).
>>
>> For this reason I'd like to include fixes for regressions only from now
>> on. This would imply option A.
> 
> Works for me.

I'll send a patch to README (which mentions python) saying that we
require python2; and saying how to build correctly if python points to
python3.

 -George

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxxx
https://lists.xenproject.org/mailman/listinfo/xen-devel

 


Rackspace

Lists.xenproject.org is hosted with RackSpace, monitoring our
servers 24x7x365 and backed by RackSpace's Fanatical Support®.