installing vsc-base v2.4.2 fails with Python 2.7.10 due to known bug in distutils

[boegel]

== 2015-10-13 19:58:05,611 main ERROR cmd " python setup.py easy_install --no-deps --zip-ok --prefix=/user/scratch/gent/vsc400/vsc40023/easybuild_REGTEST/SL6/sandybridge/software/vsc-base/2.4.2-intel-2015b-Python-2.7.10  ." exited with exitcode 1 and output:
New target = {'maintainer': 'Stijn De Weirdt;Jens Timmermans;Andy Georges;Kenneth Hoste', 'version': '2.4.2', 'name': 'vsc-base', 'author': 'Stijn De Weirdt;Jens Timmermans;Andy Georges;Kenneth Hoste', 'url': 'https://github.com/hpcugent/vsc-base', 'zip_safe': 1, 'download_url': 'https://github.com/hpcugent/vsc-base/tarball/master', 'maintainer_email': '[email protected];[email protected];[email protected];[email protected]', 'cmdclass': {'test': <class vsc.install.shared_setup.VscTestCommand at 0x2ac30c4d7258>}, 'test_suite': 'test', 'scripts': ['bin/logdaemon.py', 'bin/startlogdaemon.sh', 'bin/bdist_rpm.sh', 'bin/optcomplete.bash'], 'author_email': '[email protected];[email protected];[email protected];[email protected]', 'install_requires': ['setuptools'], 'packages': ['vsc', 'vsc.install', 'vsc.utils'], 'package_dir': {'': 'lib'}}
running easy_install
Creating /user/scratchdelcatty/gent/gvo000/gvo00002/vsc40023/easybuild_REGTEST/SL6/sandybridge/software/vsc-base/2.4.2-intel-2015b-Python-2.7.10/lib/python2.7/site-packages/site.py
Processing .
Writing /tmp/vsc40023/easybuild_build/vscbase/2.4.2/intel-2015b-Python-2.7.10/vsc-base-2.4.2/setup.cfg
Running setup.py -q bdist_egg --dist-dir /tmp/vsc40023/easybuild_build/vscbase/2.4.2/intel-2015b-Python-2.7.10/vsc-base-2.4.2/egg-dist-tmp-Ogj7tm
Traceback (most recent call last):
  File "setup.py", line 74, in <module>
    shared_setup.action_target(PACKAGE, urltemplate=URL_GH_HPCUGENT)
  File "/tmp/vsc40023/easybuild_build/vscbase/2.4.2/intel-2015b-Python-2.7.10/vsc-base-2.4.2/lib/vsc/install/shared_setup.py", line 507, in action_target
    setupfn(**x)
  File "/user/scratch/gent/vsc400/vsc40023/easybuild_REGTEST/SL6/sandybridge/software/Python/2.7.10-intel-2015b/lib/python2.7/distutils/core.py", line 151, in setup
    dist.run_commands()
  File "/user/scratch/gent/vsc400/vsc40023/easybuild_REGTEST/SL6/sandybridge/software/Python/2.7.10-intel-2015b/lib/python2.7/distutils/dist.py", line 955, in run_commands
    self.run_command(cmd)
  File "/user/scratch/gent/vsc400/vsc40023/easybuild_REGTEST/SL6/sandybridge/software/Python/2.7.10-intel-2015b/lib/python2.7/distutils/dist.py", line 974, in run_command
    cmd_obj.run()
  File "build/bdist.linux-x86_64/egg/setuptools/command/easy_install.py", line 389, in run

  File "build/bdist.linux-x86_64/egg/setuptools/command/easy_install.py", line 619, in easy_install

  File "build/bdist.linux-x86_64/egg/setuptools/command/easy_install.py", line 668, in install_item

  File "build/bdist.linux-x86_64/egg/setuptools/command/easy_install.py", line 851, in install_eggs

  File "build/bdist.linux-x86_64/egg/setuptools/command/easy_install.py", line 1079, in build_and_install

  File "build/bdist.linux-x86_64/egg/setuptools/command/easy_install.py", line 1065, in run_setup

  File "build/bdist.linux-x86_64/egg/setuptools/sandbox.py", line 240, in run_setup
  File "/user/scratch/gent/vsc400/vsc40023/easybuild_REGTEST/SL6/sandybridge/software/Python/2.7.10-intel-2015b/lib/python2.7/contextlib.py", line 35, in __exit__
    self.gen.throw(type, value, traceback)
  File "build/bdist.linux-x86_64/egg/setuptools/sandbox.py", line 193, in setup_context
  File "/user/scratch/gent/vsc400/vsc40023/easybuild_REGTEST/SL6/sandybridge/software/Python/2.7.10-intel-2015b/lib/python2.7/contextlib.py", line 35, in __exit__
    self.gen.throw(type, value, traceback)
  File "build/bdist.linux-x86_64/egg/setuptools/sandbox.py", line 164, in save_modules
  File "build/bdist.linux-x86_64/egg/setuptools/sandbox.py", line 139, in resume
  File "build/bdist.linux-x86_64/egg/setuptools/sandbox.py", line 152, in save_modules
  File "build/bdist.linux-x86_64/egg/setuptools/sandbox.py", line 193, in setup_context
  File "build/bdist.linux-x86_64/egg/setuptools/sandbox.py", line 237, in run_setup
  File "build/bdist.linux-x86_64/egg/setuptools/sandbox.py", line 267, in run
  File "build/bdist.linux-x86_64/egg/setuptools/sandbox.py", line 236, in runner
  File "build/bdist.linux-x86_64/egg/setuptools/sandbox.py", line 46, in _execfile
  File "/tmp/vsc40023/easybuild_build/vscbase/2.4.2/intel-2015b-Python-2.7.10/vsc-base-2.4.2/setup.py", line 74, in <module>
    shared_setup.action_target(PACKAGE, urltemplate=URL_GH_HPCUGENT)
  File "/tmp/vsc40023/easybuild_build/vscbase/2.4.2/intel-2015b-Python-2.7.10/vsc-base-2.4.2/lib/vsc/install/shared_setup.py", line 507, in action_target
    setupfn(**x)
  File "/user/scratch/gent/vsc400/vsc40023/easybuild_REGTEST/SL6/sandybridge/software/Python/2.7.10-intel-2015b/lib/python2.7/distutils/core.py", line 151, in setup
    dist.run_commands()
  File "/user/scratch/gent/vsc400/vsc40023/easybuild_REGTEST/SL6/sandybridge/software/Python/2.7.10-intel-2015b/lib/python2.7/distutils/dist.py", line 955, in run_commands
    self.run_command(cmd)
  File "/user/scratch/gent/vsc400/vsc40023/easybuild_REGTEST/SL6/sandybridge/software/Python/2.7.10-intel-2015b/lib/python2.7/distutils/dist.py", line 972, in run_command
    cmd_obj = self.get_command_obj(command)
  File "/user/scratch/gent/vsc400/vsc40023/easybuild_REGTEST/SL6/sandybridge/software/Python/2.7.10-intel-2015b/lib/python2.7/distutils/dist.py", line 848, in get_command_obj
    cmd_obj = self.command_obj[command] = klass(self)
  File "build/bdist.linux-x86_64/egg/setuptools/__init__.py", line 131, in __init__
  File "/user/scratch/gent/vsc400/vsc40023/easybuild_REGTEST/SL6/sandybridge/software/Python/2.7.10-intel-2015b/lib/python2.7/distutils/cmd.py", line 59, in __init__
    raise TypeError, "dist must be a Distribution instance"
TypeError: dist must be a Distribution instance

[boegel]

This is a known issue/bug in Python, but was only patched for Python 3.x, see https://bugs.python.org/issue23114 .

Supposedly, using from setuptools.dist import Distribution very early to trigger the monkeypatching done by setuptools should be a workaround, but it doesn't seem to work; I tried including it in both setup.py and vsc/install/shared_setup.py as the first import, but that doesn't fix the problem for reasons unclear to me...

[boegel]

I can only get it to work by patching the Python installation I'm using with this patch https://bugs.python.org/file37554/distutils_accomodate_distribution_ducktypes.patch (obtained via https://bugs.python.org/issue23102).

Can't seem to get it to work by patching setup.py or vsc/install/shared_setup.py in vsc-base... :(

[stdweird]

is this 2.7.10 only? and how can i reproduce this (so i can debug it)?

[boegel]

@wpoely86 tried reproducing it, but couldn't for some reason unclear to me.

It's not specific to Python 2.7.10 afaik, but to using a recent setuptools (one that does the monkey patching of Distribution in distutils.dist) to install vsc-base with.

Don't spend time on this for now, I've already spent some time on this and seem to be closing in on a fix, using an early import of setuptools.dist (which is the only workaround for this without patching Python itself).

[boegel]

This only occurs with python setup.py easy_install --prefix, not with easy_install --prefix >_<