Bug #4222

gammalib 2.0 crashes when the test suite is called

Added by Peresano Michele about 1 year ago. Updated about 1 year ago.

Status:ClosedStart date:03/13/2023
Priority:HighDue date:
Assigned To:Knödlseder Jürgen% Done:

100%

Category:-
Target version:2.1.0
Tags: BUG
Duration:

Description

I installed gammalib from conda (v2.0.0 is the last version available from this channel).

Following the documentation, I started by checking if everything works with
the test() function and the program crashed immediately.

(gammalib) [michele@d-h-122 ~]$ ipython
Python 3.10.9 | packaged by conda-forge | (main, Feb  2 2023, 20:24:27) [Clang 14.0.6 ]
Type 'copyright', 'credits' or 'license' for more information
IPython 8.11.0 -- An enhanced Interactive Python. Type '?' for help.

In [1]: import gammalib

In [2]: gammalib.test()
---------------------------------------------------------------------------
ModuleNotFoundError                       Traceback (most recent call last)
Cell In[2], line 1
----> 1 gammalib.test()

File ~/Applications/mambaforge/envs/gammalib/lib/python3.10/site-packages/gammalib-2.0.0-py3.10-macosx-10.9-x86_64.egg/gammalib/__init__.py:67, in test()
     66 def test():
---> 67     from gammalib.tests import test_python
     68     test_python.test(installed=True)

ModuleNotFoundError: No module named 'gammalib.tests'

There is indeed no test*s* module, just test.
First I tried naïvely to fix the typo, but there is also no function named test_python in that module,

ImportError: cannot import name 'test_python' from 'gammalib.test' (/Users/michele/Applications/mambaforge/envs/gammalib/lib/python3.10/site-packages/gammalib-2.0.0-py3.10-macosx-10.9-x86_64.egg/gammalib/test.py)

Recurrence

No recurrence.

Related issues

Related to GammaLib - Change request #4223: Replace distutils by setuptools Closed 03/14/2023

History

#1 Updated by Knödlseder Jürgen about 1 year ago

  • Status changed from New to In Progress
  • Assigned To set to Knödlseder Jürgen
  • % Done changed from 0 to 10

On what OS have you tried the installation?

And what do you see when you type:

ls ~/Applications/mambaforge/envs/gammalib/lib/python3.10/site-packages/gammalib-2.0.0-py3.10-macosx-10.9-x86_64.egg/gammalib/

#2 Updated by Peresano Michele about 1 year ago

On what OS have you tried the installation?

macos Big Sur 11.7.4

ls ~/Applications/mambaforge/envs/gammalib/lib/python3.10/site-packages/gammalib-2.0.0-py3.10-macosx-10.9-x86_64.egg/gammalib/

__init__.py                     _linalg.py                      _support.cpython-310-darwin.so  lat.py
__pycache__                     _model.cpython-310-darwin.so    _support.py                     linalg.py
_app.cpython-310-darwin.so      _model.py                       _test.cpython-310-darwin.so     model.py
_app.py                         _mwl.cpython-310-darwin.so      _test.py                        mwl.py
_base.cpython-310-darwin.so     _mwl.py                         _vo.cpython-310-darwin.so       numerics.py
_base.py                        _numerics.cpython-310-darwin.so _vo.py                          obs.py
_com.cpython-310-darwin.so      _numerics.py                    _xml.cpython-310-darwin.so      opt.py
_com.py                         _obs.cpython-310-darwin.so      _xml.py                         sky.py
_cta.cpython-310-darwin.so      _obs.py                         _xspec.cpython-310-darwin.so    spi.py
_cta.py                         _opt.cpython-310-darwin.so      _xspec.py                       support.py
_fits.cpython-310-darwin.so     _opt.py                         app.py                          test.py
_fits.py                        _sky.cpython-310-darwin.so      base.py                         vo.py
_lat.cpython-310-darwin.so      _sky.py                         com.py                          xml.py
_lat.py                         _spi.cpython-310-darwin.so      cta.py                          xspec.py
_linalg.cpython-310-darwin.so   _spi.py                         fits.py

#3 Updated by Knödlseder Jürgen about 1 year ago

  • % Done changed from 10 to 20

Okay, there is a tests folder missing, but everything else looks ok.

Have you tried stuff from http://cta.irap.omp.eu/gammalib/users/quickstart.html, skipping the gammalib.test() command?

Maybe the installation is correct but something went wrong with the tests installation. I need to see whether I manage to install a macos Big Sur 11.7.4 VM for testing.

#4 Updated by Peresano Michele about 1 year ago

Yes the rest of the quickstart works as expected (by the way, the link to the Doxygen documentation points to a non-existent HTML page)

#5 Updated by Knödlseder Jürgen about 1 year ago

Thanks for feedback. I will see if I can reproduce and fix the problem.

Thank also for mentioning the broken Doxygen link, I will fixed it.

#6 Updated by Knödlseder Jürgen about 1 year ago

  • Target version set to 2.1.0
  • % Done changed from 20 to 30

In the meantime I tested the conda version for Python 3.10 on my Mac to see whether the problem is related to this specific Python version, and this seems to be the case!

After creating a Python 3.10 environment and installing gammalib I see the following in the site-package folder:

$ ls /Users/jurgen/anaconda3/envs/gammalib-test-3.10/lib/python3.10/site-packages/
README.txt                    pip-23.0.1-py3.11.egg-info
_distutils_hack                    pkg_resources
distutils-precedence.pth            setuptools
gammalib                    setuptools-67.6.0-py3.11.egg-info
gammalib-2.0.0-py3.10-macosx-10.9-x86_64.egg    wheel
gammalib.pth                    wheel-0.38.4-py3.11.egg-info
pip
Here there is a gammalib folder and a gammalib-2.0.0-py3.10-macosx-10.9-x86_64.egg folder. The tests directory ends up in the gammalib folder, the remaining stuff is in the gammalib-2.0.0-py3.10-macosx-10.9-x86_64.egg folder.

For Python 3.9 the organisation of the site-packages folder is different:

$ ls /Users/jurgen/anaconda3/envs/gammalib-test-3.9/lib/python3.9/site-packages/
README.txt                        gammalib-2.0.0-py3.9-macosx-10.9-x86_64.egg-info    setuptools
_distutils_hack                        pip                            setuptools-67.6.0-py3.11.egg-info
distutils-precedence.pth                pip-23.0.1-py3.11.egg-info                wheel
gammalib                        pkg_resources                        wheel-0.38.4-py3.11.egg-info
Here, everything is in gammalib, there is no gammalib-2.0.0-py3.9-macosx-10.9-x86_64.egg folder but a gammalib-2.0.0-py3.9-macosx-10.9-x86_64.egg-info folder that does not contain the modules.

The difference between both formats is described at https://setuptools.pypa.io/en/latest/deprecated/python_eggs.html. I guess that the conda package manager changed the way how it deals with the eggs between Python 3.9 and 3.10 (the setuptools are setuptools-67.6.0 for both Python versions, as shown above).

#7 Updated by Knödlseder Jürgen about 1 year ago

I think the issue comes from the fact that the building and installation of the Python module is handled over the pyext/setup.py script, while the tests folder is generated using the gcc make system (file pyext/Makefile.am).

#8 Updated by Knödlseder Jürgen about 1 year ago

  • % Done changed from 30 to 50

It turns out that the different installation in Python 3.10 with respect to Python 3.9 is related to a behind-the-scenes switch from distutils to setuptools. See #4223 for more information.

#9 Updated by Knödlseder Jürgen about 1 year ago

#10 Updated by Knödlseder Jürgen about 1 year ago

  • Status changed from In Progress to Closed
  • % Done changed from 50 to 100

I have implemented the necessary changes (see #4223), they will go into the next release. I close the issue now.

Also available in: Atom PDF