Bug #3628
ctools and gammalib environment not correctly initialised when conda is running in tcsh
Status: | Closed | Start date: | 04/28/2021 | |
---|---|---|---|---|
Priority: | Immediate | Due date: | ||
Assigned To: | Knödlseder Jürgen | % Done: | 100% | |
Category: | - | |||
Target version: | 1.7.4 | |||
Duration: |
Description
Martin Giard had a problem when trying to install and unit test ctools, the issue being that the CTOOLS
environment variable was not set. It turned out that he was running anaconda under tcsh
, and that the activation scripts are only provided in shell.
See https://github.com/conda/conda-build/issues/3880 for a related issue. Possibly adding specific csh activation scripts can solve the problem.
Recurrence
No recurrence.
History
#1 Updated by Knödlseder Jürgen over 3 years ago
- % Done changed from 0 to 10
In order to try reproducing the problem on my Mac I installed a new miniconda version under tcsh
:
$ tcsh % ~/Downloads/Miniconda2-latest-MacOSX-x86_64.sh % conda env list # conda environments: # /Users/jurgen/anaconda3 /Users/jurgen/anaconda3/envs/astropy /Users/jurgen/anaconda3/envs/ctools-1.6.1 /Users/jurgen/anaconda3/envs/ctools-1.7.3 /Users/jurgen/anaconda3/envs/ctools-paper /Users/jurgen/anaconda3/envs/rootenv /Users/jurgen/anaconda3/envs/sharepoint base * /Users/jurgen/miniconda2-tcsh % conda info active environment : None user config file : /Users/jurgen/.condarc populated config files : /Users/jurgen/.condarc conda version : 4.8.3 conda-build version : not installed python version : 2.7.18.final.0 virtual packages : __osx=10.14.6 base environment : /Users/jurgen/miniconda2-tcsh (writable) channel URLs : https://conda.anaconda.org/conda-forge/osx-64 https://conda.anaconda.org/conda-forge/noarch https://conda.anaconda.org/cta-observatory/osx-64 https://conda.anaconda.org/cta-observatory/noarch https://repo.anaconda.com/pkgs/main/osx-64 https://repo.anaconda.com/pkgs/main/noarch https://repo.anaconda.com/pkgs/r/osx-64 https://repo.anaconda.com/pkgs/r/noarch package cache : /Users/jurgen/miniconda2-tcsh/pkgs /Users/jurgen/.conda/pkgs envs directories : /Users/jurgen/miniconda2-tcsh/envs /Users/jurgen/.conda/envs platform : osx-64 user-agent : conda/4.8.3 requests/2.22.0 CPython/2.7.18 Darwin/18.7.0 OSX/10.14.6 UID:GID : 501:20 netrc file : None offline mode : False
I then followed the gammalib installation instructions:
% conda update -n base -c defaults conda % conda create -n tcsh python=2.7 % conda activate tcsh CommandNotFoundError: Your shell has not been properly configured to use 'conda activate'. To initialize your shell, run $ conda init <SHELL_NAME> Currently supported shells are: - bash - fish - tcsh - xonsh - zsh - powershell See 'conda init --help' for more information and options. IMPORTANT: You may need to close and restart your shell after running 'conda init'. % conda activate tcsh modified /Users/jurgen/.tcshrc ==> For changes to take effect, close and re-open your current shell. <== % exit exit macp0135:~ jurgen$ tcsh % conda activate tcsh % conda install gammalib % conda activate tcsh % python -c 'import gammalib; gammalib.test()' % conda install ctools % python -c 'import ctools; ctools.test()' Traceback (most recent call last): File "<string>", line 1, in <module> File "/usr/local/gamma/lib/python2.7/site-packages/ctools/tools.py", line 2060, in test test_python_ctools.test(installed=True) File "/usr/local/gamma/lib/python2.7/site-packages/ctools/tests/test_python_ctools.py", line 102, in test os.system('cp -r %s/syspfiles/*.par pfiles/' % (os.environ['CTOOLS'])) File "/Users/jurgen/miniconda2-tcsh/envs/tcsh/lib/python2.7/UserDict.py", line 40, in __getitem__ raise KeyError(key) KeyError: 'CTOOLS'While GammaLib seems to work, ctools doesn’t. This reproduces Martin’s problem.
#2 Updated by Knödlseder Jürgen over 3 years ago
- Status changed from New to In Progress
- % Done changed from 10 to 20
I added the file 01-ctools-activate.csh
to the /Users/jurgen/miniconda2-tcsh/envs/tcsh/etc/conda/activate.d
directory:
#!/bin/csh
setenv CTOOLS ${CONDA_PREFIX}
source ${CTOOLS}/bin/ctools-init.csh
unsetenv PYTHONPATH
unsetenv LD_LIBRARY_PATH
unsetenv DYLD_LIBRARY_PATH
Doing now
% conda activate tcsh (tcsh) [macp0135:~] jurgen% echo $CTOOLS /Users/jurgen/miniconda2-tcsh/envs/tcsh (tcsh) [macp0135:~] jurgen% python -c 'import ctools; ctools.test()' (tcsh) [macp0135:~] jurgen% python -c 'import cscripts; cscripts.test()' (tcsh) [macp0135:~] jurgen% python -c 'import gammalib; gammalib.test()'works!
#3 Updated by Knödlseder Jürgen over 3 years ago
- % Done changed from 20 to 30
I added 00-gammalib-activate.csh
and 01-gammalib-deactivate.csh
to GammaLib’s dev
directory and typed in the GammaLib directory the following to build GammaLib locally:
$ /Users/jurgen/anaconda3/bin/conda-build dev/conda.recipe BUILD START: ['gammalib-1.7.4-py35_1.tar.bz2'] Could not download http://cta.irap.omp.eu/ctools/releases/gammalib/gammalib-1.7.4.tar.gzThis means that code downloading is needed to make this work.
#4 Updated by Knödlseder Jürgen over 3 years ago
- % Done changed from 30 to 40
Following the bugfix 1.7.4 release that includes conda activation scripts for csh I tested the gammalib installation using conda under tcsh on Mac OS:
% conda create -n tcsh python=2.7 % conda activate tcsh % conda install gammalib The following packages will be downloaded: package | build ---------------------------|----------------- gammalib-1.7.4 | py27h59582e6_1 26.7 MB cta-observatory ------------------------------------------------------------ Total: 26.7 MB % echo $GAMMALIB GAMMALIB: Undefined variable. % conda activate tcsh % echo $GAMMALIB GAMMALIB: Undefined variable. % ls /Users/jurgen/miniconda2-tcsh/envs/tcsh/etc/conda/activate.d/ 00-gammalib-activate.shThe
csh
activation script is missing, as well as the deactivation script. I forgot to add the inclusion of these scripts in the tarball.#5 Updated by Knödlseder Jürgen over 3 years ago
- % Done changed from 40 to 50
After adding the csh scripts to the conda distribution I redid the check. Now everything works nicely. It turns out that the re-activation after installation of GammaLib is not needed anymore, conda seems to do the reactivation automatically.
% echo $GAMMALIB GAMMALIB: Undefined variable. % conda create -n tcsh python=2.7 % conda activate tcsh % echo $GAMMALIB GAMMALIB: Undefined variable. % conda install gammalib The following packages will be downloaded: package | build ---------------------------|----------------- gammalib-1.7.4 | py27h59582e6_1 26.7 MB cta-observatory ------------------------------------------------------------ Total: 26.7 MB % echo $GAMMALIB /Users/jurgen/miniconda2-tcsh/envs/tcsh % python -c 'import gammalib; gammalib.test()'
#6 Updated by Knödlseder Jürgen over 3 years ago
I verified the same procedure with Python 3.5, 3.6, 3.7 and 3.8 and they all work as for Python 2.7.
I also verified that using bash
no re-activation of the environment is needed. This seems therefore a generic feature of conda now:
echo $GAMMALIB $ conda create -n bash python=2.7 $ conda activate bash $ echo $GAMMALIB $ conda install gammalib $ echo $GAMMALIB /Users/jurgen/anaconda3/envs/bash $ python -c 'import gammalib; gammalib.test()' $ conda deactivate $ echo $GAMMALIBNote that deactivate properly removes the environment variable. Same was also checked for
tcsh
.#7 Updated by Knödlseder Jürgen over 3 years ago
- % Done changed from 50 to 60
I now did the same thing for ctools, first on tcsh
. Everything works as expected.
echo $CTOOLS CTOOLS: Undefined variable. % conda create -n tcsh python=2.7 % conda activate tcsh % echo $CTOOLS CTOOLS: Undefined variable. % conda install ctools The following packages will be downloaded: package | build ---------------------------|----------------- ctools-1.7.4 | py27h97731cd_1 7.5 MB cta-observatory ------------------------------------------------------------ Total: 7.5 MB % echo $CTOOLS /Users/jurgen/miniconda2-tcsh/envs/tcsh % python -c 'import ctools; ctools.test()' % python -c 'import cscripts; cscripts.test()' % python -c 'import gammalib; gammalib.test()' % conda deactivate % echo $CTOOLS CTOOLS: Undefined variable.
#8 Updated by Knödlseder Jürgen over 3 years ago
- Status changed from In Progress to Closed
- % Done changed from 60 to 100
It also works perfectly for Python 3.5 - 3.8.
I also checked for bash
, and also here everything is fine. Close the issue now.