Bug #3628
ctools and gammalib environment not correctly initialised when conda is running in tcsh
| Status: | Closed | Start date: | 04/28/2021 | |
|---|---|---|---|---|
| Priority: | Immediate | Due date: | ||
| Assigned To: | % Done: | 100% | ||
| Category: | - | |||
| Target version: | 1.7.4 | |||
| Duration: |
Description
Martin Giard had a problem when trying to install and unit test ctools, the issue being that the CTOOLS environment variable was not set. It turned out that he was running anaconda under tcsh, and that the activation scripts are only provided in shell.
See https://github.com/conda/conda-build/issues/3880 for a related issue. Possibly adding specific csh activation scripts can solve the problem.
Recurrence
No recurrence.
History
#1
Updated by Knödlseder Jürgen over 5 years ago
- % Done changed from 0 to 10
In order to try reproducing the problem on my Mac I installed a new miniconda version under tcsh:
$ tcsh
% ~/Downloads/Miniconda2-latest-MacOSX-x86_64.sh
% conda env list
# conda environments:
#
/Users/jurgen/anaconda3
/Users/jurgen/anaconda3/envs/astropy
/Users/jurgen/anaconda3/envs/ctools-1.6.1
/Users/jurgen/anaconda3/envs/ctools-1.7.3
/Users/jurgen/anaconda3/envs/ctools-paper
/Users/jurgen/anaconda3/envs/rootenv
/Users/jurgen/anaconda3/envs/sharepoint
base * /Users/jurgen/miniconda2-tcsh
% conda info
active environment : None
user config file : /Users/jurgen/.condarc
populated config files : /Users/jurgen/.condarc
conda version : 4.8.3
conda-build version : not installed
python version : 2.7.18.final.0
virtual packages : __osx=10.14.6
base environment : /Users/jurgen/miniconda2-tcsh (writable)
channel URLs : https://conda.anaconda.org/conda-forge/osx-64
https://conda.anaconda.org/conda-forge/noarch
https://conda.anaconda.org/cta-observatory/osx-64
https://conda.anaconda.org/cta-observatory/noarch
https://repo.anaconda.com/pkgs/main/osx-64
https://repo.anaconda.com/pkgs/main/noarch
https://repo.anaconda.com/pkgs/r/osx-64
https://repo.anaconda.com/pkgs/r/noarch
package cache : /Users/jurgen/miniconda2-tcsh/pkgs
/Users/jurgen/.conda/pkgs
envs directories : /Users/jurgen/miniconda2-tcsh/envs
/Users/jurgen/.conda/envs
platform : osx-64
user-agent : conda/4.8.3 requests/2.22.0 CPython/2.7.18 Darwin/18.7.0 OSX/10.14.6
UID:GID : 501:20
netrc file : None
offline mode : False
I then followed the gammalib installation instructions:
% conda update -n base -c defaults conda
% conda create -n tcsh python=2.7
% conda activate tcsh
CommandNotFoundError: Your shell has not been properly configured to use 'conda activate'.
To initialize your shell, run
$ conda init <SHELL_NAME>
Currently supported shells are:
- bash
- fish
- tcsh
- xonsh
- zsh
- powershell
See 'conda init --help' for more information and options.
IMPORTANT: You may need to close and restart your shell after running 'conda init'.
% conda activate tcsh
modified /Users/jurgen/.tcshrc
==> For changes to take effect, close and re-open your current shell. <==
% exit
exit
macp0135:~ jurgen$ tcsh
% conda activate tcsh
% conda install gammalib
% conda activate tcsh
% python -c 'import gammalib; gammalib.test()'
% conda install ctools
% python -c 'import ctools; ctools.test()'
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "/usr/local/gamma/lib/python2.7/site-packages/ctools/tools.py", line 2060, in test
test_python_ctools.test(installed=True)
File "/usr/local/gamma/lib/python2.7/site-packages/ctools/tests/test_python_ctools.py", line 102, in test
os.system('cp -r %s/syspfiles/*.par pfiles/' % (os.environ['CTOOLS']))
File "/Users/jurgen/miniconda2-tcsh/envs/tcsh/lib/python2.7/UserDict.py", line 40, in __getitem__
raise KeyError(key)
KeyError: 'CTOOLS'
While GammaLib seems to work, ctools doesn’t. This reproduces Martin’s problem.#2
Updated by Knödlseder Jürgen over 5 years ago
- Status changed from New to In Progress
- % Done changed from 10 to 20
I added the file 01-ctools-activate.csh to the /Users/jurgen/miniconda2-tcsh/envs/tcsh/etc/conda/activate.d directory:
#!/bin/csh
setenv CTOOLS ${CONDA_PREFIX}
source ${CTOOLS}/bin/ctools-init.csh
unsetenv PYTHONPATH
unsetenv LD_LIBRARY_PATH
unsetenv DYLD_LIBRARY_PATH
Doing now
% conda activate tcsh (tcsh) [macp0135:~] jurgen% echo $CTOOLS /Users/jurgen/miniconda2-tcsh/envs/tcsh (tcsh) [macp0135:~] jurgen% python -c 'import ctools; ctools.test()' (tcsh) [macp0135:~] jurgen% python -c 'import cscripts; cscripts.test()' (tcsh) [macp0135:~] jurgen% python -c 'import gammalib; gammalib.test()'works!
#3
Updated by Knödlseder Jürgen over 5 years ago
- % Done changed from 20 to 30
I added 00-gammalib-activate.csh and 01-gammalib-deactivate.csh to GammaLib’s dev directory and typed in the GammaLib directory the following to build GammaLib locally:
$ /Users/jurgen/anaconda3/bin/conda-build dev/conda.recipe BUILD START: ['gammalib-1.7.4-py35_1.tar.bz2'] Could not download http://cta.irap.omp.eu/ctools/releases/gammalib/gammalib-1.7.4.tar.gzThis means that code downloading is needed to make this work.
#4
Updated by Knödlseder Jürgen over 5 years ago
- % Done changed from 30 to 40
Following the bugfix 1.7.4 release that includes conda activation scripts for csh I tested the gammalib installation using conda under tcsh on Mac OS:
% conda create -n tcsh python=2.7
% conda activate tcsh
% conda install gammalib
The following packages will be downloaded:
package | build
---------------------------|-----------------
gammalib-1.7.4 | py27h59582e6_1 26.7 MB cta-observatory
------------------------------------------------------------
Total: 26.7 MB
% echo $GAMMALIB
GAMMALIB: Undefined variable.
% conda activate tcsh
% echo $GAMMALIB
GAMMALIB: Undefined variable.
% ls /Users/jurgen/miniconda2-tcsh/envs/tcsh/etc/conda/activate.d/
00-gammalib-activate.sh
The csh activation script is missing, as well as the deactivation script. I forgot to add the inclusion of these scripts in the tarball.#5
Updated by Knödlseder Jürgen over 5 years ago
- % Done changed from 40 to 50
After adding the csh scripts to the conda distribution I redid the check. Now everything works nicely. It turns out that the re-activation after installation of GammaLib is not needed anymore, conda seems to do the reactivation automatically.
% echo $GAMMALIB
GAMMALIB: Undefined variable.
% conda create -n tcsh python=2.7
% conda activate tcsh
% echo $GAMMALIB
GAMMALIB: Undefined variable.
% conda install gammalib
The following packages will be downloaded:
package | build
---------------------------|-----------------
gammalib-1.7.4 | py27h59582e6_1 26.7 MB cta-observatory
------------------------------------------------------------
Total: 26.7 MB
% echo $GAMMALIB
/Users/jurgen/miniconda2-tcsh/envs/tcsh
% python -c 'import gammalib; gammalib.test()'
#6
Updated by Knödlseder Jürgen over 5 years ago
I verified the same procedure with Python 3.5, 3.6, 3.7 and 3.8 and they all work as for Python 2.7.
I also verified that using bash no re-activation of the environment is needed. This seems therefore a generic feature of conda now:
echo $GAMMALIB $ conda create -n bash python=2.7 $ conda activate bash $ echo $GAMMALIB $ conda install gammalib $ echo $GAMMALIB /Users/jurgen/anaconda3/envs/bash $ python -c 'import gammalib; gammalib.test()' $ conda deactivate $ echo $GAMMALIBNote that deactivate properly removes the environment variable. Same was also checked for
tcsh.#7
Updated by Knödlseder Jürgen over 5 years ago
- % Done changed from 50 to 60
I now did the same thing for ctools, first on tcsh. Everything works as expected.
echo $CTOOLS
CTOOLS: Undefined variable.
% conda create -n tcsh python=2.7
% conda activate tcsh
% echo $CTOOLS
CTOOLS: Undefined variable.
% conda install ctools
The following packages will be downloaded:
package | build
---------------------------|-----------------
ctools-1.7.4 | py27h97731cd_1 7.5 MB cta-observatory
------------------------------------------------------------
Total: 7.5 MB
% echo $CTOOLS
/Users/jurgen/miniconda2-tcsh/envs/tcsh
% python -c 'import ctools; ctools.test()'
% python -c 'import cscripts; cscripts.test()'
% python -c 'import gammalib; gammalib.test()'
% conda deactivate
% echo $CTOOLS
CTOOLS: Undefined variable.
#8
Updated by Knödlseder Jürgen over 5 years ago
- Status changed from In Progress to Closed
- % Done changed from 60 to 100
It also works perfectly for Python 3.5 - 3.8.
I also checked for bash, and also here everything is fine. Close the issue now.