Action #2211

Feature #1490: Build binary conda packages for Gammalib and ctools

Prepare packages with conda

Added by Brau-Nogué Sylvie over 6 years ago. Updated over 6 years ago.

Status:ClosedStart date:10/09/2017
Priority:NormalDue date:10/31/2017
Assigned To:Brau-Nogué Sylvie% Done:

100%

Category:-
Target version:1.5.0
Tags: PACKAGING
Duration: 23

Description

Tout est dans le titre ...

En s’inspirant largement des recipes préparées par K. Kosack, voir https://github.com/cta-observatory/cta-conda-recipes/tree/master/extern

Recurrence

No recurrence.

History

#1 Updated by Brau-Nogué Sylvie over 6 years ago

  • Status changed from New to In Progress
  • % Done changed from 0 to 90

Create environment for Python 2.7

$ conda create -n ctatest python=2.7 anaconda
$ source activate ctatest
(ctatest) $ conda install -c conda-forge cfitsio
(ctatest) $ conda update --all

Prepare recipes

git clone https://github.com/cta-observatory/cta-conda-recipes
cd cta-conda-recipes/extern
(ctatest) $ conda update conda

Create package

(ctatest) $ conda build gammalib-1.3.0.conda --python=2.7
BUILD START: gammalib-1.3.0-py27_1
Can't build $HOME/cta-conda-recipes/extern/gammalib-1.3.0.conda due to unsatisfiable dependencies:

  - cfitsio
  - gammalib ==1.3
  - cfitsio

La solution : conda config --append channels conda-forge ⇐

 
(ctatest) $ conda build gammalib-1.2.0.conda --python=2.7

BUILD START: gammalib-1.3.0-py27_1

The following NEW packages will be INSTALLED:

    ca-certificates: 2017.08.26-h1d4fec5_0     
    certifi:         2017.7.27.1-py27h9ceb091_0
    libedit:         3.1-heed3624_0            
    libffi:          3.2.1-h4deb6c0_3          
    libgcc-ng:       7.2.0-h7cc24e2_2          
    libstdcxx-ng:    7.2.0-h7a57d05_2          
    ncurses:         6.0-h06874d7_1            
    openssl:         1.0.2l-h9d1a558_3         
    pcre:            8.41-hc71a17e_0           
    pip:             9.0.1-py27hbf658b2_3      
    python:          2.7.14-h1aa7481_19        
    readline:        7.0-hac23ff0_3            
    setuptools:      36.5.0-py27h68b189e_0     
    sqlite:          3.20.1-h6d8b0f3_1         
    swig:            3.0.10-0                  
    tk:              8.6.7-h5979e9b_1          
    wheel:           0.29.0-py27h411dd7b_1     
    zlib:            1.2.11-hfbfcf68_1         

WARNING: conda-build appears to be out of date. You have version 2.1.17 but the
latest version is 3.0.25. Run

conda update -n root conda-build

to get the latest version.

Source cache directory is: /home/osboxes/anaconda2/conda-bld/src_cache
Downloading source to cache: gammalib-1.3.0.tar.gz
Downloading http://cta.irap.omp.eu/ctools/releases/gammalib/gammalib-1.3.0.tar.gz
INFO:fetch.start:(u'gammalib-1.3.0', 29075351)
INFO:fetch.update:16384
INFO:fetch.update:32768

[......]

INFO:fetch.update:29065216
INFO:fetch.update:29075351
INFO:fetch.stop:None
Success
Extracting download
Package: gammalib-1.3.0-py27_1
source tree in: /home/osboxes/anaconda2/conda-bld/gammalib_1507557279726/work/gammalib-1.3.0
+ source /home/osboxes/anaconda2/bin/activate /home/osboxes/anaconda2/conda-bld/gammalib_1507557279726/_b_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_p
+ ./configure --prefix=/home/osboxes/anaconda2/conda-bld/gammalib_1507557279726/_b_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_p
checking for a BSD-compatible install... /usr/bin/install -c
checking whether build environment is sane... yes
checking for a thread-safe mkdir -p... /usr/bin/mkdir -p
checking for gawk... gawk
 .....

/home/osboxes/anaconda2/conda-bld/linux-64/gammalib-1.3.0-py27_1.tar.bz2
updating index in: /home/osboxes/anaconda2/conda-bld
updating index in: /home/osboxes/anaconda2/conda-bld/noarch
updating index in: /home/osboxes/anaconda2/conda-bld/linux-64
updating: gammalib-1.3.0-py27_1.tar.bz2
updating index in: /home/osboxes/anaconda2/conda-bld
Nothing to test for: /home/osboxes/anaconda2/conda-bld/linux-64/gammalib-1.3.0-py27_1.tar.bz2
INFO:conda_build.config:--dirty flag and --keep-old-work not specified.Removing build/test folder after successful build/test.

# Automatic uploading is disabled
# If you want to upload package(s) to anaconda.org later, type:

anaconda upload /home/osboxes/anaconda2/conda-bld/linux-64/gammalib-1.3.0-py27_1.tar.bz2

# To have conda build upload to anaconda.org automatically, use
# $ conda config --set anaconda_upload yes

anaconda_upload is not set.  Not uploading wheels: []

####################################################################################
Source and build intermediates have been left in /home/osboxes/anaconda2/conda-bld.
There are currently 4 accumulated.
To remove them, you can run the ```conda build purge``` comman

Local install

conda create -n py27 python=2.7 anaconda
source activate py27
conda install --offline /home/osboxes/anaconda2/conda-bld/linux-64/gammalib-1.3.0-py27_1.tar.bz2

#2 Updated by Brau-Nogué Sylvie over 6 years ago

  • % Done changed from 90 to 70

Install then preliminary tests

Set conda environment

  • source activate py27 ← py27 created by the command : conda create -n py27 python=2.7 anaconda
  • conda config --append channels conda-forge
  • conda config --append channels cta-observatory ← optional

Set gammalib/ctools environment

export GAMMALIB=$HOME/anaconda2/envs/py27
export CTOOLS=$HOME/anaconda2/envs/py27

Gammalib is OK


--------- INSTALL ---------

(py27) $ conda install --offline /home/osboxes/anaconda2/conda-bld/linux-64/gammalib-1.3.0-py27_1.tar.bz2 

--------- TEST ---------

(py27) $ source $GAMMALIB/bin/gammalib-init.sh

(py27) $ python -c 'import gammalib; gammalib.test()'

****************************
* Python interface testing *
****************************
Test GLog: ....................................... ok
Test GApplication: ...... ok
Test GApplicationPars: .. ok
Test GFits class: ...... ok
Test GFitsImage class: ........................... ok
Test GFitsTable class: . ok
Test GFitsTableCol classes: ....................... ok
Test GVector: ......................................... ok
Test GMatrix: ................ ok
Test GMatrixSparse: ................ ok
Test GMatrixSymmetric: .......... ok
Test GModels: ... ok
Test GFft: .. ok
Test GEnergy class: ................. ok
Test GEnergies class: .... ok
Test GTime class: ............... ok
Optimizer module dummy test: . ok
Test sky map methods: ............... ok
Test sky map friend methods: ...... ok
Test sky map operators: .......................... ok
Test HEALPix map: ................................................................................................. ok
Test AIT projection map: ................................................... ok
Test AZP projection map: ................................................... ok
Test CAR projection map: ................................................... ok
Test GLS projection map: ................................................... ok
Test MER projection map: ................................................... ok
Test MOL projection map: ................................................... ok
Test SFL projection map: ................................................... ok
Test SIN projection map: ................................................... ok
Test STG projection map: ................................................... ok
Test TAN projection map: ................................................... ok
Test FK5 to Galactic coordinate conversion: .. ok
Test GNodeArray: ..................... ok
Test GUrlFile: ... ok
Test GUrlString: ... ok
Test GFilename: ......... ok
Test GCsv: ................... ok
Test module dummy test: . ok
XML module dummy test: . ok
Test GPha: ... ok
Test GArf: ... ok
Test GRmf: ..... ok
Test GVOClient: . ok
MWL dummy test: . ok
Test CTA effective area classes: ................. ok
Test CTA PSF classes: ......................... ok
Test CTA energy dispersion classes: ......... ok
Test CTA response classes: . ok
Test CTA On/Off analysis: ......... ok
LAT dummy test: . ok
COMPTEL dummy test: . ok

ctools is OK


--------- INSTALL ---------

(py27)$ conda install --offline /home/osboxes/anaconda2/conda-bld/linux-64/ctools-1.3.0-py27_1.tar.bz2 

--------- TEST ---------
(py27) $ source $CTOOLS/bin/ctools-init.sh

(py27) $ python -c 'import ctools; ctools.test()'

***********************
* ctools unit testing *
***********************
Test ctobssim on command line: ....... ok
Test ctobssim from Python: ......................................................................................... ok
Test ctselect on command line: ......... ok
Test ctselect from Python: ............................................................. ok
Test ctphase on command line: ....... ok
Test ctphase from Python: .............. ok
Test ctbin on command line: .......... ok
Test ctbin from Python: .................................................... ok
Test ctlike on command line: ................ ok
Test ctlike from Python: ..................................................... ok
Test cttsmap on command line: ........................ ok
Test cttsmap from Python: ................................ ok
Test ctmodel on command line: ........ ok
Test ctmodel from Python: .................... ok
Test ctskymap on command line: ........ ok
Test ctskymap from Python: ....................... ok
Test ctexpcube on command line: ...... ok
Test ctexpcube from Python: ........ ok
Test ctpsfcube on command line: ...... ok
Test ctpsfcube from Python: ........ ok
Test ctedispcube on command line: ....... ok
Test ctedispcube from Python: .............. ok
Test ctbkgcube on command line: ....... ok
Test ctbkgcube from Python: .............. ok
Test ctmapcube on command line: ........ ok
Test ctmapcube from Python: .................... ok
Test ctcubemask on command line: ....... ok
Test ctcubemask from Python: ............. ok
Test ctbutterfly on command line: ....... ok
Test ctbutterfly from Python: ....... ok
Test ctulimit on command line: ..... ok
Test ctulimit from Python: ........................ ok
Test cterror on command line: ................ ok
Test cterror from Python: ...................................................... ok
Test ctprob on command line: ........ ok
Test ctprob from Python: ................ ok
Test unbinned pipeline with FITS file saving: . ok
Test unbinned in-memory pipeline: . ok

#3 Updated by Brau-Nogué Sylvie over 6 years ago

  • % Done changed from 70 to 90

<<<<< All tests passed successfully >>>>>

Procedure summary

Prerequisites
  • the py27 environment exists (previously created by the command : conda create -n py27 python=2.7 anaconda)
  • set variable ANACONDA_PATH ($HOME/anaconda2 in this example)
Set conda environment
  • source activate py27
  • conda config --append channels conda-forge
  • conda config --append channels cta-observatory ← optional
Set gammalib/ctools environment
  • export GAMMALIB=$ ANACONDA_PATH/envs/py27
  • export CTOOLS=$ ANACONDA_PATH/envs/py27
Local install conda packages
  • conda install --offline $ANACONDA_PATH/conda-bld/linux-64/gammalib-1.3.0-py27_1.tar.bz2
  • conda install --offline $ANACONDA_PATH/conda-bld/linux-64/ctools-1.3.0-py27_1.tar.bz2
Gammalib
  • source $GAMMALIB/bin/gammalib-init.sh
  • python -c 'import gammalib; gammalib.test()'
ctools
  • source $CTOOLS/bin/ctools-init.sh
  • python -c 'import ctools; ctools.test()'

#4 Updated by Knödlseder Jürgen over 6 years ago

  • Status changed from In Progress to Closed
  • % Done changed from 90 to 100

Conda package generation is now integrated in the release pipeline (#2278).

Also available in: Atom PDF