Bug #1425
cssens excessive memory usage
| Status: | Closed | Start date: | 02/19/2015 | |
|---|---|---|---|---|
| Priority: | Normal | Due date: | ||
| Assigned To: |  Knödlseder Jürgen | % Done: | 100% | |
| Category: | - | |||
| Target version: | 1.0.0 | |||
| Duration: |
Description
Paola Grandi noted a problem when running cssens that is likely related to excessive memory usage (ctools version 00-08-01):
cssens Calibration database [$CTOOLS/share/caldb/data/cta/dummy/bcf] Instrument response function [cta_dummy_irf] Source model type (point/gauss/shell/disk) [point] Source offset angle (deg) (0.0) [0.0] Background model file function (none=power law for E) [$CTOOLS/share/models/bkg_dummy.txt] Effective exposure time (s) [10] Radius of ROI (deg) [0.5] 1 Warning: obsutils.set is obsolete, use obsutils.set_obs instead. Traceback (most recent call last): File "/home/grandi/ctools-00-08-01_build01/bin/cssens", line 724, in <module> app.execute() File "/home/grandi/ctools-00-08-01_build01/bin/cssens", line 167, in execute self.run() File "/home/grandi/ctools-00-08-01_build01/bin/cssens", line 234, in run result = self.get_sensitivity(obs, bkg_model, full_model) File "/home/grandi/ctools-00-08-01_build01/bin/cssens", line 445, in get_sensitivity log=self.m_log, debug=self.m_debug) File "/home/grandi/ctools-00-08-01_build01/lib64/python2.6/site-packages/ctools/obsutils.py", line 61, in sim sim.run() File "/home/grandi/ctools-00-08-01_build01/lib64/python2.6/site-packages/ctools/tools.py", line 268, in run return _tools.ctobssim_run(self) RuntimeError: std::bad_alloc the LOG file: more cssens.log ******************************************************************************** * cssens * * ---------------------------------------------------------------------------- * * Version: 0.4.0 * ******************************************************************************** 2015-02-18T12:47:55: +============+ 2015-02-18T12:47:55: | Parameters | 2015-02-18T12:47:55: +============+ 2015-02-18T12:47:55: outfile ...................: sensitivity.dat 2015-02-18T12:47:55: caldb .....................: $CTOOLS/share/caldb/data/cta/dummy/bcf 2015-02-18T12:47:55: irf .......................: cta_dummy_irf 2015-02-18T12:47:55: type ......................: point 2015-02-18T12:47:55: offset ....................: 0.0 2015-02-18T12:47:55: bkg .......................: $CTOOLS/share/models/bkg_dummy.txt 2015-02-18T12:47:55: duration ..................: 10 2015-02-18T12:47:55: rad .......................: 1 2015-02-18T12:47:55: enumbins ..................: 0 2015-02-18T12:47:55: npix ......................: 200 2015-02-18T12:47:55: binsz .....................: 0.05 2015-02-18T12:47:55: sigma .....................: 5.0 2015-02-18T12:47:55: ts_use ....................: yes 2015-02-18T12:47:55: index .....................: -2.48 2015-02-18T12:47:55: radius ....................: 0.1 2015-02-18T12:47:55: width .....................: 0.05 2015-02-18T12:47:55: max_iter ..................: 50 2015-02-18T12:47:55: num_avg ...................: 3 2015-02-18T12:47:55: chatter ...................: 2 2015-02-18T12:47:55: clobber ...................: yes 2015-02-18T12:47:55: debug .....................: no 2015-02-18T12:47:55: mode ......................: ql 2015-02-18T12:47:55: 2015-02-18T12:47:55: +========+ 2015-02-18T12:47:55: | Models | 2015-02-18T12:47:55: +========+ 2015-02-18T12:47:55: +------------------+ 2015-02-18T12:47:55: | Background model | 2015-02-18T12:47:55: +------------------+ 2015-02-18T12:47:55: === GModels === 2015-02-18T12:47:55: Number of models ..........: 1 2015-02-18T12:47:55: Number of parameters ......: 3 2015-02-18T12:47:55: === GCTAModelRadialAcceptance === 2015-02-18T12:47:55: Name ......................: Background 2015-02-18T12:47:55: Instruments ...............: CTA 2015-02-18T12:47:55: Instrument scale factors ..: unity 2015-02-18T12:47:55: Observation identifiers ...: all 2015-02-18T12:47:55: Model type ................: "Gaussian" * "FileFunction" * "Constant" 2015-02-18T12:47:55: Number of parameters ......: 3 2015-02-18T12:47:55: Number of radial par's ....: 1 2015-02-18T12:47:55: Sigma ....................: 3 +/- 0 [7.71728e-08,infty[ deg2 (free,scale=1,gradient) 2015-02-18T12:47:55: Number of spectral par's ..: 1 2015-02-18T12:47:55: Normalization ............: 1 +/- 0 [0,1000] (free,scale=1,gradient) 2015-02-18T12:47:55: Number of temporal par's ..: 1 2015-02-18T12:47:55: Constant .................: 1 (relative value) (fixed,scale=1,gradient) 2015-02-18T12:47:55: 2015-02-18T12:47:55: +------------+ 2015-02-18T12:47:55: | Full model | 2015-02-18T12:47:55: +------------+ 2015-02-18T12:47:55: === GModels === 2015-02-18T12:47:55: Number of models ..........: 2 2015-02-18T12:47:55: Number of parameters ......: 9 2015-02-18T12:47:55: === GCTAModelRadialAcceptance === 2015-02-18T12:47:55: Name ......................: Background 2015-02-18T12:47:55: Instruments ...............: CTA 2015-02-18T12:47:55: Instrument scale factors ..: unity 2015-02-18T12:47:55: Observation identifiers ...: all 2015-02-18T12:47:55: Model type ................: "Gaussian" * "FileFunction" * "Constant" 2015-02-18T12:47:55: Number of parameters ......: 3 2015-02-18T12:47:55: Number of radial par's ....: 1 2015-02-18T12:47:55: Sigma ....................: 3 +/- 0 [7.71728e-08,infty[ deg2 (free,scale=1,gradient) 2015-02-18T12:47:55: Number of spectral par's ..: 1 2015-02-18T12:47:55: Normalization ............: 1 +/- 0 [0,1000] (free,scale=1,gradient) 2015-02-18T12:47:55: Number of temporal par's ..: 1 2015-02-18T12:47:55: Constant .................: 1 (relative value) (fixed,scale=1,gradient) 2015-02-18T12:47:55: === GModelSky === 2015-02-18T12:47:55: Name ......................: Test 2015-02-18T12:47:55: Instruments ...............: all 2015-02-18T12:47:55: Instrument scale factors ..: unity 2015-02-18T12:47:55: Observation identifiers ...: all 2015-02-18T12:47:55: Model type ................: PointSource 2015-02-18T12:47:55: Model components ..........: "SkyDirFunction" * "PowerLaw" * "Constant" 2015-02-18T12:47:55: Number of parameters ......: 6 2015-02-18T12:47:55: Number of spatial par's ...: 2 2015-02-18T12:47:55: RA .......................: 266.405 deg (fixed,scale=1) 2015-02-18T12:47:55: DEC ......................: -28.9362 deg (fixed,scale=1) 2015-02-18T12:47:55: Number of spectral par's ..: 3 2015-02-18T12:47:55: Prefactor ................: 5.7e-16 +/- 0 [0,infty[ ph/cm2/s/MeV (free,scale=5.7e-16,gradient) 2015-02-18T12:47:55: Index ....................: -2.48 [10,-10] (fixed,scale=-2.48,gradient) 2015-02-18T12:47:55: PivotEnergy ..............: 300000 MeV (fixed,scale=300000,gradient) 2015-02-18T12:47:55: Number of temporal par's ..: 1 2015-02-18T12:47:55: Constant .................: 1 (relative value) (fixed,scale=1,gradient) 2015-02-18T12:47:55: 2015-02-18T12:47:55: +===========================+ 2015-02-18T12:47:55: | Sensitivity determination | 2015-02-18T12:47:55: +===========================+ 2015-02-18T12:47:55: 2015-02-18T12:47:55: +-------------------------------------+ 2015-02-18T12:47:55: | Energies: 15.8489 GeV - 25.1189 GeV | 2015-02-18T12:47:55: +-------------------------------------+ 2015-02-18T12:55:52:
Repeating this run on a Linux box indeed shows a hugh memory usage. This should be fixed. A possible reason could be a large background rate and a correspondingly large memory allocation.
Recurrence
No recurrence.
History
#1
Updated by Knödlseder Jürgen almost 10 years ago
- File cssens added
- File show_sensitivity.py
added - Status changed from New to Feedback
- % Done changed from 0 to 100
The problem was due to a wrong internal scaling of the source flux, related to an interface change a couple of time ago. Since the script is not really tested regularly this escaped my attention.
Attached a modified script that fixes the problem. The modifications have also been merged into the actual devel branch.
As bonus, attached is also a Python script that allows plotting of the result file. The script needs matplotlib installed.
#2
Updated by Knödlseder Jürgen over 9 years ago
- Status changed from Feedback to Closed