Support #1141

ctlike enters an infinite NaN/Inf loop

Added by Okumura Akira almost 11 years ago. Updated over 10 years ago.

Status:ClosedStart date:02/08/2014
Priority:NormalDue date:
Assigned To:Knödlseder Jürgen% Done:

100%

Category:-
Target version:00-09-00
Duration:

Description

I got the following error messages while using ctlike (GammaLib-00-08-01 and ctools-00-07-01). What should I do to remove this?

*** ERROR: GModelSky::npred: NaN/Inf encountered (npred=nan, npred_spatial=2.35824e+10, npred_spectral=nan, npred_temporal=1, srcEng=13.2614 TeV, srcTime=-3.15576e+08 s (TT))
*** ERROR: GObservation::npred_spec_kern::eval(x=16.4004):  NaN/Inf encountered (value=nan (value_out=nan exp(x)=1.32614e+07)
*** ERROR: GModelSpectralPlaw2::eval(srcEng=13.2633 TeV, srcTime=-3.15576e+08 s (TT)): NaN/Inf encountered (value=nan, integral=nan, m_norm=3.09797e+21, m_power=3.23146e-29)
*** ERROR: GModelSky::npred: NaN/Inf encountered (npred=nan, npred_spatial=2.35828e+10, npred_spectral=nan, npred_temporal=1, srcEng=13.2633 TeV, srcTime=-3.15576e+08 s (TT))
*** ERROR: GObservation::npred_spec_kern::eval(x=16.4005):  NaN/Inf encountered (value=nan (value_out=nan exp(x)=1.32633e+07)
*** ERROR: GModelSpectralPlaw2::eval(srcEng=13.2651 TeV, srcTime=-3.15576e+08 s (TT)): NaN/Inf encountered (value=nan, integral=nan, m_norm=3.09797e+21, m_power=3.22965e-29)

You can reproduce this with the following steps. Please use the CO FITS file that I sent to Jürgen a few days ago.

$ ctobssim
Model [model_OrionB_norm.xml] 
Calibration database [$GAMMALIB/share/caldb/cta] 
Instrument response function [kb_I_50h_v3] 
RA of pointing (degrees) (0-360) [86.1716] 
Dec of pointing (degrees) (-90-90) [-1.47746] 
Radius of FOV (degrees) (0-180) [5] 
Start time (MET in s) (0) [0] 
End time (MET in s) (0) [360000] 
Lower energy limit (TeV) (0) [0.1] 
Upper energy limit (TeV) (0) [100] 
Output event data file or observation definition file [events.fits]


$ ctselect
Input event list or observation definition file [events.fits] 
RA for ROI centre (degrees) (0-360) [86.1716] 
Dec for ROI centre (degrees) (-90-90) [-1.47746] 
Radius of ROI (degrees) (0-180) [5] 
Start time (CTA MET in seconds) (0) [0] 
End time (CTA MET in seconds) (0) [360000] 
Lower energy limit (TeV) (0) [10] 
Upper energy limit (TeV) (0) [31.6228] 
Output event list or observation definition file [select.fits] 

$ ctlike 
Event list, counts map or observation definition file [select.fits]
Calibration database [$GAMMALIB/share/caldb/cta] 
Instrument response function [kb_I_50h_v3] 
Source model [unbinned_initial_ebin04.xml ]
Source model output file [results.xml]
*** ERROR: GModelSpectralPlaw2::eval(srcEng=10 TeV, srcTime=-3.15576e+08 s (TT)): NaN/Inf encountered (value=nan, integral=nan, m_norm=3.09797e+21, m_power=1e-28)
*** ERROR: GModelSky::npred: NaN/Inf encountered (npred=nan, npred_spatial=2.14572e+10, npred_spectral=nan, npred_temporal=1, srcEng=10 TeV, srcTime=-3.15576e+08 s (TT))
*** ERROR: GObservation::npred_spec_kern::eval(x=16.1181):  NaN/Inf encountered (value=nan (value_out=nan exp(x)=1e+07)
*** ERROR: GModelSpectralPlaw2::eval(srcEng=31.6228 TeV, srcTime=-3.15576e+08 s (TT)): NaN/Inf encountered (value=nan, integral=nan, m_norm=3.09797e+21, m_power=9.99997e-31)
.
.
.

Two XML files are very similar. But in nubinned_initial_ebin04.xml, the spectral model uses PowerLaw2. The Prefactor was calculated with the PowerLaw in model_OrionB_norm.xml.

model_OrionB_norm.xml Magnifier (1.05 KB) Okumura Akira, 02/08/2014 01:24 AM

unbinned_initial_ebin04.xml Magnifier (1.17 KB) Okumura Akira, 02/08/2014 01:24 AM

select.fits (1.84 MB) Okumura Akira, 02/08/2014 01:24 AM

unbinned_initial_ebin04_small_scale.xml Magnifier (1.17 KB) Okumura Akira, 02/08/2014 09:15 PM

unbinned_initial_ebin04_small_scale_v2.xml Magnifier (1.17 KB) Okumura Akira, 02/08/2014 09:27 PM

Recurrence

No recurrence.

History

#1 Updated by Knödlseder Jürgen almost 11 years ago

  • Status changed from New to In Progress
  • Assigned To set to Knödlseder Jürgen
  • % Done changed from 0 to 10

I started investigating the issue. The immediate thing I recognized was that you use pretty large scaling factors and pretty small values. I guess this is the reason for the problems, but I’ll track this down. The goal of the scaling factors is to get the values close to 1, which is needed by the optimizer. When I run ctlike, I see

2014-02-08T19:55:40:  *Iteration   0: logL=-251768.322, Lambda=1.0e-03
2014-02-08T19:55:40:    Parameter "Index" drives optimization step (step=0.0974524)
2014-02-08T19:55:40:  *Iteration   1: logL=-251767.550, Lambda=1.0e-03, delta=0.772, max(|grad|)=3314564820958284183016407117594624.000000 [Integral:1]
2014-02-08T19:55:40:    Parameter "Index" does not drive optimization step anymore.
2014-02-08T19:55:40:    Parameter "Integral" drives optimization step (step=0.0204968)
2014-02-08T19:55:40:    Parameter "Index" hits minimum: -7.05339 < -4 (1)
2014-02-08T19:55:40:    Parameter "Index" has zero curvature. Fix parameter.
2014-02-08T19:55:40:  *Iteration   2: logL=-251767.427, Lambda=1.0e-04, delta=0.124, max(|grad|)=244.694009 [Normalization:7]
2014-02-08T19:55:40:    Parameter "Integral" does not drive optimization step anymore.

which signals an enormous gradient, which likely leeds to some numerical problems. I guess that rescaling your model will solve the problem, but I will track it down ...

#2 Updated by Okumura Akira almost 11 years ago

I should have mentioned that I had tried another XML in which scale was changed to a much smaller value. I will attach this new XML. But I get the same error.

$ diff unbinned_initial_ebin04*xml
5c5
<       <parameter name="Integral" value="1.78746e-34" error="0" scale="1.38148e+19" min="0" max="1.78746e-32" free="1" />
---
>       <parameter name="Integral" value="1.78746e-15" error="0" scale="1.38148" min="0" max="1.78746e-13" free="1" />

$ ctlike 
Event list, counts map or observation definition file [OrionB_norm/100hours/events_ebin00.fits] select.fits 
Calibration database [$GAMMALIB/share/caldb/cta] 
Instrument response function [kb_I_50h_v3] 
Source model [model_OrionB_norm_like_pl2.xml] unbinned_initial_ebin04_small_scale.xml
Source model output file [OrionB_norm/100hours/unbinned_results_ebin00.xml] results.xml

#3 Updated by Okumura Akira almost 11 years ago

I tried another XML in which value was changed to get close to 1. See the attached. This one worked as expected.

The goal of the scaling factors is to get the values close to 1, which is needed by the optimizer.

It would be nice if the optimizer could rescale the scale and value automatically inside GammaLib. Users do not care how the optimizer runs behind the scenes...

#5 Updated by Knödlseder Jürgen almost 11 years ago

The problem occurs because the minimum of the integral is set to 0, and the code does not like a Powerlaw2 integral of 0. There was in fact a division by zero problem in Powerlaw2, which produced the error. Corrected this and now runs through. Still, correct scaling should be done.

#6 Updated by Okumura Akira almost 11 years ago

Thank you Jürgen. I will keep this in mind, and will tell my colleagues to check the scaling factor.

#7 Updated by Knödlseder Jürgen almost 11 years ago

Okumura Akira wrote:

I tried another XML in which value was changed to get close to 1. See the attached. This one worked as expected.

The goal of the scaling factors is to get the values close to 1, which is needed by the optimizer.

It would be nice if the optimizer could rescale the scale and value automatically inside GammaLib. Users do not care how the optimizer runs behind the scenes...

GammaLib has means to evaluate this automatically, but the goal of having scale and value in the XML file is precisely to get control over the optimizer (the same is true for the Fermi Science Tools, otherwise there is no point in having scale in addition to value). So rule of thumb: put the true value in scale and set value to 1. This should always leads to a reasonable fit.

#8 Updated by Okumura Akira almost 11 years ago

So rule of thumb: put the true value in scale and set value to 1. This should always leads to a reasonable fit.

To be honest, I did not know this rule, but it seems that I have been using “good” XMLs for LAT so far. The reason why I used a very small value was that I wanted to use scale for the scaling factor of the FITS file. So I did not want to mix two scaling factors in scale.

Anyway, thank you again for your helpful comments.

#9 Updated by Knödlseder Jürgen almost 11 years ago

  • Status changed from In Progress to Feedback

Merged into the GammaLib devel branch. Will move the issue to GammaLib as the GammaLib code was concerned.

#10 Updated by Knödlseder Jürgen almost 11 years ago

  • Project changed from ctools to GammaLib

#11 Updated by Knödlseder Jürgen almost 11 years ago

  • Target version set to 00-08-02

#12 Updated by Knödlseder Jürgen almost 11 years ago

  • % Done changed from 10 to 100

#13 Updated by Knödlseder Jürgen over 10 years ago

  • Status changed from Feedback to Resolved
  • Target version changed from 00-08-02 to 00-09-00

Skip 0.8.2 release.

#14 Updated by Knödlseder Jürgen over 10 years ago

  • Status changed from Resolved to Closed

Also available in: Atom PDF