Action #2654

Correct handling of H.E.S.S. 3D energy dispersion IRFs

Added by Knödlseder Jürgen over 4 years ago. Updated about 4 years ago.

Status:ClosedStart date:07/31/2018
Priority:NormalDue date:
Assigned To:Knödlseder Jürgen% Done:

100%

Category:-
Target version:1.6.0
Duration:

Description

Andreas Specovius reported some issues with the 3D IRF energy dispersion kludge when handling H.E.S.S. data. The smoothing-kludge does not work well for HESS data since the interpolation between optical phases leads to double peaks in the energy dispersion. There also seems to be an issue with the renormalization.

edisp_before.png (36.7 KB) Knödlseder Jürgen, 07/31/2018 06:04 PM

edisp_after.png (39.5 KB) Knödlseder Jürgen, 07/31/2018 06:05 PM

integration-issues.png (39.3 KB) Knödlseder Jürgen, 09/07/2018 04:36 PM

Edisp_before Edisp_after Integration-issues

Recurrence

No recurrence.

History

#1 Updated by Knödlseder Jürgen over 4 years ago

The energy dispersion kludge indeed distorts considerably the H.E.S.S. energy dispersion. The panels below show the energy dispersion as provided in the FITS file hess_edisp_2d_047802.fits as red histograms and the energy dispersion as returned by GCTAEdisp2D as blue histograms. Left panel is with the energy dispersion kludge, right panel is without the energy dispersion kludge.

Note that the internal GCTAEdisp2D normalization still leads to a different magnitude of the energy dispersion, but that internal normalization is needed since the integral of the GCTAEdisp2D operator over log(E_obs) should be unity. To check this I computed the integrals using a simple Python code

logEsrc = math.log10(etrues[ietrue])
migs    = [0.01*i+0.01 for i in range(500)]
sum     = 0.0
for mig in migs:
    logEobs     = math.log10(mig) + logEsrc
    logEobs_max = math.log10(mig+0.005)
    logEobs_min = math.log10(mig-0.005)
    dlogEobs    = logEobs_max-logEobs_min
    value       = edisp2D(logEobs, logEsrc, 0.0)
    sum        += value * dlogEobs
print('Sum of edisp2D %f' % sum)

The results for both histograms are
Sum of edisp2D 1.000108
Sum of edisp2D 0.999869

while without the internal normalisation they are
Sum of edisp2D 0.281121
Sum of edisp2D 0.445356

I guess that HAP fills the histograms so that the integral over the migration is unity, but the normalisation needed is that the integral over the log10 of reconstructed energy is unity.

So my conclusion is that the normalization is done correctly, and you should keep it to get the correct fit results.

#2 Updated by Knödlseder Jürgen over 4 years ago

  • Status changed from New to Feedback
  • % Done changed from 50 to 100

I added some more Doxygen documentation to the GCTAEdisp2D class (and also the other energy dispersion classes) to clarify the mathematics. From my side this issue is done (well, code still needs to be merged in). I put the status to Feedback.

#3 Updated by Specovius Andreas over 4 years ago

Knödlseder Jürgen wrote:

Note that the internal GCTAEdisp2D normalization still leads to a different magnitude of the energy dispersion, but that internal normalization is needed since the integral of the GCTAEdisp2D operator over log(E_obs) should be unity.
I guess that HAP fills the histograms so that the integral over the migration is unity, but the normalisation needed is that the integral over the log10 of reconstructed energy is unity.

So my conclusion is that the normalization is done correctly, and you should keep it to get the correct fit results.

Indeed the H.E.S.S. energy dispersion is normalized so that the integral over the migration axis is unity.
As you described, the internal ctools normalization may hence make sense!

The only point I am still a bit worried about is whether the integral algorithm works correctly concerning the double peak structure observed in the H.E.S.S. edisp and also concerning isolated nonzero bins.
Do you have experience about that or has this been tested yet?

#4 Updated by Knödlseder Jürgen over 4 years ago

I made a quick check of the file you sent me using the simple algorithm from above. Below the list where the differences from 1 are larger than 1%:

0.0 deg   0.07 TeV sum: 0.025961
0.0 deg   0.08 TeV sum: 1.197912
0.0 deg   0.09 TeV sum: 1.295726
0.0 deg   0.10 TeV sum: 1.068557
0.0 deg   0.13 TeV sum: 1.020230
0.0 deg  95.32 TeV sum: 1.012170
0.5 deg   0.07 TeV sum: 0.025961
0.5 deg   0.08 TeV sum: 1.197912
0.5 deg   0.09 TeV sum: 1.295726
0.5 deg   0.10 TeV sum: 1.068557
0.5 deg   0.13 TeV sum: 1.020230
0.5 deg  95.32 TeV sum: 1.012170
1.0 deg   0.07 TeV sum: 0.025961
1.0 deg   0.08 TeV sum: 1.197912
1.0 deg   0.09 TeV sum: 1.295726
1.0 deg   0.10 TeV sum: 1.068557
1.0 deg   0.13 TeV sum: 1.020230
1.0 deg  95.32 TeV sum: 1.012170
1.5 deg   0.07 TeV sum: 0.025961
1.5 deg   0.08 TeV sum: 1.197912
1.5 deg   0.09 TeV sum: 1.295726
1.5 deg   0.10 TeV sum: 1.068557
1.5 deg   0.13 TeV sum: 1.020230
1.5 deg  95.32 TeV sum: 1.012170
2.0 deg   0.07 TeV sum: 0.025961
2.0 deg   0.08 TeV sum: 1.197912
2.0 deg   0.09 TeV sum: 1.295726
2.0 deg   0.10 TeV sum: 1.068557
2.0 deg   0.13 TeV sum: 1.020230
2.0 deg  95.32 TeV sum: 1.012170
2.5 deg   0.07 TeV sum: 0.025961
2.5 deg   0.08 TeV sum: 1.197912
2.5 deg   0.09 TeV sum: 1.295726
2.5 deg   0.10 TeV sum: 1.068557
2.5 deg   0.13 TeV sum: 1.020230
2.5 deg  95.32 TeV sum: 1.012170

So there are differences up to almost 30%, but they occur at very low energy. The plot below shows that those arise when only a few bins are set in the histogram.

#5 Updated by Knödlseder Jürgen about 4 years ago

  • Status changed from Feedback to Closed

Also available in: Atom PDF