Action #2712

Add H.E.S.S. DR1 analysis tutorials

Added by Knödlseder Jürgen about 6 years ago. Updated almost 6 years ago.

Status:ClosedStart date:11/05/2018
Priority:NormalDue date:
Assigned To:Tibaldo Luigi% Done:

100%

Category:-
Target version:1.6.0
Duration:

Description

User tutorials should be added to explain how to analyse the H.E.S.S. DR1 data. Preferentially, these tutorials should be in form of Jupyter notebooks.

residualmap.png (48 KB) Tibaldo Luigi, 11/06/2018 05:46 PM

residuals_0.png (15.6 KB) Tibaldo Luigi, 11/06/2018 05:46 PM

residuals_1.png (15 KB) Tibaldo Luigi, 11/06/2018 05:46 PM

residuals_2.png (15.3 KB) Tibaldo Luigi, 11/06/2018 05:46 PM

residuals_3.png (15.1 KB) Tibaldo Luigi, 11/06/2018 05:46 PM

residuals_stacked.png (12.7 KB) Tibaldo Luigi, 11/06/2018 05:46 PM

residualmap_new.png (42.9 KB) Tibaldo Luigi, 11/07/2018 12:38 PM

residuals_new.png (15.8 KB) Tibaldo Luigi, 11/07/2018 12:38 PM

residuals_fixed.png (16.4 KB) Tibaldo Luigi, 11/07/2018 02:23 PM

Residualmap Residuals_0 Residuals_1 Residuals_2 Residuals_3 Residuals_stacked Residualmap_new Residuals_new Residuals_fixed

Recurrence

No recurrence.

History

#1 Updated by Tibaldo Luigi about 6 years ago

  • Status changed from New to In Progress
  • Assigned To set to Tibaldo Luigi
Some material already exists. This are the priorities to address:
  • revise unbinned analysis to use csbkgmodel
  • complete classical analysis (two ways to treat background, WSTAT or csbkgmodel)
  • stacked analysis (workflow to run using csbkgmodel)
  • joint HESS/Fermi analysis
  • do we need to treat separately analysis of extended sources?

#2 Updated by Tibaldo Luigi about 6 years ago

I also redid event selection as jupyter notebook to have a consistent workflow.
I am making also small modifications to gammalib (so far only to fix a typo)

#3 Updated by Tibaldo Luigi about 6 years ago

  • % Done changed from 0 to 10

Refashioned tutorials on data selection, skymap generation/source finding, and classical analysis as notebooks. Added tutorial on csbkgmodel.
Along the way also:

  • fixed problem in initialisation of cssrcdetect
  • fixed reference manual of csbkgmodel

Classical analysis needs to be completed with non-WSTAT background as soon as available.

#4 Updated by Tibaldo Luigi about 6 years ago

Added Jupyter tutorial for unbinned analysis. The results from the fit look correct, but the stacked residuals are not consistent. The stacked spectral residuals have large negative deficit near the threshold

although non-stacked spectral residuals are very good




Also the residual map looks biased toward negative residuals.

This is for the energy range 0.66-100 TeV, and accounting for energy dispersion. Neglecting energy dispersion leads to very similar results.

#5 Updated by Tibaldo Luigi about 6 years ago

We understood a bit better the issue with residuals:

  • for csresmap it is related to the intrinsic approximations of the Significance formulae for low-counts regime, the tutorial will be done with SUBDIV and a warning will be added to the reference manuals
  • for csresspec currently the combination of unbinned observations passes through the stacked response; I will change it to do this directly from the IRFs using ctmodel

#6 Updated by Tibaldo Luigi about 6 years ago

The changes above make the residuals consistent.
Residual map (with 0.1 deg smoothing)

#7 Updated by Tibaldo Luigi about 6 years ago

There was obviously a problem with the computation of individual components in the spectral residuals. I fixed this in csresspec

I would propose to merge in devel what is available so far.

#8 Updated by Knödlseder Jürgen about 6 years ago

I merged the documentation into devel.

I see some strange warnings containing a path on your computer at http://cta.irap.omp.eu/ctools-devel/users/tutorials/hess_dr1/skymap.html.

You will also see that the Jupyter notebook numbers are not starting from [1]. I guess once the notebooks are finished you have them run cleanly from the beginning and then save the files, which will then conserve the numbering system for when the documentation is built.

#9 Updated by Tibaldo Luigi about 6 years ago

thanks, I made a commit to solve the problems above.

#10 Updated by Tibaldo Luigi about 6 years ago

  • % Done changed from 40 to 80

I updated the classical analysis tutorial based on the latest modifications and also to include an example of classical analysis with background model.
In the process I also modified the dummy source creation in csphagen so that, if a source model is not provided, the dummy source is initialised with reasonable parameter values.
Note I moved the development to a new branch '2712_HESSDR1_tutorials_new’ branched from devel yesterday (to solve a bizarre conflict issue)

#11 Updated by Tibaldo Luigi about 6 years ago

  • Status changed from In Progress to Pull request
  • % Done changed from 80 to 100

Finalised tutorial of joint Fermi/HESS analysis.
Ready for integration from my side.

#12 Updated by Tibaldo Luigi about 6 years ago

  • Status changed from Pull request to In Progress
  • % Done changed from 100 to 90

The Fermi observatory released an updated dataset (P8R3). I will modify the tutorial using this most up to date dataset.

#13 Updated by Tibaldo Luigi almost 6 years ago

  • Status changed from In Progress to Pull request
  • % Done changed from 90 to 100

I updated the joint Fermi-HESS tutorial using P8R3 data. Ready for integration from my side.

#14 Updated by Tibaldo Luigi almost 6 years ago

  • Status changed from Pull request to In Progress
  • % Done changed from 100 to 90

Sorry, I forgot again to run all the notebooks to have a clean cell numbering, I will do and resend the pull request later.

#15 Updated by Tibaldo Luigi almost 6 years ago

  • Status changed from In Progress to Pull request
  • % Done changed from 90 to 100

Done, can merge now!

#16 Updated by Knödlseder Jürgen almost 6 years ago

  • Status changed from Pull request to Closed

The tutorial was merged into the devel branch.

Also available in: Atom PDF