Action #2752
Check On/Off analysis for H.E.S.S. DR1
| Status: | Closed | Start date: | 11/14/2018 | |
|---|---|---|---|---|
| Priority: | Normal | Due date: | ||
| Assigned To: |  Knödlseder Jürgen | % Done: | 100% | |
| Category: | - | |||
| Target version: | 1.6.0 | |||
| Duration: |
residual_joint.png (26.2 KB)
residual_stack_clipping_nonorm.png (22.2 KB)
residual_stack_clipping.png (26.3 KB)
residual_stack_noclipping.png (26.2 KB)
Recurrence
No recurrence.
Related issues
History
#1
Updated by Knödlseder Jürgen over 5 years ago
- Status changed from New to In Progress
- % Done changed from 0 to 10
Here the H.E.S.S. analysis results for On/Off analysis using 40 reconstructed energy bins from 0.67-30 TeV, compared to the results for unbinned and stacked analysis. The wstat statistic was used. 300 true energy bins between 0.1-100 TeV were used, the On region was centred on Right Ascension of 83.633 deg and Declination of 22.0145 deg with a radius of 0.2 deg.
| Analysis | Edisp | logL | TS | Prefactor | Index | CPU |
| Unbinned | Yes | 98196.591 | 2030.800 | 4.148e-17 +/- 2.005e-18 | 2.734 +/- 0.070 | 116.4 |
| Stacked | Yes | 54059.739 | 1874.717 | 3.918e-17 +/- 2.022e-18 | 2.698 +/- 0.076 | 2737.0 |
| Joint On/Off | Yes | 78.972 | 1126.106 | 4.458e-17 +/- 2.539e-18 | 2.626 +/- 0.074 | 0.1 |
| Stacked On/Off | Yes | 24.074 | 1349.707 | 4.507e-17 +/- 2.704e-18 | 2.612 +/- 0.077 | 0.0 |
#2
Updated by Knödlseder Jürgen over 5 years ago
And here the impact of disabling the clipping in true energy:
| Analysis | logL | TS | Prefactor | Index | CPU |
| Joint On/Off | 78.972 | 1126.106 | 4.458e-17 +/- 2.539e-18 | 2.626 +/- 0.074 | 0.1 |
| Stacked On/Off (clipping) | 24.074 | 1349.707 | 4.507e-17 +/- 2.704e-18 | 2.612 +/- 0.077 | 0.0 |
| Stacked On/Off (no clipping) | 23.206 | 1351.443 | 4.558e-17 +/- 2.531e-18 | 2.623 +/- 0.072 | 0.0 |
#3
Updated by Knödlseder Jürgen over 5 years ago
- % Done changed from 10 to 20
Here the results when fitting the background model using CSTAT (no clipping is implemented in the code anymore). The background model produces using csbkgmodel was used. See #2711 for the adaption of the code that makes this possible.
| Analysis | logL | TS | Prefactor | Index | CPU |
| Joint On/Off | 2022.488 | 988.621 | 4.364e-17 +/- 2.548e-18 | 2.633 +/- 0.078 | 0.6 |
| Stacked On/Off | 3860.357 | 1105.846 | 4.407e-17 +/- 2.543e-18 | 2.633 +/- 0.076 | 0.1 |
Note that some hand-tweeking of the model was needed. When running csphagen, the instrument in the model is HESS, but we need a model with HESSOnOff observation for model fitting. Hence we should add an output model to csphagen that produces a model that can be readily fitted using ctlike (see #2711).
#4
Updated by Knödlseder Jürgen over 5 years ago
- Related to Action #2711: Adapt cscripts to support any kind of background model added
#5
Updated by Knödlseder Jürgen over 5 years ago
- File residual_joint.png added
- File residual_stack_clipping_nonorm.png added
- File residual_stack_clipping.png added
- File residual_stack_noclipping.png added
To understand whether the response should be clipped or not I inspected the residual spectra for a simulation of Crab observations with different energy thresholds. Below a few plots for different analysis configurations.
The first plot is for a joint analysis, hence clipping does not impact the results since the data are not stacked.
The second plot shows the residuals for a stacked analysis where the ARF and RMF where clipped, but the RMF was not re-normalized. A big residual appears.
The third plot is for a stacked analysis where the ARF and RMF where clipped and the RMF was renormalized. This makes the residual disappear.
The forth plot is for a stacked analysis without clipping. In details the residuals are closer to the joint residuals, hence the ARF and RMF should not be clipped, consistent with the stacked 3D analysis.
| Joint |  |
| Stacked, no normalization of RMF, clipping |  |
| Stacked, clipping |  |
| Stacked, no clipping |  |
#6
Updated by Knödlseder Jürgen over 5 years ago
- % Done changed from 20 to 50
I also checked the On/Off analysis for the RX J1713.7-3946 observation. The tricky thing here is that the emission is quite extended, and if a On region radius of 0.8 deg is chosen, no Off regions exist. I therefore reduced the On region radius to 0.5 deg, which resulted in Off regions for 4 observations.
The results of the On/Off analysis for these 4 observations are listed below, where they are compared to the unbinned and stacked analysis results without energy dispersion.
| Analysis | logL | TS | Prefactor | Index | Cutoff |
| Unbinned | 537778.100 | 741.007 | 2.025e-17 +/- 1.983e-18 | 1.925 +/- 0.118 | 5.569 +/- 2.057 |
| Stacked | 212467.591 | 688.935 | 1.776e-17 +/- 1.483e-18 | 1.931 +/- 0.108 | 7.092 +/- 2.612 |
| Joint, cstat | 13964.036 | 132.212 | 1.746e-17 +/- 2.139e-18 | 1.923 +/- 0.180 | 16.002 +/- 10.444 |
| Stacked, cstat | 20490.631 | 110.990 | 2.017e-17 +/- 3.502e-18 | 1.773 +/- 0.254 | 6.843 +/- 4.821 |
| Joint, wstat | 72.500 | 149.492 | 2.603e-17 +/- 6.593e-18 | 1.787 +/- 0.309 | 4.523 +/- 3.617 |
| Stacked, wstat | 22.311 | 163.271 | 2.215e-17 +/- 2.763e-18 | 1.956 +/- 0.181 | 11.185 +/- 7.684 |
For comparison also the results for a power law fit.
| Analysis | logL | TS | Prefactor | Index |
| Joint, cstat | 13962.781 | 129.697 | 1.568e-17 +/- 1.690e-18 | 2.158 +/- 0.108 |
| Stacked, cstat | 20488.540 | 106.807 | 1.602e-17 +/- 1.666e-18 | 2.182 +/- 0.108 |
| Joint, wstat | 75.526 | 143.440 | 1.881e-17 +/- 1.638e-18 | 2.257 +/- 0.102 |
| Stacked, wstat | 23.804 | 160.285 | 1.910e-17 +/- 1.634e-18 | 2.216 +/- 0.090 |
Overall the results look good. I therefore consider the On/Off analysis as validated.
#7
Updated by Knödlseder Jürgen over 5 years ago
- Status changed from In Progress to Closed
- % Done changed from 50 to 100