Bug #2266
Verify computation of WSTAT in GCTAOnOffObservation::likelihood_wstat
| Status: | Closed | Start date: | 10/28/2017 | |
|---|---|---|---|---|
| Priority: | Normal | Due date: | ||
| Assigned To: |  Knödlseder Jürgen | % Done: | 100% | |
| Category: | - | |||
| Target version: | 1.5.0 | |||
| Duration: |
Description
The pull distribution obtained with the WSTAT statistic seems to be to broad, the fit results appear however relatively unbiased (see #2263).
The formulae for the WSTAT should be verified to correct this issue (is there for example somewhere a factor of 2 in the log-likelihood that could lead to too large errors?)
PrefactorHist_newwstat_b05.png (40.7 KB)
Index_newwstat_b05.png (44.4 KB)
IndexHist_newwstat_b05.png (38.8 KB)
Prefactor_newwstat_b05.png (42.5 KB)
PrefactorHist_newwstat_b10.png (40.3 KB)
IndexHist_newwstat_b10.png (39.3 KB)
Prefactor_newwstat_b10.png (42.5 KB)
Index_newwstat_b10.png (44.4 KB)
PrefactorHist_newwstat_0.05-100_b40.png (40.9 KB)
Prefactor_newwstat_0.05-100_b40.png (48.3 KB)
Index_newwstat_0.05-100_b40.png (33.3 KB)
IndexHist_newwstat_0.05-100_b40.png (46.5 KB)
Recurrence
No recurrence.
History
#1
Updated by Knödlseder Jürgen over 6 years ago
- File PrefactorHist_newwstat_b05.png added
- File Index_newwstat_b05.png added
- File IndexHist_newwstat_b05.png added
- File Prefactor_newwstat_b05.png added
- Status changed from New to In Progress
- Assigned To set to Knödlseder Jürgen
- % Done changed from 0 to 50
The formulae for the derivative in the general case were not correct. After correcting the formula things seems to be okay. Here an example for 5 energy bins between 100 GeV and 100 TeV.
 |
 |
 |
 |
#2
Updated by Knödlseder Jürgen over 6 years ago
- File PrefactorHist_newwstat_b10.png added
- File IndexHist_newwstat_b10.png added
- File Prefactor_newwstat_b10.png added
- File Index_newwstat_b10.png added
- % Done changed from 50 to 60
Here the same for 10 bins. Also looks okay.
 |
 |
 |
 |
#3
Updated by Knödlseder Jürgen over 6 years ago
- File PrefactorHist_newwstat_0.05-100_b40.png added
- File Prefactor_newwstat_0.05-100_b40.png added
- File Index_newwstat_0.05-100_b40.png added
- File IndexHist_newwstat_0.05-100_b40.png added
- Status changed from In Progress to Closed
- % Done changed from 60 to 100
I also checked the pull distributions for 40 bins and an energy range from 50 GeV to 100 TeV. Now the Prefactor seems to be slightly biased and too broad. I double checked all formulae and did not find any error. I guess this behaviour comes from the fact that in many cases the number of On or Off events per bin is zero and the special cases where the background estimate is set to zero is encountered.
 |
 |
 |
 |
I don’t think that we can do something about this, it seems to be an inherent limitation of the statistic.
The Xspec page (https://heasarc.nasa.gov/xanadu/xspec/manual/XSappendixStatistics.html) mentions by the way:
In practice, it works well for many cases but for weak sources can generate an obviously wrong best fit. It is not clear why this happens although binning to ensure that every bin contains at least one count often seems to fix the problem.