Feature #1712
Create tool to compute systematic errors by bracketing IRFs
| Status: | New | Start date: | ||
|---|---|---|---|---|
| Priority: | Normal | Due date: | ||
| Assigned To: | - | % Done: | 0% | |
| Category: | - | |||
| Target version: | - | |||
| Duration: |
Description
To compute systematic errors in Fermi-LAT, it is common to use a method where IRFs are bracketed. This means, the IRFs (e.g the effective area) are varied by a certain percentage while computing the impact on the source parameters.
We could add a tool,e g. ctbracket, ctirfbrack, or ctvarirf (or similar), that has e.g. the following interface:
$ ctvarirf Input event list or observation definition XML file [selected_obs.xml] Input model XML file [fitted_models.xml] Source name [Crab] Effective area variation (%) [5] PSF variation (%) [5] Energy dispersion variation (%) [5] Background variation (%) [5]
The script would then use e.g. the
GCTAResponseTable::scale function to internally scale the IRFs (one after the other in positive and negative direction) and redo the fit. At the end, the script could print a summary about the variations on the source parameters from individual IRFs. And of course a total systematic uncertainty (the square root of the sum of errors^2). Of course we could also think about storing this information somehow in a file.Eventually for CTA, the variation parameters could be provided by the consortium (e.g. MC group, and maybe even become energy-dependent).
Recurrence
No recurrence.
History
#1
Updated by Knödlseder Jürgen about 8 years ago
I like the idea.
Note that in Fermi-LAT we only modify the effective area IRFs, but that this modification is energy dependent. We should also think about an energy dependent method. Maybe we could encore the information about IRF systematics in an XML file and simply pass such a file as the argument to the method. Maybe the systematic uncertainties are relatively static, hence only a single XML file needs to be provided (but the systematics can of course also depend on the zenith angle, for example).
#2
Updated by Knödlseder Jürgen about 8 years ago
P.S. I just would use a more explicit name for the tool, for example ctsyserr or ctsyserror.
#3
Updated by Mayer Michael about 8 years ago
Using an XML file as input for the dependence of the systematic sounds reasonable. For an energy dependent scaling of the effective area we could for instance use the GModelSpectralNodes class. The user could simply provide energy dependent scaling factors (i.e. bracketing values) in the XML file. Of course, this could be generalised to an XML interface for the GNodeArray class.
It makes total sense to bracket the effective area (probably the simplest IRF component). I guess also rather straight forward would be the background scaling. For the PSF, its parameters could also simply be scaled (we noticed in HESS that a scaling of 10% can also have quite some impact on the source flux (for a bright source like the Crab). The only component which seems not so easy is the energy dispersion as we store already a PDF. I guess a scaling here would mean shifting the “MIGRA” column, i.e. shifting reco vs true energy.
How is the energy-dependent bracketing of the effective area done in Fermi? Is there a public guideline how this should be done?
P.S. I just would use a more explicit name for the tool, for example ctsyserr or ctsyserror.
I was actually hesitant to use “sys error” in the name since it indicates that this tool would provide the full systematic error which might be misleading (in particular to unexperienced users). However, for a full estimate of the systematic error on the source flux a much deeper study would be necessary, right?