Bug #3889

Fix COMPTEL response computation for extended models

Added by Knödlseder Jürgen about 3 years ago. Updated over 2 years ago.

Status:ClosedStart date:10/27/2021
Priority:NormalDue date:
Assigned To:Knödlseder Jürgen% Done:

100%

Category:-
Target version:2.0.0
Duration:

Description

Following the analysis described on https://cta-redmine.irap.omp.eu/projects/comptel/wiki/Carina_analysis there are some issues with the convergence for radial disk models. It appears that for small initial disk radii the initial parameter values are little changed, while for larger radii they change. This issue also follows up on #2973.

test_likelihood_profile_RA.png (113 KB) Knödlseder Jürgen, 10/29/2021 05:11 PM

test_likelihood_profile_Radius.png (92.2 KB) Knödlseder Jürgen, 10/29/2021 11:11 PM

likelihood-profile-new.png (95.1 KB) Knödlseder Jürgen, 11/22/2021 08:47 AM

Test_likelihood_profile_ra Test_likelihood_profile_radius Likelihood-profile-new

Recurrence

No recurrence.

History

#1 Updated by Knödlseder Jürgen about 3 years ago

I made some likelihood profiles for the 11 bin Eta Carinae analysis (see https://cta-redmine.irap.omp.eu/projects/comptel/wiki/Carina_analysis) and they clearly show that there is a noise problem with the likelihood profiles that obviously prevents convergence of the maximum likelihood optimiser. Consequently, the algorithm used for the radial model computation should be reinvestigated.

#3 Updated by Knödlseder Jürgen almost 3 years ago

I implemented the extended model response convolution in the system of the model, which is similar to the method that is implemented for CTA. This stabilised the model fit and led to smooth likelihood profiles, as illustrated below

#4 Updated by Knödlseder Jürgen almost 3 years ago

I redid the fitting of the Crab using the new implementation. I used an initial extension of 0.1° for the tests. Initially I implemented a common extended method for radial and elliptical models, yet I switched the to a specific radial method for speed reasons.

Implementation logL TS RA Dec Extension Prefactor (1e-5) Index CPU (sec)
Old -67047.046 1081.959 83.78 ± 0.12 21.66 ± 0.11 1.10 ± 0.45 178.83 ± 9.29 -2.147 ± 0.037 ?
New (extended) -67044.586 1076.430 83.79 ± 0.12 21.66 ± 0.11 1.15 ± 0.43 178.55 ± 9.28 -2.149 ± 0.037 1513.17
New (radial) -67044.586 1076.430 83.79 ± 0.12 21.66 ± 0.11 1.15 ± 0.43 178.55 ± 9.28 -2.149 ± 0.037 1209.85

#5 Updated by Knödlseder Jürgen almost 3 years ago

  • % Done changed from 50 to 60

Here the results with the final code as function of initial radial disk model extent:

Initial extent logL TS RA Dec Extension Prefactor (1e-5) Index CPU (sec)
0.1 -67044.586 1076.430 83.79 ± 0.12 21.66 ± 0.11 1.15 ± 0.43 178.55 ± 9.28 -2.149 ± 0.037 1209.85
1.0 -67044.610 1081.130 83.79 ± 0.12 21.66 ± 0.11 1.18 ± 0.42 179.14 ± 9.29 -2.149 ± 0.037 1116.22
5.0 -67044.477 1069.184 83.79 ± 0.12 21.66 ± 0.11 1.13 ± 0.44 177.13 ± 9.25 -2.146 ± 0.037 1137.58

While the fit results are very close, yet not identical, there is a significant difference in the TS values of the results.

#6 Updated by Knödlseder Jürgen almost 3 years ago

I redid the same analysis using navgr=3 and nincl=13:

Initial extent logL TS RA Dec Extension Prefactor (1e-5) Index CPU (sec)
0.1 -67574.643 1105.155 83.812 ± 0.12 21.63 ± 0.11 1.24 ± 0.40 184.91 ± 9.39 -2.167 ± 0.037 833.48
1.0 -67574.715 1107.232 83.811 ± 0.12 21.63 ± 0.11 1.23 ± 0.41 185.07 ± 9.39 -2.168 ± 0.037 801.59
5.0 -67574.782 1109.803 83.811 ± 0.12 21.63 ± 0.11 1.23 ± 0.40  185.51 ± 9.40 -2.169 ± 0.037 1063.94

This reduced the variation of the TS value and also stabilised the final source extension. I also did an analysis using navgr=3 and nincl=5 to investigate whether the change to navgr=3 can explain the stabilisation:

Initial extent logL TS RA Dec Extension Prefactor (1e-5) Index CPU (sec)
0.1 -70446.727 988.203 83.77 ± 0.12 21.67 ± 0.11 0.71 ± 0.67 172.94 ± 9.19 -2.187 ± 0.039 1863.39
1.0 -70446.725 990.848 83.77 ± 0.12 21.67 ± 0.11 0.69 ± 0.69 173.24 ± 9.18 -2.188 ± 0.039 1807.3
5.0 -70446.724 978.498 83.77 ± 0.12 21.67 ± 0.11 0.68 ± 0.70  171.71 ± 9.16 -2.185 ± 0.039 2041.49

There is still some variability in TS, hence the stabilisation seems more to come from the change to nincl=13. Interestingly, the extension is smaller for navgr=3 compared to navgr=5 which calls for some parametric exploration. This was all done with an initial extent of 5.0 to be off the best-fitting values.

navgr nincl logL TS RA Dec Extension Prefactor (1e-5) Index CPU (sec)
3 5 -70446.724 978.498 83.77 ± 0.12 21.67 ± 0.11 0.68 ± 0.70 171.71 ± 9.16 -2.185 ± 0.039 2041.49
3 7 -69267.790 1067.124 83.83 ± 0.12 21.66 ± 0.11 1.04 ± 0.48 184.90 ± 9.43 -2.199 ± 0.038 1295.11
3 9 -68458.036 1117.489 83.85 ± 0.12 21.65 ± 0.11 1.20 ± 0.41 188.26 ± 9.45 -2.185 ± 0.037 1076.23
3 11 -67924.532 1120.166 83.83 ± 0.12 21.63 ± 0.10 1.22 ± 0.41 186.44 ± 9.40 -2.171 ± 0.037 1072.88
3 13 -67574.782 1109.803 83.81 ± 0.12 21.63 ± 0.11 1.23 ± 0.40 185.51 ± 9.40 -2.169 ± 0.037 1063.94
3 15 -67259.101 1121.117 83.80 ± 0.12 21.61 ± 0.11 1.27 ± 0.39 187.92 ± 9.44 -2.175 ± 0.037 872.02
3 17 -67002.840 1120.663 83.79 ± 0.12 21.61 ± 0.11 1.22 ± 0.41 186.94 ± 9.41 -2.171 ± 0.037 831.56
3 19 -66850.621 1120.838 83.77 ± 0.12 21.61 ± 0.11 1.22 ± 0.41 187.78 ± 9.44 -2.175 ± 0.037 937.08
5 5 -67044.477 1069.184 83.79 ± 0.12 21.66 ± 0.11 1.13 ± 0.44 177.13 ± 9.25 -2.146 ± 0.037 1137.58
5 7 -66664.677 1112.682 83.82 ± 0.12 21.65 ± 0.11 1.25 ± 0.40 186.69 ± 9.45 -2.171 ± 0.037 1039.57
5 9 -66391.486 1120.405 83.84 ± 0.12 21.64 ± 0.10 1.23 ± 0.40 188.29 ± 9.48 -2.177 ± 0.037 1083.66
5 11 -66185.036 1123.623 83.82 ± 0.12 21.63 ± 0.10 1.24 ± 0.40 187.18 ± 9.44 -2.169 ± 0.037 1083.93

#7 Updated by Knödlseder Jürgen almost 3 years ago

  • % Done changed from 60 to 70

I now analysed the Crab data for different spatial models to check if they give consistent results. I switched back to navgr=5 and nincl=5 for this analysis. I used the default value for the initial extent.

Model logL TS RA Dec Extension Prefactor (1e-5) Index CPU (sec)
Point source -67045.600 1004.167 83.79 ± 0.11 21.63 ± 0.10 167.64 ± 8.56 -2.149 ± 0.038 ?
Disk -67044.610 1081.130 83.79 ± 0.12 21.66 ± 0.11 1.18 ± 0.42 179.14 ± 9.29 -2.149 ± 0.037 1116.22
Gauss -67044.718 1100.844 83.79 ± 0.12 21.65 ± 0.11 0.62 ± 0.21 181.32 ± 9.37 -2.150 ± 0.037  1118.39
Elliptical disk -67043.742 1027.716 83.78 ± 0.12 21.69 ± 0.10 90*, 1.23 ± 0.11, 0.46 ± 46287.68 178.49 ± 18014520.40 -2.151 ± 0.038 8225.75
Elliptical Gauss -67045.211 1127.502 83.78 ± 0.11 21.65 ± 0.11 145.09 ± 15.99, 1.14 ± 0.74, 0.31 ± 0.77 196.44 ± 75.87 -2.132 ± 0.035 4212.23

*Fitting the elliptical disk model gave the notification Parameter "PA" has zero curvature. Fix parameter. and consequently the position angle was not fitted. This all led to large errors for the semi-minor axis value and the flux. Maybe the model extension was too small to give a non-zero gradient.

#8 Updated by Knödlseder Jürgen over 2 years ago

  • Status changed from In Progress to Closed
  • % Done changed from 70 to 100

The response computation seems to work now, close the issue.

Also available in: Atom PDF