Action #1301
Improve LM optimizer for the case of many stall close to optimum
| Status: | Closed | Start date: | 07/27/2014 | |
|---|---|---|---|---|
| Priority: | Normal | Due date: | ||
| Assigned To: |  Knödlseder Jürgen | % Done: | 100% | |
| Category: | - | |||
| Target version: | 00-09-00 | |||
| Duration: |
Description
Fitting spatial parameters for CTA gives often optimizer sequences like
>Iteration 0: -logL=34182.371, Lambda=1.0e-03 >Iteration 1: -logL=34180.081, Lambda=1.0e-03, delta=2.290, max(|grad|)=20.841878 [DEC:1] >Iteration 2: -logL=34180.070, Lambda=1.0e-04, delta=0.011, max(|grad|)=5.453326 [RA:0] >Iteration 3: -logL=34180.069, Lambda=1.0e-05, delta=0.001, max(|grad|)=-1.169762 [RA:0] >Iteration 4: -logL=34180.069, Lambda=1.0e-06, delta=0.000, max(|grad|)=0.299899 [RA:0] Iteration 5: -logL=34180.069, Lambda=1.0e-07, delta=-0.000, max(|grad|)=-0.066802 [RA:0] (stalled) Iteration 6: -logL=34180.069, Lambda=1.0e-06, delta=-0.000, max(|grad|)=-0.066802 [RA:0] (stalled) Iteration 7: -logL=34180.069, Lambda=1.0e-05, delta=-0.000, max(|grad|)=-0.066799 [RA:0] (stalled) Iteration 8: -logL=34180.069, Lambda=1.0e-04, delta=-0.000, max(|grad|)=-0.066763 [RA:0] (stalled) Iteration 9: -logL=34180.069, Lambda=1.0e-03, delta=-0.000, max(|grad|)=-0.066408 [RA:0] (stalled) Iteration 10: -logL=34180.069, Lambda=1.0e-02, delta=-0.000, max(|grad|)=-0.062898 [RA:0] (stalled) Iteration 11: -logL=34180.069, Lambda=1.0e-01, delta=-0.000, max(|grad|)=-0.031152 [RA:0] (stalled) >Iteration 12: -logL=34180.069, Lambda=1.0e+00, delta=0.000, max(|grad|)=0.123998 [RA:0] Iteration 13: -logL=34180.069, Lambda=1.0e-01, delta=-0.000, max(|grad|)=-0.014693 [RA:0] (stalled) Iteration 14: -logL=34180.069, Lambda=1.0e+00, delta=-0.000, max(|grad|)=0.049271 [RA:0] (stalled) Iteration 15: -logL=34180.069, Lambda=1.0e+01, delta=-0.000, max(|grad|)=0.110592 [RA:0] (stalled) Iteration 16: -logL=34180.069, Lambda=1.0e+02, delta=-0.000, max(|grad|)=0.122543 [RA:0] (stalled) >Iteration 17: -logL=34180.069, Lambda=1.0e+03, delta=0.000, max(|grad|)=0.122399 [RA:0]
Here, the optimum is basically reached in iteration 4, from then on the process basically stalls, due to numerical noise in the derivatives (see #1299). The process stops when just “by chance” the right conditions are reached. The logic should be improved so that such sequences will be catched early on.
Recurrence
No recurrence.
History
#1
Updated by Knödlseder Jürgen over 9 years ago
- Status changed from New to Closed
- Assigned To set to Knödlseder Jürgen
- Target version set to 00-09-00
- % Done changed from 0 to 100
- Remaining (hours) set to 0.0
The easy solution here is to reduce the precision requirement for the log-likelihood. I set this now by default to 5e-3 in ctlike.