GammaLib

The actual Levenberg-Marquard optimizer needs still some tuning.

For example, sometimes the optimizer stalls for many iteration, and finally stops with the same values it had at the beginning of the stall. This is not very CPU efficient. The algorithm has to be studied to better understand where these stalls come from.

Maybe related to this, the gradients are actually not used in the stopping criterion, but sometimes, gradients become very small, and the algorithm will start to oscillate around the minimum due to numerical fluctuations. This could be prevented by stopping once the gradients are below a certain threshold. Alternatively, one could set small gradients to zero to avoid fluctuations of the algorithm due to these gradients.

A study should therefore be conducted that leads to a better understanding of the numerical properties of the Levenberg-Marquard optimizer.