GammaLib

While looking into the simulated data challenge, I’ve noticed that it takes a very long time to loop through all of the data files. Particularly, it has taken over 40 minutes to run csobsselect on my machine when passing it the full list of observations. I’ve tracked this speed issue down to the reading in of the FITS header keywords, specifically the keywords associated with the Monte Carlo source name and ID (MMN##### and MID#####). Since there are over 1500 models and each observation contains the full list of source names and IDs, this read in takes a very long time.

Currently, the keyword values from a FITS file are stored in the GFitsHeader class as a vector of GFitsHeaderCard objects. This means that when a given keyword is needed the class has to loop through the entire vector until it finds the desired value. For most implementations where there are only a few simulated models in an observation this is reasonably fast, however in the event that a given observation has MANY keywords (as in the case above) this can slow down the loading of observation files. Changing the storage of the keywords from 'std::vector<GFitsHeaderCard>' to 'std::map<std::string, GFitsHeaderCard>' has the potential to speed up the lookup time, since std::map stores a sorted version of its entries and uses a binary search algorithm to lookup items.

After talking with Jürgen, he has changed how the observations store the simulated source names and IDs so that each observation FITS file only contains header keywords associated with the models actually simulated within that observation. This will probably dramatically improve the time issue I’m seeing.