Action #3316

Implement a specific response cache for diffuse models and binned data

Added by Knödlseder Jürgen over 3 years ago. Updated over 3 years ago.

Status:ClosedStart date:08/07/2020
Priority:HighDue date:
Assigned To:Knödlseder Jürgen% Done:

100%

Category:-
Target version:2.0.0
Duration:

Description

In case that a diffuse model should be used for binned data, the cache will cover a large faction (if not all) the data space. A specific response caching should be implemented for that case so that the overhead related to storing of the cached values in a std::map structure can be avoided.


Recurrence

No recurrence.


Related issues

Related to GammaLib - Action #3404: Implement a sparse vector cache in GResponse Closed 10/17/2020

History

#2 Updated by Knödlseder Jürgen over 3 years ago

  • Status changed from New to In Progress
  • Assigned To set to Knödlseder Jürgen
  • % Done changed from 0 to 10

I just tested (on my Mac) that it was possible to overload the virtual base class method GResponse::irf_spatial by a method GCTAResponseCube::irf_spatial without the need to recompile ctools. Below the output of valgrind --tool=massif ctmodel that demonstrates that the overloaded method is actually used:

->27.68% (68,584,320B) 0x100168AFC: std::__1::map<unsigned long long, double, std::__1::less<unsigned long long>, std::__1::allocator<std::__1::pair<unsigned long long const, double> > >::operator[](unsigned long long const&) (in /usr/local/gamma/lib/libgamma.8.dylib)
| ->27.68% (68,584,320B) 0x100168BE9: GResponseCache::set(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, GInstDir const&, GEnergy const&, GEnergy const&, double const&) (in /usr/local/gamma/lib/libgamma.8.dylib)
|   ->27.68% (68,584,320B) 0x10026B7A5: GCTAResponseCube::irf_spatial(GEvent const&, GSource const&, GObservation const&) const (in /usr/local/gamma/lib/libgamma.8.dylib)
|     ->27.68% (68,584,320B) 0x1001655CA: GResponse::eval_prob(GModelSky const&, GEvent const&, GEnergy const&, GTime const&, GObservation const&, bool const&) const (in /usr/local/gamma/lib/libgamma.8.dylib)

This opens up the possibility to add a kludge to release 1.7.1 that implements a memory-efficient storage of response cubes for stacked analysis. I will implement this now, leaving a more general interface change to release 1.8.0.

#3 Updated by Knödlseder Jürgen over 3 years ago

  • % Done changed from 10 to 50

I implemented the new caching mechanism and it actually works without recompilation. Here is the memory usage compared for the old and the new caching. As expected, the new caching takes just the size of a data-space cube, the reduction factor is (at least) a factor of 6.

Old cache:
->27.68% (68,584,320B) 0x10026B7A5: GCTAResponseCube::irf_spatial(GEvent const&, GSource const&, GObservation const&) const (in /usr/local/gamma/lib/libgamma.8.dylib)

New cache:
->06.38% (11,430,720B) 0x10026BDD5: GCTAResponseCube::irf_spatial(GEvent const&, GSource const&, GObservation const&) const (in /usr/local/gamma/lib/libgamma.8.dylib)

#4 Updated by Knödlseder Jürgen over 3 years ago

  • Priority changed from High to Normal

Put the priority to Normal for release 1.8.0.

#5 Updated by Knödlseder Jürgen over 3 years ago

  • Priority changed from Normal to High

I tried to move the modification to the level of GCTAResponse, but this led to segfaults when running ctmodel. Apparently this touches too much the interface so that cross-linking is not possible anymore. The relevant code is in https://cta-gitlab.irap.omp.eu/gammalib/gammalib/tree/3316-cta-stacked-cache-kludge for the record. It should fold in a release 1.8.0 interface change.

#6 Updated by Knödlseder Jürgen over 3 years ago

  • Related to Action #3404: Implement a sparse vector cache in GResponse added

#7 Updated by Knödlseder Jürgen over 3 years ago

  • Status changed from In Progress to Closed
  • % Done changed from 50 to 100

This issue becomes obsolete for the stacked analysis since the introduction of the vector response. The caching is now covered by a sparse vector cache, see #3404.

Also available in: Atom PDF