ctools: Developmenthttps://cta-redmine.irap.omp.eu/https://cta-redmine.irap.omp.eu/favicon.ico?14312453732016-08-13T09:10:24ZCTA IRAP Project Gateway
Redmine Development: RE: Unify ctools and cscriptshttps://cta-redmine.irap.omp.eu/boards/14/topics/239?r=241#message-2412016-08-13T09:10:24ZMayer Michaelmichael.mayer@physik.hu-berlin.de
<blockquote>
<p>What are the main drivers behind your proposal? Have you encountered problems with having the two variants of tools?</p>
</blockquote>
<p>No not at all. The main driver is the usability from a user perspective. I have received the question why some tools start with <code>cs...</code>, some with <code>ct...</code> and just when applying and using the tools the user doesn’t have to know the programming language behind it. Especially from Python, the user currently has to know if you need to do <code>import cscripts</code>. Intuitively one would expect from <code>import ctools</code> to have all ctools functionality available.</p>
<blockquote>
<p>Btw: there are small differences between ctools and cscripts which come for example from differences on how CTRL-C is handled in binaries and Python. The error messages are also a bit different.</p>
</blockquote>
<p>I agree they behave differently regarding CTRL+C and also how exceptions are raised. Probably the reason for two different types of tools has to be made more clear in the user documentation?<br />I am not in favour of one or the other way. I just wanted to bring it up here in order to have a discussion platform and to gather some opinions about it.</p> Development: RE: Unify ctools and cscriptshttps://cta-redmine.irap.omp.eu/boards/14/topics/239?r=240#message-2402016-08-12T16:33:36ZKnödlseder Jürgenjurgen.knodlseder@irap.omp.eu
<p>What are the main drivers behind your proposal? Have you encountered problems with having the two variants of tools?</p>
<p>Btw: there are small differences between ctools and cscripts which come for example from differences on how CTRL-C is handled in binaries and Python. The error messages are also a bit different.</p> Development: Unify ctools and cscriptshttps://cta-redmine.irap.omp.eu/boards/14/topics/2392016-08-12T14:26:31ZMayer Michaelmichael.mayer@physik.hu-berlin.de
From a user perspective, it shouldn't really matter wether an analysis tool is a <code>ctool</code> or a <code>cscript</code>. From command line or from python, these tools behave exactly the same and there is no reason to expose the programming language via the prefix to the user. Accordingly, I believe we could think about simply unifying the two.<br />The change would include:
<ul>
<li>rename all <code>cscripts</code> to <code>ct...</code>.</li>
<li>get all scripts and tools when using <code>import ctools</code> in Python.</li>
</ul>
<p>I can’t really foresee what would be the impact and effort of such a change regarding the build system and other parts of the software. Therefore, I opened up this forum entry for discussion. I just believe from a user perspective, the reason why some tools start with <code>ct...</code> and some with <code>cs...</code> might seem confusing it he long run.</p>
<p>Of course, I know that <code>cscripts</code> usually internally use some <code>ctools</code> for their analysis tasks which probably makes the simultaneous import more difficult.</p>
<p>Any opinions, ideas?</p> Development: RE: Workflow implementationhttps://cta-redmine.irap.omp.eu/boards/14/topics/237?r=238#message-2382016-02-23T12:41:28ZKnödlseder Jürgenjurgen.knodlseder@irap.omp.eu
Attached an IVOA document about scientific workflows. Here some interesting associated links:
<ul>
<li>ESO reflex: <a class="external" href="https://www.eso.org/sci/software/reflex/">https://www.eso.org/sci/software/reflex/</a></li>
<li>AstroGrid workflow system: <a class="external" href="http://www.ivoa.net/documents/latest/AstrogridWorkflow.html">http://www.ivoa.net/documents/latest/AstrogridWorkflow.html</a></li>
<li>VO France: <a class="external" href="http://www.france-ov.org/twiki/pub/GROUPEStravail/Workflow/adass2007.pdf">http://www.france-ov.org/twiki/pub/GROUPEStravail/Workflow/adass2007.pdf</a></li>
</ul>
<p>Unfortunately, it does not look like that we are close on an agreed format for workflow management.</p> Development: Workflow implementationhttps://cta-redmine.irap.omp.eu/boards/14/topics/2372016-02-23T08:21:58ZKnödlseder Jürgenjurgen.knodlseder@irap.omp.eu
<p>We would like to implement the support for analysis workflows in ctools, see the discussion at <a href="https://cta-redmine.irap.omp.eu/issues/1508" class="issue tracker-2 status-5 priority-3 priority-lowest closed" title="Allow to run analysis via config file (Closed)">#1508</a>.</p>
<p>The question is how exactly this should be done. This basically boils done to specifying the interface for defining the workflow.</p>
Here a number of options that are identified so far (list not exhaustive):
<ul>
<li>use an existing workflow manager and interface (e.g. <a class="external" href="http://www.taverna.org.uk/download/workbench/2-5/astronomy/">http://www.taverna.org.uk/download/workbench/2-5/astronomy/</a>)</li>
<li>create a cscripts code generator (producing a Python script that then needs to be executed)</li>
<li>define a custom format for workflow definition (mor user friendly than XML?)</li>
</ul>
<p>Please add your thoughts on this issue in this thread.</p> Development: Review ctools parameter nameshttps://cta-redmine.irap.omp.eu/boards/14/topics/2092014-10-30T15:24:02ZKnödlseder Jürgenjurgen.knodlseder@irap.omp.eu
<p>The ctools parameter names are basically a legacy from the Fermi/LAT Science Tools, yet the tools do not have always a coherent a clear naming convention. I think we should correct this for the ctools before we bring out release 1.0.</p>
<p>This thread is for discussion a naming convention.</p>
Here a list of suggestions (subject to discussion, of course):
<ul>
<li>Files that are input or output should have names starting with “in...” and “out...”. The “...” should correspond to the file type. For example, if we deal with observation definition files or optionally event lists or counts maps, I would propose “obs”. This would result in “inobs” for an input parameter and “outobs” for an output parameter. For the moment we have “infile” and “outfile”, which becomes non-informative once more than a single file is used. I recognize, however, that in the initial ftools idea one probably is always dealing with a single input and output file ...</li>
<li>Following the above logic, models should be expanded from “mdl” to “model” to be more informative. Hence “inmodel” and “outmodel”. The term “srcmdl” is anyways confusion as the model contains in general source and background components.</li>
<li>For event lists (i.e. if only a single event list file is given), use “events”, i.e. “inevents” and “outevents” </li>
<li>For counts maps (i.e. if only a single counts map is given), use “cube”, i.e. “incube” and “outcube”; counts map is anyways not a proper terms as we’re dealing with cubes</li>
<li>Keep “ra”, “dec” and “rad” for the definition of a region of interest</li>
<li>Keep “usepnt” flag if pointing should be used for RoI centre</li>
<li>Keep “ebinalg”, “emin”, “emax”, “enumbins”, “ebinfile” as the standard set of parameters for energy binning definition.</li>
<li>Keep “nxpix”, “nypix”, “binsz”, “coordsys”, “xref”, “yref”, “axisrot”, and “proj” as the standard set of parameters for spatial binning definition.</li>
<li>Keep “tmin” and “tmax” as the standard set of parameters for time interval definition; note that these parameters should probably be changed to strings to allow also specifying times in UTC (now supported by GammaLib)</li>
<li>Keep “chatter”, “clobber”, “debug”, “mode” and “logfile” as standard parameters for every ctool</li>
<li>Keep “caldb” and “irf” for IRF response definition; note that this may evolve once we get a better understanding of how to handle the instrument response</li>
<li>Introduce “expcube” and “psfcube” for Cube response definition (needed for stacked analysis); I still don’t know how to automatically recognize the response type that should be used.</li>
<li>Keep “edisp” flag to signal usage of energy dispersion</li>
<li>Keep “deadc” for deadtime correction</li>
<li>Keep “stat” and “refit” for specifying the statistics and allowing for refit.</li>
<li>Keep “seed” for the seed value in simulations</li>
<li>Keep “amax” and “anumbins” for the angular binning definition in the psfcube tool</li>
<li>Keep “expr” for the ctselect expression</li>
</ul>