ctools

Another option which came to my mind is to use the XML interface to pass the config file.
I attached how such a file could possibly look like. No new functionality on the gammalib level would be needed because the I/O is already supported.
Any thoughts?

I’ve looked quite a bit at analysis config in the past years, I think having a config-file driven analysis is valuable for many users. Not just beginners ... it’s also how I like to run analyses ... declare parameters and configure algorithm in one file, execute one or very few commands.

So here are my 2 cents ...

IMO a pfile isn’t a good choice, because it’s flat, and having something hierarchichal is better to structure the config file.

XML works, but is IMO a bit hard to read / write ... but since ctools uses it for many other things such as model specs or obs specs, it might be a very good / consistent choice.

Just using Python scripts is IMO very nice. In this approach you’d add few XYZAnalysis classes with a config attribute and a run method that execute the right ctools in the right order with the right options based on config.
Then the end-user would write short Python scripts that create such analysis classes and set all the config objects.
I.e. the main difference to a config file is that you have a few extra lines: the import at the top, creating the object and calling run at the end.
If you don’t like that users have to write these extra lines, you can do something like scons: use Python scripts, but have them executed by scons instead of Python directly, which executes some things (like the imports) automatically:
https://en.wikipedia.org/wiki/SCons#Examples

Other good options are INI or YAML.

A few years ago I used INI files via configobj (a single .py file you can bundle):
http://enrico.readthedocs.org/en/latest/configfile.html

By now I like YAML better (more common, more flexible), so a few months ago in Gammapy we decided to use YAML for config files and result output (example: https://gammapy.readthedocs.org/en/latest/tutorials/gammapy-spectrum/index.html#spectral-fitting-with-gammapy-spectrum).
It’s also what Matt chose for Fermipy. I think looking at this example how to configure a Fermi analysis makes it clear that a flat pfile doesn’t work well, no?
http://fermipy.readthedocs.org/en/latest/quickstart.html
Here’s another example how YAML is used for playbooks:
http://docs.ansible.com/ansible/playbooks_intro.html

So to summarise ... if you don’t want to write a YAML parser, my recommendation would be: use Python.
(and actually, even if YAML were readily available, just using Python XYZAnalysis high-level classes might still be better).

Looking forward to what you decide on for ctools!

To make a proof of principle for an XML-driven analysis workflow I pushed a branch 1508-implement-workflow into the ctools repository. It contains a script csworkflow.py to execute a workflow described by the test/data/workflow.xml file.

The format of the XML file is:

<?xml version="1.0" encoding="UTF-8" standalone="no"?>
<workflow>
    <actor name="input">
        <output>
            <parameter name="inmodel" value="$CTOOLS/share/models/crab.xml" />
            <parameter name="caldb"   value="prod2" />
            <parameter name="irf"     value="South_50h" />
            <parameter name="edisp"   value="no" />
            <parameter name="deadc"   value="0.95" />
            <parameter name="ra"      value="83.63" />
            <parameter name="dec"     value="22.01" />
            <parameter name="target"  value="Crab" />
            <parameter name="emin"    value="0.1" />
            <parameter name="emax"    value="100.0" />
            <parameter name="tmin"    value="0.0" />
            <parameter name="tmax"    value="1800.0" />
            <parameter name="chatter" value="2" />
            <parameter name="clobber" value="yes" />
            <parameter name="debug"   value="no" />
            <parameter name="mode"    value="ql" />
        </output>
    </actor>
    <actor name="ctobssim" tool="ctobssim">
        <input>
            <parameter name="inobs"   value="NONE" />
            <parameter name="inmodel" value="inmodel" actor="input" />
            <parameter name="caldb"   value="caldb" actor="input" />
            <parameter name="irf"     value="irf"   actor="input" />
            <parameter name="edisp"   value="edisp" actor="input" />
            <parameter name="prefix"  value="sim_events_" />
            <parameter name="seed"    value="1" />
            ...

<parameter name="inmodel" value="inmodel" actor="input" />

The csworkflow.py script then analysis the XML file to find out which actor depends of which other actors, and then executes the actors in the order that resolves the dependencies (so far, no parallel processing is implemented in case that actors could be executed simultaneously). It uses the tool attribute to execute the actor, so that for example different ctobssim actors could be defined by using different names, e.g. ctobssim1, ctobssim2, etc.

Note that the input actor has no tool associated and only output parameters. That’s the way how “general” parameters are specified. The script can certainly be modified to query for example all input parameters. One could also imagine to have intrinsic functionalities attached to a given tool. Or a general Python command.

It would be interesting to see whether complex workflows can be implemented by this scheme, and what features are missing.

I merged in the draft version of the csworkflow script. I also added a unit test.

We still need to gain more experience on that, but at least we have a workflow manager that can be used now.

Since the workflow manager has been merged in I close the issue now.