Feature #989
Updated by Knödlseder Jürgen over 10 years ago
Variable length columns arrays are needed for RMF format reading and writing.
Variable length columns are read and written row by row. For each row, the number of elements in the column can differ. It remains to be decided how the storage of variable length information should be handled in GammaLib. CCFits uses @std::vector<std::valarray<T> >@ to store the column data, allowing thus rows of variable length. If a similar strategy is to be implemented in GammaLib, the FITS column classes memory handler would need to be rewritten.
An alternative would be to store the variable length column in a table of maximum length. This would inflate memory but simplify the housekeeping. The only thing needed would be an additional array with one index per row that gives the number of elements per row.
Here a couple of notes from the cfitsio manual:
_When reading from a variable length array field one can only read as many elements as actually exist in that row of the table; reading does not automatically continue with the next row of the table as occurs when reading an ordinary fixed length table field. Attempts to read more than this will cause an error status to be returned. One can determine the number of elements in each row of a variable column with the fits_read_descript routine._
_When writing row of data to a variable length field the entire array of values for a given row of the table must be written with a single call to fits_write_col. The total length of the array is given by nelements + firstelem - 1. Additional elements cannot be appended to an existing vector at a later time since any attempt to do so will simply overwrite all the previously written data and the new data will be written to a new area of the heap. The fits_compress_heap routine is provided to compress the heap and recover any unused space. To avoid having to deal with this issue, it is recommended that rows in a variable length field should only be written once. An exception to this general rule occurs when setting elements of an array as undefined. It is allowed to first write a dummy value into the array with fits_write_col, and then call fits_write_col_nul to flag the desired elements as undefined. Note that the rows of a table, whether fixed or variable length, do not have to be written consecutively and may be written in any order._
_When writing to a variable length ASCII character field (e.g., TFORM = '1PA') only a single character string can be written. The `firstelem' and `nelements' parameter values in the fits_write_col routine are ignored and the number of characters to write is simply determined by the length of the input null-terminated character string._
Variable length columns are read and written row by row. For each row, the number of elements in the column can differ. It remains to be decided how the storage of variable length information should be handled in GammaLib. CCFits uses @std::vector<std::valarray<T> >@ to store the column data, allowing thus rows of variable length. If a similar strategy is to be implemented in GammaLib, the FITS column classes memory handler would need to be rewritten.
An alternative would be to store the variable length column in a table of maximum length. This would inflate memory but simplify the housekeeping. The only thing needed would be an additional array with one index per row that gives the number of elements per row.
Here a couple of notes from the cfitsio manual:
_When reading from a variable length array field one can only read as many elements as actually exist in that row of the table; reading does not automatically continue with the next row of the table as occurs when reading an ordinary fixed length table field. Attempts to read more than this will cause an error status to be returned. One can determine the number of elements in each row of a variable column with the fits_read_descript routine._
_When writing row of data to a variable length field the entire array of values for a given row of the table must be written with a single call to fits_write_col. The total length of the array is given by nelements + firstelem - 1. Additional elements cannot be appended to an existing vector at a later time since any attempt to do so will simply overwrite all the previously written data and the new data will be written to a new area of the heap. The fits_compress_heap routine is provided to compress the heap and recover any unused space. To avoid having to deal with this issue, it is recommended that rows in a variable length field should only be written once. An exception to this general rule occurs when setting elements of an array as undefined. It is allowed to first write a dummy value into the array with fits_write_col, and then call fits_write_col_nul to flag the desired elements as undefined. Note that the rows of a table, whether fixed or variable length, do not have to be written consecutively and may be written in any order._
_When writing to a variable length ASCII character field (e.g., TFORM = '1PA') only a single character string can be written. The `firstelem' and `nelements' parameter values in the fits_write_col routine are ignored and the number of characters to write is simply determined by the length of the input null-terminated character string._