[lsst-dm-stack-users] LsstSimMapper questions

[Kian-Tat Lim]

Chris,

> The first is a small bug/issue.  At least with some tests I am running
> with PhoSim version 3.3.2 the program produces FITS files which have
> VERSION = none in the metadata.  This causes the line
> 
> if md.exists("VERSION") and md.getInt("VERSION") < 16952:
> 
> in lsstSimMapper.py to fail with a type error since although it is
> empty it the field still exists.

Is the value of VERSION literally "none" or is it just empty?  I'm not
sure that an empty value is legal FITS.  In any case, I'd be tempted to
say the fault is PhoSim's on this one.

> This was stopping with:
> 
> AttributeError: 'LsstSimMapper' object has no attribute 'map_processTest_config'

The problem is that we persist the config used for running a task (for
provenance purposes) as well as the schemas for any FITS table catalogs
generated and the metadata produced by running the task (e.g.
performance statistics).  This is done using the data butler.
Currently, each such dataset needs to be specified independently in the
.paf file used by the butler.  There's no API for doing this on the fly
(yet), no way to add to or override the policy file, and no (good) way
to have a generic config dataset.

Some workarounds are:

1) Set the _DefaultName to something that already has a _config dataset
defined (as you found out).

2) Override _getConfigName() and _getMetadataName() to return None,
bypassing writing the config and metadata (but not bypassing the writing
of schemas, if you define them by overriding getSchemaCatalogs()).

3) Override writeConfig(), writeSchemas(), and writeMetadata() to do
nothing (or write using something other than the butler).

4) Override the TaskRunner's precall() and __call__() to not write
these.

Of these, I think #2 or #3 may be your best bets.

-- 
Kian-Tat Lim, LSST Data Management, ktl at slac.stanford.edu