[lsst-dm-stack-users] LsstSimMapper questions

[Chris Walter]

For the sake of the email archives and people who search this thread in the future:

Following KT's suggestion, the following CmdLineTask code skeleton:
----

import lsst.pex.config as pexConfig
import lsst.pipe.base as pipeBase

class ProcessTestConfig(pexConfig.Config):
    """Config for ProcessTest"""

class ProcessTestTask(pipeBase.CmdLineTask):
    ConfigClass = ProcessTestConfig
    _DefaultName = "processTest"

    def _getConfigName(self):
        return None

    def _getMetadataName(self):
        return None
    
    def run(self, sensorRef):
        print "HELLO!"
        self.log.info("Processing %s" % (sensorRef.dataId))    

ProcessTestTask.parseAndRun()
----

Runs without crashing. Of course, the config and metadata will not written out in this case.

-Chris


On Aug 27, 2013, at 3:55 PM, Kian-Tat Lim <ktl at slac.stanford.edu> wrote:

> Chris,
> 
>> The first is a small bug/issue.  At least with some tests I am running
>> with PhoSim version 3.3.2 the program produces FITS files which have
>> VERSION = none in the metadata.  This causes the line
>> 
>> if md.exists("VERSION") and md.getInt("VERSION") < 16952:
>> 
>> in lsstSimMapper.py to fail with a type error since although it is
>> empty it the field still exists.
> 
> Is the value of VERSION literally "none" or is it just empty?  I'm not
> sure that an empty value is legal FITS.  In any case, I'd be tempted to
> say the fault is PhoSim's on this one.
> 
>> This was stopping with:
>> 
>> AttributeError: 'LsstSimMapper' object has no attribute 'map_processTest_config'
> 
> The problem is that we persist the config used for running a task (for
> provenance purposes) as well as the schemas for any FITS table catalogs
> generated and the metadata produced by running the task (e.g.
> performance statistics).  This is done using the data butler.
> Currently, each such dataset needs to be specified independently in the
> .paf file used by the butler.  There's no API for doing this on the fly
> (yet), no way to add to or override the policy file, and no (good) way
> to have a generic config dataset.
> 
> Some workarounds are:
> 
> 1) Set the _DefaultName to something that already has a _config dataset
> defined (as you found out).
> 
> 2) Override _getConfigName() and _getMetadataName() to return None,
> bypassing writing the config and metadata (but not bypassing the writing
> of schemas, if you define them by overriding getSchemaCatalogs()).
> 
> 3) Override writeConfig(), writeSchemas(), and writeMetadata() to do
> nothing (or write using something other than the butler).
> 
> 4) Override the TaskRunner's precall() and __call__() to not write
> these.
> 
> Of these, I think #2 or #3 may be your best bets.
> 
> -- 
> Kian-Tat Lim, LSST Data Management, ktl at slac.stanford.edu