[lsst-dm-stack-users] Using obs_decam
Simon Krughoff
simon.krughoff at gmail.com
Tue Apr 29 08:27:38 PDT 2014
Hi JIm,
This sort of thing happens when the butler tries to figure out a data id
from a partial data id. You are only giving the visit, so the butler has
to figure out the rest of the data id. Ideally it would loop over all
chips in that visit, but somewhere there is a degeneracy that is not being
broken by the query the butler is doing.
One thing you can try is to give a fully qualified data id (all the keys it
needs to find a raw exposure). Without looking more closely I can't say
why it can't fill in the rest of your data id. One thing to note is that
you can qualify most of the data id and use the range (..) and
concatenation (^) operators to specify the ccds you want to process.
Simon
On Mon, Apr 28, 2014 at 6:14 PM, James Chiang <jchiang at slac.stanford.edu>wrote:
> Hi All,
>
> I'm trying to use obs_decam with DM stack v7_3 to analyze DECam data. I'm
> able to run ingestImages.py to get my data into a repository, but when I
> try to run processCcd.py, I get this:
>
> [obs_decam]$ processCcd.py data_repo_v7_3 --id visit=155293
> : Loading config overrride file '/u/gl/jchiang/ki18/DESC/DES/obs_decam/
> 0.0.0.0/config/processCcd.py'
> : Config override file does not exist:
> '/u/gl/jchiang/ki18/DESC/DES/obs_decam/0.0.0.0/config/decam/processCcd.py'
> : input=/nfs/slac/g/ki/ki18/jchiang/DESC/DES/data_repo_v7_3
> : calib=None
> : output=None
> CameraMapper: Loading registry registry from
> /nfs/slac/g/ki/ki18/jchiang/DESC/DES/data_repo_v7_3/registry.sqlite3
> CameraMapper: Loading calibRegistry registry from
> /nfs/slac/g/ki/ki18/jchiang/DESC/DES/data_repo_v7_3/calibRegistry.sqlite3
> processCcd FATAL: Failed in task initialization: no such table: raw_skyTile
> Traceback (most recent call last):
> File "/opt/lsst/rhel-6-x86_64/Linux64/pipe_base/7.3.1.0+1/python/lsst/pipe/base/cmdLineTask.py",
> line 172, in precall
> task.writeConfig(parsedCmd.butler, clobber=self.clobberConfig)
> File "/opt/lsst/rhel-6-x86_64/Linux64/pipe_base/7.3.1.0+1/python/lsst/pipe/base/cmdLineTask.py",
> line 306, in writeConfig
> elif butler.datasetExists(configName):
> File "/opt/lsst/rhel-6-x86_64/Linux64/daf_persistence/7.3.1.0+1/python/lsst/daf/persistence/butler.py",
> line 187, in datasetExists
> location = self.mapper.map(datasetType, dataId)
> File "/opt/lsst/rhel-6-x86_64/Linux64/daf_persistence/7.3.1.0+1/python/lsst/daf/persistence/mapper.py",
> line 116, in map
> return func(self.validate(dataId), write)
> File "/opt/lsst/rhel-6-x86_64/Linux64/daf_butlerUtils/7.3.1.1+1/python/lsst/daf/butlerUtils/cameraMapper.py",
> line 282, in mapClosure
> return mapping.map(mapper, dataId, write)
> File "/opt/lsst/rhel-6-x86_64/Linux64/daf_butlerUtils/7.3.1.1+1/python/lsst/daf/butlerUtils/mapping.py",
> line 118, in map
> actualId = self.need(self.keyDict.iterkeys(), dataId)
> File "/opt/lsst/rhel-6-x86_64/Linux64/daf_butlerUtils/7.3.1.1+1/python/lsst/daf/butlerUtils/mapping.py",
> line 199, in need
> lookups = self.lookup(newProps, newId)
> File "/opt/lsst/rhel-6-x86_64/Linux64/daf_butlerUtils/7.3.1.1+1/python/lsst/daf/butlerUtils/mapping.py",
> line 168, in lookup
> where, self.range, values)
> File "/opt/lsst/rhel-6-x86_64/Linux64/daf_butlerUtils/7.3.1.1+1/python/lsst/daf/butlerUtils/registries.py",
> line 120, in executeQuery
> c = self.conn.execute(cmd, values)
> OperationalError: no such table: raw_skyTile
> [obs_decam]$
>
>
> There is, of course, no raw_skyTile table in registry.sqlite3, so I
> commented out the corresponding lines in policy/DecamMapper.paf and get the
> following:
>
> [obs_decam]$ processCcd.py data_repo_v7_3 --id visit=155293
> : Loading config overrride file '/u/gl/jchiang/ki18/DESC/DES/obs_decam/
> 0.0.0.0/config/processCcd.py'
> : Config override file does not exist:
> '/u/gl/jchiang/ki18/DESC/DES/obs_decam/0.0.0.0/config/decam/processCcd.py'
> : input=/nfs/slac/g/ki/ki18/jchiang/DESC/DES/data_repo_v7_3
> : calib=None
> : output=None
> CameraMapper: Loading registry registry from
> /nfs/slac/g/ki/ki18/jchiang/DESC/DES/data_repo_v7_3/registry.sqlite3
> CameraMapper: Loading calibRegistry registry from
> /nfs/slac/g/ki/ki18/jchiang/DESC/DES/data_repo_v7_3/calibRegistry.sqlite3
> processCcd FATAL: Failed in task initialization: No unique lookup for
> ['object', 'visit', 'date', 'ccd', 'proposal', 'side'] from {}: 62 matches
> Traceback (most recent call last):
> File "/opt/lsst/rhel-6-x86_64/Linux64/pipe_base/7.3.1.0+1/python/lsst/pipe/base/cmdLineTask.py",
> line 172, in precall
> task.writeConfig(parsedCmd.butler, clobber=self.clobberConfig)
> File "/opt/lsst/rhel-6-x86_64/Linux64/pipe_base/7.3.1.0+1/python/lsst/pipe/base/cmdLineTask.py",
> line 306, in writeConfig
> elif butler.datasetExists(configName):
> File "/opt/lsst/rhel-6-x86_64/Linux64/daf_persistence/7.3.1.0+1/python/lsst/daf/persistence/butler.py",
> line 187, in datasetExists
> location = self.mapper.map(datasetType, dataId)
> File "/opt/lsst/rhel-6-x86_64/Linux64/daf_persistence/7.3.1.0+1/python/lsst/daf/persistence/mapper.py",
> line 116, in map
> return func(self.validate(dataId), write)
> File "/opt/lsst/rhel-6-x86_64/Linux64/daf_butlerUtils/7.3.1.1+1/python/lsst/daf/butlerUtils/cameraMapper.py",
> line 282, in mapClosure
> return mapping.map(mapper, dataId, write)
> File "/opt/lsst/rhel-6-x86_64/Linux64/daf_butlerUtils/7.3.1.1+1/python/lsst/daf/butlerUtils/mapping.py",
> line 118, in map
> actualId = self.need(self.keyDict.iterkeys(), dataId)
> File "/opt/lsst/rhel-6-x86_64/Linux64/daf_butlerUtils/7.3.1.1+1/python/lsst/daf/butlerUtils/mapping.py",
> line 201, in need
> raise RuntimeError, "No unique lookup for %s from %s: %d matches" %
> (newProps, newId, len(lookups))
> RuntimeError: No unique lookup for ['object', 'visit', 'date', 'ccd',
> 'proposal', 'side'] from {}: 62 matches
> [obs_decam]$
>
>
> Any help or suggestions on how to proceed would be most welcome.
>
> Thanks,
>
> -Jim
>
> --
> James Chiang SLAC, MS 29 home: (650) 605-3346
> Fermi ISOC 2575 Sand Hill Rd office: (650) 926-2930
> Menlo Park CA 94025 FAX: (650) 926-5566
>
> _______________________________________________
> lsst-dm-stack-users mailing list
> lsst-dm-stack-users at lsstcorp.org
> http://listserv.lsstcorp.org/mailman/listinfo/lsst-dm-stack-users
>
>
--
K. Simon Krughoff
University of Washington
Department of Astronomy
Physics-Astronomy Bldg.
3910 15th Ave NE
Room C319
Seattle WA 98195-0002
USA
(206)543-9487
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://listserv.lsstcorp.org/pipermail/dm-users/attachments/20140429/8cd823af/attachment.html>
More information about the dm-users
mailing list