[lsst-dm-stack-users] In response to Mario's request

[Dave Monet]

Here is a brief discussion of how far I have gotten
trying to use the LSST DM Stack on recently arrived
DECam images.

1) I followed the instructions on the Wiki back in
    September about how to download (from the tarball)
    and install the DM Stack.  I got as far as
    the final line, but this ended up taking more than
    a week before I gave up on it.  I followed Mario's
    suggestion to do
      eups distrib install -t Summer2012 obs_lsstSim
      eups distrib install -t Summer2012 obs_sdss
    So far as I can tell, this worked.

2) I installed git() from some free download site.
    It needed something else, and it must be installed
    without root privilege or putting it in obvious
    places /usr anything.  So far as I know, the installation
    was successful.  I can to the following:
       lethe>  git --version
       git version 1.7.12.GIT

3) I found viz12-lupton-tutorial.pdf based on Andy's
    suggestion that there was a Hands On session using
    the DM stack at the Seattle LSST-Gaia meeting.  I
    got as far as page 4.  In response to
       import utils
    I get a failure that this particular version of
    utils.py is not in my path.  However, it was
    possible to proceed with the next few lines
    without these functions so long as I was willing
    to type.  I didn't have the Hands On files, but
    I have various other local files.  I could do
       import lsst.afw.image as afwImage
       import lsst.afw.display.ds9 as ds9
       im = afwImage,ImageF(some complete path)
    So I think that some of the DM stack is present
    and mostly working.

4) I moved on to other things, but had a desire to
    return to the DM Stack to process the DECam images.
    Robert Lupton and Paul Price pitched in, but Paul
    sent me the appended note.  To me, this reads as if
    it were in a dialect of regional Martian that I simply
    do not understand.  I asked Paul for clarification, but
    he was too busy.  I have no idea what "tickets" are about
    and how to update based on these.  I have no idea how
    to proceed with the git() to get the obs_decam package.

This just underlines the complete discontinuity between those
who work with the DM Stack on a daily basis, and folks like
me who need serious hand-holding.  Honest.  I would like to
be a Team Player, and I would like to use utilities that are
known to work and have support, but I just have no idea how
to proceed.

Pointers, examples, or clues would be gratefully accepted.
-Dave


-------- Original Message --------
Subject: Re: Plan B for DECam images?
Date: Tue, 2 Oct 2012 15:24:32 -0400
From: Paul Price <price at astro.princeton.edu>
To: Dave Monet <dgm at nofs.navy.mil>
CC: Robert Lupton the Good <rhl at astro.princeton.edu>

On Oct 2, 2012, at 8:49 AM, Dave Monet wrote:
> I hope Paul is taking notes about how he is feeding the DM stack.  That would be an ideal introduction
> on how to do that for clueless folks like myself.


You need a relatively recent version of the stack.  LSST will be pushing 
something out soon-ish, or I could try to get you access to our HSC 
build somehow.  Then you need a couple of recent updates: tickets/2352 
on daf_persistence and afw; and tickets/2353 on pipe_tasks; and master 
on meas_astrom and pipe_base.  Then get the obs_decam package 
(git at git.lsstcorp.org:contrib/price/obs_decam.git).

Download the DECam observations (in MEF format) from the NOAO science 
archive and run:

$ ingestImages.py decam /path/to/repo/ --create --mode=link 
/path/to/data/*.fits.fz

That will suck the images into a new repository at /path/to/repo/ . 
I've specified "--mode=link" which means it's set up as symbolic links 
to your raw data; you can "copy" or "move" if you desire.  If you've 
already got files in the repo, leave off the "--create" flag.

Once that's done, set up your astrometry.net catalogue.  I'm using 
2MASS, courtesy of Dustin (if you need the configuration files for 
running it with the LSST code, let me know and I'll hook you up; RHL: 
it's on /tigress-hsm/astro/hsc/products/astrometry_net_data/2mass):

$ setup astrometry_net_data 2mass

Then reduce the data with:

$ processCcd.py decam /path/to/repo/ --id visit=135633..135662 -c 
calibrate.doPhotoCal=False

The range of visits (135633..135662) is all the Monet 1 observations. 
I'm turning photometric calibration off because I only have J-band 
magnitudes from the 2MASS catalogue.  If you only want to do a single 
CCD, use, e.g.,  "--id visit=135633 side=N ccd=4" (to do the N4 CCD from 
exposure number 135633). It looks like there's a bug in my geometry file 
that's offsetting and chopping off 50 or so rows for the 'N' CCDs; I 
hope to fix that soon.  If you've got multiple cores and your office is 
cold, you can parallelise by adding, e.g., "-j 8" to the command-line.

I hope to have some measurement files for you later this afternoon --- 
I'm burning through the data now, and I'll go back and update the 'N' 
CCDs if I can.

P.