[lsst-dm-stack-users] A few Lessons Learned

[Dave Monet]

Paul Price suggested that I pass along a subset of my "lessons learned"
in my allegedly successful installation of the DM Stack and the optional
obs_decam package, and using these to measure recently arrived images
from DEcam.

1) There really needs to be a "DM For Dummies" document.  My life is too
    short to understand all of the "wonderful" things that the DM Stack
    process does for software developers.  To me, it is an amazingly
    complicated, mostly undocumented video game wherein getting lost
    is the most probable outcome.  At some level, I don't need to know
    how the Stack is built to use it.  What I need is a cookbook that
    lists the steps I need to take, and that every once in a while gives
    me a test case to verify my progress.

    Paul was very kind to assist in my random walk.  There were two areas
    wherein my total ignorance kept causing problems.

    a) Code:  The sequence for updating a package, and the subsequent
         failure of another package due to new needs in the updated
         package, is pretty frustrating.  It would sure be nice if one
         of the "wonderful" things was a mechanism to follow dependencies.
           "If you update AFW, then you must update XXX and YYY, and you
           might as well start all over again because it still won't work."

    b) Environment:  Much of my lack of progress can be traced to not
         knowing which packages were involved, and not having the
         appropriate "setup" done.  All I had were rather cryptic
         error messages about things not being found.  It seems to be
         my problem if I wanted to unwind these to understand which package
         is either missing or not in the environment.

2) When Paul managed to drag me to the finish line, I was greeted by
    astoundingly poor performance.  Not only did I not really know what
    I could or should do, but the simple test case took 2 hours to find
    stars in a single image.  Paul managed to trace this to the default
    for "scons" is for no optimization.  When rerun with "scons opt=3",
    the performance went from 2 hours to 2 minutes.  It would have been
    really nice for the defaults to be reasonable, or at least for the
    cookbook to tell me what to type to get useful results.

3) My "success" is that I can run the DECam package on my own machine
    using the single instruction that Paul told me about.  I have literally
    no idea what other things I can (or should be able to) do with the
    DM Stack.  My suggestion to Mario is (still) to have some sort of
    AAS discussion of "why" one should spend the time and energy to use
    the DM Stack.  I understand that there is no choice for software
    developers in the LSST project to use it, but offering a few data
    reduction (or similar) recipes at the end of DM For Dummies might
    encourage folks like me to keep beating their heads against the
    software wall to get it working.

Again, many thanks to those who suffered to get me this far.
-Dave