[lsst-dm-stack-users] Trying out the Anaconda python with a RHEL6 installation

[Chris Walter]

Hi Mario,

Thanks, I just tried it with the new version of anaconda and it worked great.  Very smooth with no problems.  Nice!

BTW, there is a subtle thing that can happen to some people I meant to mention before but forgot.  I normally use tcsh but usually switch to bash for this installation since that is how all the instruction are written.

If you do that you will find you can't source loadLLST.sh. This screws up the building process. You will get something like this:

bash-4.1$ source Summer_2013/loadLSST.sh 
bash: setenv: command not found
bash: setenv: command not found
bash: setenv: command not found
... repeated

The reason this is happening if you dig into the code is that you are checking $SHELL to determine what shell the person is really running and adjusting appropriately.  But, it turns out that environment variable is set by the original login shell and won't be adjusted later.  Like so:

neutrino-01:LSST $ echo $SHELL
/bin/tcsh
neutrino-01:LSST $ bash -l
bash-4.1$ echo $SHELL
/bin/tcsh
bash-4.1$ 

So, I have been manually setting the SHELL variable to '/bin/bash' myself, but it might be better to find a more robust way to check the shell (or trust that people will use the appropriate file for their shell when they do the 'source').

-Chris


On Feb 12, 2014, at 12:24 PM, Mario Juric <mjuric at lsst.org> wrote:

> Heather, Chris,
> 	Thanks for the hacking effort! I've tried to collect all in a single
> script that I've placed at:
> 
> 	http://lsst-web.ncsa.illinois.edu/~mjuric/hacks/newinstall-anaconda.sh
> 
> It should incorporate all the hacks and streamline the process of
> building Summer 2013 against Anaconda. Starting with a clean
> environment, here's how to build it with the above script:
> 
> ==========
> 	export PATH="......./anaconda/bin:$PATH"
> 
> 	wget http://lsst-web.ncsa.illinois.edu/~mjuric/hacks/newinstall-anaconda.sh
> 	bash newinstall-anaconda.sh
> 
> 	source loadLSST.sh
> 	eups distrib install --nolocks -t Summer2013 lsst_apps
> ==========
> 
> newinstall-anaconda.sh will set up the manifest.remap file so that
> Anaconda's python, numpy and matplotlib are used. It will also implement
> some workarounds for build issues of pyfits and mysqlpython. Finally, it
> will add the export of MAKEFLAGS and SCONSFLAGS to loadLSST.sh (so that
> one doesn't forget about it). Diff it against the standard newinstall.sh
> to see the details of what I've had to change.
> 
> I tried this on a RHEL6 machine and it worked (it built and the unit
> tests passed). Try it out and let me know if it works for you. For the
> Mac, it's likely it will need modifications (e.g., replacing Linux with
> Darwin in all the right places).
> 
> PS: Note that since this is still Summer'13 release, the workaround to
> importing scipy before importing any of the lsst code is still required.
> 
> PPS: Note that I recommend installing lsst_apps above, not lsst_distrib.
> The latter installs lots of extra middleware for running on clusters,
> which I don't think you need/want/use (at the moment).
> 
> PPPS: Installing lsst_distrib should also work (I tested it).
> 
> Hope this helps!,
> - M.
> 
> On 2/10/14, 11:10 , Heather Kelly wrote:
>> On 2/8/2014 1:39 PM, Chris Walter wrote:
>>> Hi Heather,
>>> 
>>>>> Also, you will need to make the config file for numpy and declare numpy like Paul pointed out to me in yesterday's message and you will need to also declare matplotlib as being a system entry (but you don't need to make a config file for it).
>>>>> 
>>>>> -Chris
>>>> So I moved to Spring2013..and added the matplotlib system entry, though
>>>> I was able to avoid doing so entirely with the Winter2013 attempt.  Did
>>>> the config step for numpy, and skipped the config file as you suggested
>>>> but it ended with this:
>>>> 
>>>> eups distrib: No source is available for package matplotlib system
>>>> 
>>> I think my text was probably a bit unclear.  You can sort of see how this all works from the old instructions here:
>>> 
>>> https://dev.lsstcorp.org/trac/wiki/Installing/Winter2013
>>> 
>>> in the section about using macports.
>>> 
>>> So, for numpy you need to make the config file (I guess for package dependencies or something) and then declare it as being external.
>>> 
>>> eups declare numpy system -r numpy -m none
>>> 
>>> like you did.
>>> 
>>> But then for matplotlib and scipy even though you don't need the cfg file you still need to tell eups they are external like this:
>>> 
>>> eups declare matplotlib system -r none -m none
>>> eups declare scipy system -r none -m none
>>> 
>>> -Chris
>>> 
>>> 
>> 
>> Hi Chris,
>> 
>> Yes, thanks - this occurred to me after hitting "send".  Looks like
>> things are rebuilt and I'll give another try to running the analysis
>> scripts against the DMstack.
>> Thanks for your help.
>> 
>> Take care,
>> Heather
>> 
> 
> 
> -- 
> Mario Juric,
> Data Mgmt. Project Scientist, Large Synoptic Survey Telescope
> Web : http://research.majuric.org     Phone : +1 617 744 9003