[LSST|dm-users 586] Re: pipe_tasks 10.0+1 build error

[John Swinbank]

> On 13 Feb 2015, at 21:40, Robert Lupton the Good <rhl at astro.princeton.edu> wrote:
> 
> Is there some other problem with this test, giving different results under different circumstances (https://jira.lsstcorp.org/browse/DM-1984).  Or did we demonstrate that those circumstances were numpy version?

I’ve established that I get the same numbers (and test failure) as reported by you, Jacek & Will when using NumPy 1.7, but the test passes with NumPy 1.9. Our installation procedure only checks for NumPy >= 1.5.1 (https://github.com/lsst/numpy/blob/master/ups/eupspkg.cfg.sh#L2), so it’s quite possible to run into this issue while still satisfying all our (documented) dependencies. This seems to me to be an overwhelmingly likely explanation, but I’ve not actually checked that all of the test failures folks have seen occurred when using an old version of NumPy.

I’ll update DM-1984 to reflect that, and give folks a chance to pitch in with counter-examples if they have them.

> Anyway, I don't think that we should be using numpy.mean() on large arrays.  It's not guaranteed to be smart, and switching to 64-bit floats rather than fixing the algorithm is among the refuges of the scoundrel.

We can’t retroactively fix the v10_0 release, of course, and the risk of test failure goes away in the current master (thanks to the float64 workaround). However, it seems cleaner to rely on our own API where possible, so I come down slightly in favour of switching this code to use afwMath. I don’t feel strongly enough to argue the point with Russell though.

Cheers,

John