HN Gopher Feed (2017-06-27) - page 1 of 10 ___________________________________________________________________
Why Is NumPy Only Now Getting Funded?
212 points by numfocusfnd
https://www.numfocus.org/blog/why-is-numpy-only-now-getting-funded/___________________________________________________________________
js8 - 2 hours ago
"The problem of sustainability for open source scientific software
projects is significant."Yeah, William Stein can tell these stories
too: http://sagemath.blogspot.cz/2015/09/funding-open-source-
math...
kronos29296 - 2 hours ago
Yeah, every open source scientific software seems to have an
uphill task before it gets recognized. Sagemath more so than
others because of its scale and having a huge code base. Having
to integrate multiple open source projects isn't helping
either.But its a fine project and one that I use. For a
collection of other projects it works better than most other
individual projects."Every successful science library has the
ashes of an academic career in the making" - Same guy
rjbwork - 2 hours ago
We have this problem in the .NET world. Accord.NET is written by a
brilliant academic and programmer. It's well written, and has a
good API, but it is largely the effort of this one dude, with minor
contributions from a smattering of other fellows.Again, it is great
in general, but it has bugs and rough edges here or there, and a
lot of people don't trust it for production. I wish there was a
way for people to be properly compensated for building and
maintaining such vital scientific and mathematical computing
software.
devoply - 1 hours ago
That's because people don't actually value free things, until
they come to depend on them, and then there is no way for that to
continue unless they pay. And even then they will pay the bare
minimum to keep it going. That's the whole psychology behind open
source. Companies could pay for it and those using it can more
than afford to do so, but it's free, and apparently it's going to
stay that way so why not ride that gravy train?
[deleted]
hiram112 - 55 minutes ago
.Net had always had a problem with open source because Microsoft
implemented APIs and kept them closed source. Users ended up
paying for it via licensing, MSDN, and Visual Studio.I often see
a lot of really mice Java frameworks with a .Net clone which gets
much less attention (Hibernate, some of the Spring stuff,
etc).Whole I understand Microsoft is going their best to open up
.Net, I'm hesitant to believe they'll ever get to the point where
original implementations are done in it, and then ported to other
langs - it just has a deservedly bad rep.
rjbwork - 32 minutes ago
I started programming C# professionally in 2013...the ecosystem
has grown and been improved by leaps and bounds since then.
I'm hopeful, personally.I think one thing that keeps C# things
from being implemented elsewhere is that there are just very
different styles between even C# and Java. Some fancy
libraries will do lots of reflectiony things, build syntax
trees, and compile on the fly for performance, allowing complex
configuration to be simplified and get fast code at runtime.
There's also a big culture of LINQ/functional style, and fluent
interfaces in libraries, etc. which doesn't seem to be present
in something like the Java community (at least until streams),
whereas more dynamic/weakly typed languages don't have, for
obvious reasons, a culture of strong typing and letting the
compiler do the work for you.Then when you go to the hardcore
functional languages like Haskell, ML, etc. you don't have a
culture of OO much, not to mention they just aren't used much
in production environments.So, you see a lot of things from
elsewhere ported and brought to C#/.NET, but not a lot of
things moving from C# to other ecosystems.
largote - 1 hours ago
Because it's an important tool for Machine Learning, which makes
money from that (of which there's plenty going around right now)
flow into it.
ssivark - 1 hours ago
There is a long-standing problem in open source software, which is
that there is no "business model" associated with funneling
resources to people putting significant effort into it. Setting up
a consulting business to monetize software creates the perverse
incentive to make software harder to use, but there seem to be some
examples where this model has worked out reasonably.Open source
projects are typically started by people working in the field, who
have a strong urge to scratch some itch. Even if we find a way to
find money for them to work full-time, they often don't have the
desire to "productize" software, or to create/nurture/govern an
organization around bringing together different stakeholders who
might be able to use, or contribute to the software. (We got really
lucky with Linus+Linux)
santaclaus - 2 hours ago
It blows my mind that NumPy is just getting funding. How did the
Eigen (used in TensorFlow, among other things) folks keep it going?
ajtulloch - 2 hours ago
A few of the Eigen core devs (Benoit Jacob, Benoit Steiner, ...)
work at Google.
danjoc - 1 hours ago
>the entire scientific Python stack was essentially relying upon on
the ?free time? work of only about 30 people?and no one had
funding!30 people? I remember a time when a certain fruit company
would enter a field, literally hire all 30 of those guys, and put
them behind closed doors. Then in 2 years they'd dominate the field
for the next decade.Are these guys turning down offers? Or is the
fruit company that poorly managed now?
breatheoften - 58 minutes ago
I wonder too -- Apple should be hiring _somebody_ to build a
modern nd-array for swift -- and evolve playgrounds into the
ultimate scientific computing environment.The tools for
scientific computing are amazingly scattered - seems like a big
company with a modern platform strategy could make a huge impact
...
webmaven - 53 minutes ago
The fruit company (and really all of AFGAM and much of the rest
of the industry) is biased toward hiring recent grads or folks
who have a clear record of steadily increasing responsibility,
which disadvantages those who have followed a more... eclectic
path, as well as those who have had major dislocations of some
sort derail their career.And almost by definition, "those 30
guys" building and maintaining a library with their volunteer
labor in some "field" (that didn't exist as such until relatively
recently) mostly won't have standard-looking career
paths.Sometimes there is a critical mass of those folks
concentrated in a company, prompting an acquihire.
santaclaus - 42 minutes ago
They seem more likely than others -- see the recent Homebrew
author grousing about getting rejected by Google, only to land
at Apple, for instance.
mschaef - 1 hours ago
I don't know who you're talking about, but Orange Micro went out
of business in 2004. ( https://en.wikipedia.org/wiki/Orange_Micro
)
JPKab - 1 hours ago
Not sure if you're being sarcastic, but he means Apple.
danjoc - 1 hours ago
I think he knows. Mike has a 5 digit /. ID :)
ngoldbaum - 40 minutes ago
There isn't much money in writing libraries like NumPy. Why would
a company hire a bunch of expensive devs to write some software
to compete with Matlab? The market just isn't big enough to
justify the cost.
santaclaus - 28 minutes ago
Google hires devs to develop similar libraries... to power
their machine learning efforts. There is tons of numerical work
in finance, too. I don't think the money is to be made by
selling copies of a Matlab-like piece of software, but in the
application of the tools.
mattfrommars - 2 hours ago
Ok, so the problem seems to be 'lack of maintainer' or could be
stretched to contributer. The article later linked to
https://www.slideshare.net/NadiaEghbal/consider-the-maintain...
which kind of reminded me a problem which I'm facing.After getting
through basic of "Introduction to Computer Science Using Python"
and forever pending goal to become a "Python Developer", is anyone
here who is experience in Python willing to be my mentor? In
return, free Python labor. :)
webmaven - 1 hours ago
What sort of things ("Python development" is a bit nebulous) are
you most interested in?
pc2g4d - 2 hours ago
Ask me questions and I'll do my best to respond
projectramo - 2 hours ago
Q: Are we conflating two issues?Is there a difference between the
"sole developer problem" and the "lack of funding" problem.I mean,
even if a project finds funding, does it follow that it will
attract more talented developers?One way to distinguish the two
issues is to look at for-profit software. In the cases where there
is one primary developer, do they find it easy to keep the software
going when the person retires?I ask this because, I think, beyond
the very real monetary issue, there is a question of how
development works. Do we need one very talented individual who does
the lion's share of the lifting?
webmaven - 1 hours ago
> even if a project finds funding, does it follow that it will
attract more talented developers?It depends on what the funding
is used for. Outreach in various forms (better documentation,
presenting at conferences, a nicer website, moderating community
venues from mailing lists to chat channels, triaging and grooming
issues and PRs, etc.) takes time and effort (and therefore needs
to be funded) as well.
williamstein - 40 minutes ago
For SageMath, what funding we?ve got has been mostly used for
organizing many ?Sage Days? workshop coding sprints ? we?ve had
nearly 100 now over the last decade! They have made an
absolutely tremendous difference in attracting the over 600
people who have contributed. But also key design decision,
e.g., 100% test coverage, helped too. I clearly remember the
moment ? in a big discussion at a Sage Days (funded by the Clay
Math Institute) ? that Craig Citro argued for 100% doctest
coverage, and we all decided to do it. Without funding,
those Sage Days wouldn?t have happened, and the community would
not have developed. I also remember a Sage Days that we did
jointly with Numpy at Enthought, where discussions and work we
did together grew into many important things later. Funding is
ridicuously absolutely critical to growing certain types of
open source projects. In another direction, a lot of our
infrastructure (e.g., build, testing, etc.,) is hosted on a big
computer bought with a grant.
mtmail - 1 hours ago
If your open source project needs funding there is
https://opencollective.com/opensource (currently waiting, I'm not
affiliated)
carapace - 1 hours ago
tl;dr:> "And if you?d like to take action to contribute to project
sustainability, consider becoming a NumFOCUS member
today."https://www.numfocus.org/community/donate/
jordigh - 2 hours ago
Also a problem for Octave. Remember
this?https://news.ycombinator.com/item?id=13603575It also stings a
little when people say that it's completely obsolete because Matlab
itself is "legacy" and we should all be abandoning the language,
Octave included... and yet, even though I like numpy and Python and
matplotlib and Julia and R, I still find myself reaching for Octave
whenever I need a quick visualisation of some data.
maliker - 1 hours ago
You might like matplotlib for visualization. It's a Python clone
of Matlab's plotting tools. And it's compatible with numpy if you
need to do linear algebra or operate on big datasets.
cwyers - 1 hours ago
He mentions matplotlib by name.
in9 - 2 hours ago
Really? Why is that? In my mind, it is tough to beat ggplot
mschaef - 1 hours ago
It's surprising to me that people are surprised by this.Even
setting aside the fact that the people that can do this work are
few in number, the vast majority of people need a way to support
themselves and their family. If the number of people that have
these skills is low, the subset that is both altruistic enough to
donate them for a sufficient period of time and personally able to
do so must be vanishingly small. (And the negative feedback a lot
of OSS maintainers receive doesn't help.)Companies have the same
issue... there has to be a fairly direct connection between an
expenditure (paying developers) and a return on that investment.
That can be a very difficult argument to make.
ArneBab - 2 hours ago
Another project which is easy to overlook: Think about how many
scientists use Emacs for most of their development and writing. But
there is (to my knowledge) not even a single paid developer working
on it.( http://gnu.org/s/emacs )
jeremynixon - 2 hours ago
This is still an important problem for numerical compute in other
languages. It's a struggle to do data analysis and write machine
learning applications in Scala, Java, C++, etc. due to a lack of
Numpy / Pandas style ease of use and functionality.