Well, that's not true at all. Scientists care about performance, but it turns out that Python is really good for number crunching since it is really good for using very fast C libraries. I know people who use pandas to manipulate huge datasets from radar astronomy. Also, of course, it's used in machine learning. If Python was "only" used in situations where you don't care about performance, it would not be used in so many scenarios that definitely need high performance. Sure, it is not pure Python, but it's still Python being used, just used to orchestrate C libraries
If you’re actually building and shipping software as a business Python is great. The advantages of Python for a startup are many. Large pool of talent that can pickup the codebase on essentially day 1. Fairly easy to reason about, mature, code velocity, typically one and only one way to do things as opposed to JavaScript. There is way more to the story than raw performance.
It's not that great when you see that the majority of the Python code in businesses is a totally unmaintainable mess because it has incorrect, partial, or no type annotations, and is littered with serious errors that a most basic type checker would flag.
> Why not use a faster language in the first place?
Well for the obvious reason that there isn't really anything like a Jupyter notebook for C. I can interactively manipulate and display huge datasets in Python, and without having to buy a Matlab license. That's why Python took off in this area, really
I believe I heard that argument since before jupyter became popular.
Usually it was accompanied by saying that the time needed to write code is often more important than the time it takes to run, which is also often true.
All that said, jupyter is probably part of python's success, although I'm not the only one who actively avoids it and views it as a bit of a code smell.
I love Jupyter! What I don’t love is people writing large projects in a workbook, then asking how to run it as-is in production so they can continue to iterate on it in that form.
It’s not impossible, but neither is it the sort of thing you want to encourage.
I agree - Jupyter notebook is really the key feature Python has that makes it attractive for research/scientific computing. I would say the REPL too but until very recently it was extremely shoddy so I doubt many people did any serious work in it.
> I would say the REPL too but until very recently it was extremely shoddy
Can you elaborate? I've been using the Python REPL for more than two decades now, and I've never found it to be "shoddy". Indeed, in pretty much every Python project I work on, one of the first features I add for development is a standard way to load a REPL with all of the objects that the code works with set up properly, so I can inspect them.
Very obvious example - you can't paste code containing blank lines.
Another example: navigating this history is done line by line instead of using whole inputs.
It's just bare minimum effort - probably gnu readline piped directly into the interpreter or something.
I think they did improve it a lot very recently by importing the REPL from some other Python interpreter but I haven't upgraded to use that version yet so I don't know how good it is now.
> probably gnu readline piped directly into the interpreter or something
That is more or less how the REPL originally was implemented. I think there's more under the hood there now.
I still don't think what you describe qualifies as "shoddy". There are certainly limitations to the REPL, but "shoddy" to me implies that it's not really usable. I definitely would not agree with that.
The predecessors were really popular before it too - MATLAB in engineering in particular, Mathematica also popular in Physics and Maths departments particularly where the symbolic functionality was more useful. I used both in academia and IPython (later renamed Jupyter) was clearly a natural extension of those but open source, and without the baggage of MATLAB (only one function definition per file, etc.)
>Which was true, but maybe not the strongest argument. Why not use a faster language in the first place?
Because most faster languages sucks donkeys balls when it comes to using them quickly and without ceremony. Never mind trying to teach non-programmers (e.g. physics, statistics, etc people) them...
>>> you absolutely don't care whatsoever about performance and can guarantee that you never will.
Those are actually pretty good bets, better than most other technological and business assumptions made during projects. After all, a high percentage of projects, perhaps 95%, are either short term or fail outright.
And in my own case, anything I write that is in the 5% is certain to be rewritten from scratch by the coding team, in their preferred language.
Sure but you're still screwing yourself over on that 5% and for no real reason - there are plenty of languages that are just as good as Python (or better!) but aren't as hilariously slow.
And in my experience rewrites are astonishingly rare. That's why Dropbox uses Python and Facebook uses PHP.
A painful rewrite in another language is usually the only option in my experience.
If you're really lucky you have a small hot part of the code and can move just that to another language (a la Pandas, Pytorch, etc.). But that's usually only the case for numerical computing. Most Python code has its slowness distributed over the entire codebase.
It’s not painful, that’s the point. You have a working prototype now ready to port. (If the destination language is painful perhaps, but don’t do that.)
I recently ported a Python program to Rust and it took me much less time the second time, even though I write Rust more slowly per-line. Because I knew definitively what the program needed.
And if even that is too much optimizing the Python or adding Cython to a few hot loops is less difficult.
I have also ported a Python program to Rust (got a ~50x speedup) but this was a smallish program, under 10k lines of code.
Porting larger programs is rarely tractable. You can tell that because several large companies have decided that writing their own Python runtimes that are faster is less effort (although they all eventually gave up on that as far as I know).
Could happen with any language. The well-known Spolsky piece was about a C++ to C++ rewrite if memory serves. Blaming repeated poor decisions on a prototyping/glue language is yet another instance. Luckily there's lots of options today to dig out of them.
Probably people at some point were making same arguments about ASM and C. How many people though do ASM these days? Not arguing that for now it is relevant point, obviously Rust / C are way faster.
I doubt it. C is well within 2x of what you can achieve with hand written assembly in almost every case.
Furthermore writing large programs in pure assembly is not really feasible, but writing large programs in C++, Go, Rust, Java, C#, Typescript, etc. is totally feasible.
Well, that's not true at all. Scientists care about performance, but it turns out that Python is really good for number crunching since it is really good for using very fast C libraries. I know people who use pandas to manipulate huge datasets from radar astronomy. Also, of course, it's used in machine learning. If Python was "only" used in situations where you don't care about performance, it would not be used in so many scenarios that definitely need high performance. Sure, it is not pure Python, but it's still Python being used, just used to orchestrate C libraries
If you’re actually building and shipping software as a business Python is great. The advantages of Python for a startup are many. Large pool of talent that can pickup the codebase on essentially day 1. Fairly easy to reason about, mature, code velocity, typically one and only one way to do things as opposed to JavaScript. There is way more to the story than raw performance.
It's not that great when you see that the majority of the Python code in businesses is a totally unmaintainable mess because it has incorrect, partial, or no type annotations, and is littered with serious errors that a most basic type checker would flag.
> The advantages of Python for a startup are many. Large pool of talent that can pickup the codebase on essentially day 1.
Large pool of mediocre Python developers that can barely string a function together in my experience.
The counterargument used to be, the heavy lifting will be offloaded to python modules written in C, like numpy.
Which was true, but maybe not the strongest argument. Why not use a faster language in the first place?
But it's different now. There's huge classes of problems where pytorch, jax &co. are the only options that don't suck.
Good luck competing with python code that uses them on performance.
> Why not use a faster language in the first place?
Well for the obvious reason that there isn't really anything like a Jupyter notebook for C. I can interactively manipulate and display huge datasets in Python, and without having to buy a Matlab license. That's why Python took off in this area, really
I believe I heard that argument since before jupyter became popular.
Usually it was accompanied by saying that the time needed to write code is often more important than the time it takes to run, which is also often true.
All that said, jupyter is probably part of python's success, although I'm not the only one who actively avoids it and views it as a bit of a code smell.
I love Jupyter! What I don’t love is people writing large projects in a workbook, then asking how to run it as-is in production so they can continue to iterate on it in that form.
It’s not impossible, but neither is it the sort of thing you want to encourage.
I agree - Jupyter notebook is really the key feature Python has that makes it attractive for research/scientific computing. I would say the REPL too but until very recently it was extremely shoddy so I doubt many people did any serious work in it.
> I would say the REPL too but until very recently it was extremely shoddy
Can you elaborate? I've been using the Python REPL for more than two decades now, and I've never found it to be "shoddy". Indeed, in pretty much every Python project I work on, one of the first features I add for development is a standard way to load a REPL with all of the objects that the code works with set up properly, so I can inspect them.
Very obvious example - you can't paste code containing blank lines.
Another example: navigating this history is done line by line instead of using whole inputs.
It's just bare minimum effort - probably gnu readline piped directly into the interpreter or something.
I think they did improve it a lot very recently by importing the REPL from some other Python interpreter but I haven't upgraded to use that version yet so I don't know how good it is now.
> probably gnu readline piped directly into the interpreter or something
That is more or less how the REPL originally was implemented. I think there's more under the hood there now.
I still don't think what you describe qualifies as "shoddy". There are certainly limitations to the REPL, but "shoddy" to me implies that it's not really usable. I definitely would not agree with that.
The predecessors were really popular before it too - MATLAB in engineering in particular, Mathematica also popular in Physics and Maths departments particularly where the symbolic functionality was more useful. I used both in academia and IPython (later renamed Jupyter) was clearly a natural extension of those but open source, and without the baggage of MATLAB (only one function definition per file, etc.)
>Which was true, but maybe not the strongest argument. Why not use a faster language in the first place?
Because most faster languages sucks donkeys balls when it comes to using them quickly and without ceremony. Never mind trying to teach non-programmers (e.g. physics, statistics, etc people) them...
>>> you absolutely don't care whatsoever about performance and can guarantee that you never will.
Those are actually pretty good bets, better than most other technological and business assumptions made during projects. After all, a high percentage of projects, perhaps 95%, are either short term or fail outright.
And in my own case, anything I write that is in the 5% is certain to be rewritten from scratch by the coding team, in their preferred language.
Sure but you're still screwing yourself over on that 5% and for no real reason - there are plenty of languages that are just as good as Python (or better!) but aren't as hilariously slow.
And in my experience rewrites are astonishingly rare. That's why Dropbox uses Python and Facebook uses PHP.
Obtuse statement. There are many ways of speeding up a python project if requirements change.
A painful rewrite in another language is usually the only option in my experience.
If you're really lucky you have a small hot part of the code and can move just that to another language (a la Pandas, Pytorch, etc.). But that's usually only the case for numerical computing. Most Python code has its slowness distributed over the entire codebase.
It’s not painful, that’s the point. You have a working prototype now ready to port. (If the destination language is painful perhaps, but don’t do that.)
I recently ported a Python program to Rust and it took me much less time the second time, even though I write Rust more slowly per-line. Because I knew definitively what the program needed.
And if even that is too much optimizing the Python or adding Cython to a few hot loops is less difficult.
I have also ported a Python program to Rust (got a ~50x speedup) but this was a smallish program, under 10k lines of code.
Porting larger programs is rarely tractable. You can tell that because several large companies have decided that writing their own Python runtimes that are faster is less effort (although they all eventually gave up on that as far as I know).
Could happen with any language. The well-known Spolsky piece was about a C++ to C++ rewrite if memory serves. Blaming repeated poor decisions on a prototyping/glue language is yet another instance. Luckily there's lots of options today to dig out of them.
People use Python for things where performance matters, and it's fine
Probably people at some point were making same arguments about ASM and C. How many people though do ASM these days? Not arguing that for now it is relevant point, obviously Rust / C are way faster.
I doubt it. C is well within 2x of what you can achieve with hand written assembly in almost every case.
Furthermore writing large programs in pure assembly is not really feasible, but writing large programs in C++, Go, Rust, Java, C#, Typescript, etc. is totally feasible.
1980's and 1990's game development says hi.
C compilers weren't up to stuff, that is why books like those from Michael Abrash do exist.
Yep, exactly, It takes time to close the gap so that more and more teams taking that tradeoff
> I doubt it. C is well within 2x of what you can achieve with hand written assembly in almost every case.
Depends what you mean, if you preclude using targeted ASM in your C I think some hot loops can be much slower than 2x.
Of course programs globally written in assembly largely don't make sense.