> If you are writing a system where performance does not matter, then performance does not matter.
It's not binary. Performance always matters, but there are different levels of value to that performance. Writing hand-tweaked assembly code is rarely a good point on the ROI curve.
> Wait, what? ML engine backends and high-level descriptions, and video games are some of the most heavily tuned and optimized systems in existence.
Indeed they are. And the major language most machine learning researchers use is Python. There is highly-optimized vector code behind the scenes, which is then orchestrated and controlled by tool chains like PyTorch and Python.
> Take a look at C++, D, Rust, Zig and others. In those, changing a data structure or algorithm is as easy as in your usual JIT one like C#, Java, Python, etc.
I used to think that too before I spent years doing functional programming. I was an incredible C++ hacker, and really prided myself on being able to implement things like highly-optimized numerical code with templates. I understood every nook and cranny of the massive language. It actually took a few years before my code in Lisp, Scheme, JavaScript, and Python stopped being structured like C++.
You putting "Python" and "Java" in the same sentence shows this isn't a process you've gone through yet. Java has roughly the same limitations as C and C++. Python and JavaScript, in contrast, can be used as a Lisp.
I'd recommend working through SICP.
> You seem to be implying that a low-level language disallows you from properly designing your application. Nonsense.
Okay: Here's a challenge for you. In Scheme, I can write a program where I:
1) Write the Lagrangian, as a normal Scheme function. (one line of code)
2) Take a derivative of that, symbolically. (it passes in symbols like 'x and 'y for the parameters). I get back a Scheme function. If I pretty-print that function, I get an equation render in LaTeX
3) Compile the resulting function into optimized native code
4) Run it through an optimized numeric integrator.
This is all around 40 lines of code in MIT-Scheme. Oh, and on step 1, I can reuse functions you wrote in Scheme, without you being aware they would ever be symbolically manipulated or compiled.
If you'd like to see how this works in Scheme, you can look here:
That requires being able to duck type, introspect code, have closures, GC, and all sorts of other things which are simply not reasonably expressible in C++ (at least without first building a Lisp in C++, and having everything written in that DSL).
The MIT-Scheme compiler isn't as efficient as a good C++ compiler, so you lose maybe 10-30% performance there. And all you get back is a couple of orders of magnitude for (1) being able to symbolically convert a high-level expression of a dynamic system to the equations of motion suitable for numerical integration (2) compile that into native code.
(and yes, I understand C++11 kinda-added closures)
> And the major language most machine learning researchers use is Python.
Read again what I wrote. Even the model itself is optimized. The fact that it is written in Python or in any DSL is irrelevant.
> I used to think that too before I spent years doing functional programming.
I have done functional programming in many languages, ranging from lambda calculus itself to OCaml to Haskell, including inside and outside academia. It does not change anything I have said.
Perhaps you spent way too many years in high-level languages that you have started believing magical properties about their compilers.
> prided myself on being able to implement things like highly-optimized numerical code with templates.
Optimizing numerical code has little to do with code monomorphization.
It does sound like you were abusing C++ thinking you were "optimizing" code without actually having a clue.
Like in the previous point, it seemed you attributed magical properties to C++ compilers back then, and now you do the same with high-level ones.
> It actually took a few years before my code in Lisp, Scheme, JavaScript, and Python stopped being structured like C++.
How do you even manage write code in Lisp etc. "like C++"? What does that even mean?
> You putting "Python" and "Java" in the same sentence shows this isn't a process you've gone through yet. Java has roughly the same limitations as C and C++.
Pure nonsense. Java is nowhere close to C or C++.
> Here's a challenge for you.
I would use Mathematica or Julia for that. Not Scheme, not C++. Particularly since you already declared the last 30% of performance is irrelevant.
You are again mixing up domins. You are picking a high-level domain and then complaining a low-level tool does not fit nicely. That has nothing to do with the discussion and we could apply that flawed logic to back any statement we want.
> Perhaps you spent way too many years in high-level languages that you have started believing magical properties about their compilers.
> It does sound like you were abusing C++ thinking you were "optimizing" code without actually having a clue.
> Like in the previous point, it seemed you attributed magical properties to C++ compilers back then, and now you do the same with high-level ones.
I think at this point, I'm checking out. You're making a lot of statements and assumptions about who I am, what my background is, what I know, and so on. I neither have the time nor the inclination to debunk them. You don't know me.
When you make it personal and start insulting people, that's a good sign you've lost the technical argument. Technical errors in your posts highlight that too.
If you do want to have a little bit of fun, though, you should look up the template-based linear algebra libraries of the late nineties and early 00's. They were pretty clever, and for a while, were leading in the benchmarks. They would generate code, at compile time, optimized to the size of your vectors and matrixes, unroll loops, and similar. They seem pretty well-aligned to your background. I think you'll appreciate them.
It's not binary. Performance always matters, but there are different levels of value to that performance. Writing hand-tweaked assembly code is rarely a good point on the ROI curve.
> Wait, what? ML engine backends and high-level descriptions, and video games are some of the most heavily tuned and optimized systems in existence.
Indeed they are. And the major language most machine learning researchers use is Python. There is highly-optimized vector code behind the scenes, which is then orchestrated and controlled by tool chains like PyTorch and Python.
> Take a look at C++, D, Rust, Zig and others. In those, changing a data structure or algorithm is as easy as in your usual JIT one like C#, Java, Python, etc.
I used to think that too before I spent years doing functional programming. I was an incredible C++ hacker, and really prided myself on being able to implement things like highly-optimized numerical code with templates. I understood every nook and cranny of the massive language. It actually took a few years before my code in Lisp, Scheme, JavaScript, and Python stopped being structured like C++.
You putting "Python" and "Java" in the same sentence shows this isn't a process you've gone through yet. Java has roughly the same limitations as C and C++. Python and JavaScript, in contrast, can be used as a Lisp.
I'd recommend working through SICP.
> You seem to be implying that a low-level language disallows you from properly designing your application. Nonsense.
Okay: Here's a challenge for you. In Scheme, I can write a program where I:
1) Write the Lagrangian, as a normal Scheme function. (one line of code)
2) Take a derivative of that, symbolically. (it passes in symbols like 'x and 'y for the parameters). I get back a Scheme function. If I pretty-print that function, I get an equation render in LaTeX
3) Compile the resulting function into optimized native code
4) Run it through an optimized numeric integrator.
This is all around 40 lines of code in MIT-Scheme. Oh, and on step 1, I can reuse functions you wrote in Scheme, without you being aware they would ever be symbolically manipulated or compiled.
If you'd like to see how this works in Scheme, you can look here:
https://mitpress.mit.edu/sites/default/files/titles/content/...
That requires being able to duck type, introspect code, have closures, GC, and all sorts of other things which are simply not reasonably expressible in C++ (at least without first building a Lisp in C++, and having everything written in that DSL).
The MIT-Scheme compiler isn't as efficient as a good C++ compiler, so you lose maybe 10-30% performance there. And all you get back is a couple of orders of magnitude for (1) being able to symbolically convert a high-level expression of a dynamic system to the equations of motion suitable for numerical integration (2) compile that into native code.
(and yes, I understand C++11 kinda-added closures)