On: Being a Data Scientist: My Experience and Toolset

[achompas]

That's only part of it. A good data scientist is also good because they know how to answer hard questions.

In those situations math isn't "shitty trivia," but instead a tool to be leveraged against those hard questions.

You can consider the derivation of SVD to be shitty trivia while throwing np.linalg.svd around while engineering features. That's fine! Good luck visualizing that data in a meaningful way, or dealing with non-linear data, if you're ignoring that "shitty trivia."

[marketforlemmas → achompas]

> dealing with non-linear data

What is non-linear data?

[bigger_cheese → marketforlemmas]

Data derived from non linear inputs.

That is to say problems that can't be expressed by linear functions.

I.e. Y= mx + B is a linear function.

Y= ax^2 + bx + C is a polynomial (non linear) function.

Linear Programming (LP) involves solving a series of linear equations (something like Excel's Solver can do this).

When you are dealing with non linear functions you need to use a method such as Sequential Quadratic Programming (SQP).

[q_revert → marketforlemmas]

Using a term like nonlinear science is like referring to the bulk of zoology as the study of non-elephant animals.

— Stanislaw Ulam

https://en.wikipedia.org/wiki/Nonlinear_system