Hacker News new | past | comments | ask | show | jobs | submit login

So, you've made all that effort, how does it help you in your new role as a data scientist? Is there anything you do now that requires "mathematical maturity"? Or is it something that can be learned much quicker on as needed basis?



There are a lot of charlatans in the Data Science space who lack the necessary mathematical background for their roles. For me it was necessary to get a rigorous understanding of probability theory, and applied probability theory is basically what mathematical statistics is about. My background was CS and software so R, Python, data visualisation and ML operationalization is by and large the easy part of Data Science to me. If you pick up any book like Bishop's or ESL you will be extremely frustrated if your mathematical background is not there. I didn't feel comfortable creating predictive models for production use that I didn't completely understand what was going on "under the hood", the assumptions being made and how they could fail. It's the only ethical thing for any engineer to do.


I do deep learning research for a living. I've taken graduate classes in probability, stochastic processes, optimization algorithms, and signal analysis (ECE PhD). I almost never completely understand what's going on under the hood of my models as soon as they get larger than a single neuron XOR mapper. That does not prevent me from finding ways to improve the performance of very large models (millions of parameters and dozens of layers). I agree that there are some papers (or the two books you mentioned) that can be quite dense and heavy on math, but I can't say I've ever felt like I needed any math other than basic calculus, linear algebra, and prob/stats 101 to understand almost all ML methods that people actually use in real world. Obviously if you want to make breakthroughs in theoretical ML, then sure, you do need the mathematical maturity (mostly because you will need to be formally proving things), but if you're a regular data scientist? Can you give some example what kind of math is involved in your predictive models?




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: