As someone who recently switched from DS to platform engineering, this post really captures why I also jumped ship.
> Shitty code & shitty data science
In my opinion the bar should be higher for code quality but also general engineering know-how in data science. You'd be surprised how many are uncomfortable with git, using the command line, interacting with APIs, managing environments, etc. Being able to only work within a jupyter notebook is not good enough, at all. Otherwise, you end up with people who's entire job it is to productionize and deploy the code which is a waste of time and effort.
> Poor mentorship
There is either a lack of quality leadership and mentorship or an inability for upper management to see the value in hiring for it. What ends up happening is you have a team who doesn't know how to grow, scale, or work together. They instead focus on building models and learning statistics when they should be focusing on building systems and process for helping the business scale analytics and building models when appropriate.
I enjoyed data science but found it to also not matter in the implementation that everyone thinks it should be. Data science isn't building nothing but ML models. In most companies, in my opinion, it is actually about scaling analytics. Being able to reach further up into data engineering, get raw data, explore it, shape it, give it back to DE to automate, and then automate the delivery of data to upper management and guide them through using it. If the team thinks their job is to just build models, everyone is going to have a miserable, miserable time.
> Shitty code & shitty data science
In my opinion the bar should be higher for code quality but also general engineering know-how in data science. You'd be surprised how many are uncomfortable with git, using the command line, interacting with APIs, managing environments, etc. Being able to only work within a jupyter notebook is not good enough, at all. Otherwise, you end up with people who's entire job it is to productionize and deploy the code which is a waste of time and effort.
> Poor mentorship
There is either a lack of quality leadership and mentorship or an inability for upper management to see the value in hiring for it. What ends up happening is you have a team who doesn't know how to grow, scale, or work together. They instead focus on building models and learning statistics when they should be focusing on building systems and process for helping the business scale analytics and building models when appropriate.
I enjoyed data science but found it to also not matter in the implementation that everyone thinks it should be. Data science isn't building nothing but ML models. In most companies, in my opinion, it is actually about scaling analytics. Being able to reach further up into data engineering, get raw data, explore it, shape it, give it back to DE to automate, and then automate the delivery of data to upper management and guide them through using it. If the team thinks their job is to just build models, everyone is going to have a miserable, miserable time.