Hacker News new | past | comments | ask | show | jobs | submit login

The article starts with "You have a large chunk of data — a NumPy array, or a Pandas DataFrame — and you need to do a series of operations on it."

So it's suggesting to use these techniques when you already know that you are working with a large chunk of data, either because it was obvious from the beginning or because profiling highlighted it. In scientific computing it is common to store gigabytes of data in a single array. In fact it is common that even the compromise-solution suggested in this article, which leads to 2x memory instead of 3x, would be too much extra overhead.

But the compromise solution is nice if you can afford 2x memory overhead, so I upvoted the article.




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: