Hacker News new | past | comments | ask | show | jobs | submit login

SSD is probably not as fast as RAM, but it's much much cheaper, in the order of 10x per gigabyte. With SSD-GPU bridge you can have fast access to a multiple TiB training set, on a single machine.

Data pre-processing is indeed an issue, but hue adjustment/flipping/cropping could be implemented as Tensorflow operations, on the GPU. Similarly with input decompression - it would either have to be done on GPU, or the data would have to be stored uncompressed.




As long as the average bandwidth isn't a bottleneck, it's not going to matter - at worst, you're just going to need to prefetch (and due to SSD latency, that's likely optimal regardless).




Consider applying for YC's Spring batch! Applications are open till Feb 11.

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: