Hacker News new | past | comments | ask | show | jobs | submit login

Exactly! DataChain does lazy compute. It will read metadata/json while applying filtering and only download a sample of data files (jpg) based on the filter.

This way, you might end up downloading just 1% of your data, as defined by the metadata filter.




Join us for AI Startup School this June 16-17 in San Francisco!

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: