On: Introducing Pytorch for fast.ai

[sagivo]

> Much to our surprise, we also found that many models trained quite a lot faster on pytorch than they had on Tensorflow.

Would love to see some benchmarks for that claim.

[ActsJuvenile → sagivo]

I benchmarked Keras+TF vs PyTorch CNNs back in May 2017:

1) Compilation speed for a jumbo CNN architecture: Tensorflow took 13+ minutes to start training every time network architecture was modified, while PyTorch started training in just over 1 minute.

2) Memory footprint: I was able to fit 30% larger batch size for PyTorch over Tensorflow on Titan X cards. Exact same jumbo CNN architecture.

Both frameworks had major releases since May, so I am sure these metrics might have changed by now. However I ended up adopting PyT for my project.

[perone → sagivo]

Pytorch is way ahead of tensorflow in terms of cuDNN API usage.

[skierscott → sagivo]

Same here.