On: Hardware for Deep Learning, Part 3: GPU

[jaimex2]

Tesla apparently designed their own chip specifically for neural net vision as GPUs were not fast enough - including nVidias Tesla chip which is currently being put in their cars.

Musk mentioned they got a 10x boost from doing so in their last investor call:

"I'm a big fan of NVIDIA, they do great stuff. But using a GPU, fundamentally it's an emulation mode, and then you also get choked on the bus. So, the transfer between the GPU and the CPU ends up being one of the constraints of the system"

"And as a rough sort of whereas the current NVIDIA's hardware can do 200 frames a second, this is able to do over 2,000 frames a second and with full redundancy and fail-over."

Peter Bannon lead the development.

[che_shr_cat → jaimex2]

Many companies designed or designing their own chips. The most prominent example is Google TPU.

There will be another articles in this series on FPGAs, ASICs and so on.

[godelmachine → che_shr_cat]

Eagerly waiting for the one on FPGA’s and ASIC’s

[che_shr_cat → godelmachine]

The original plan was this:

https://blog.inten.to/hardware-for-deep-learning-current-sta...

[godelmachine → che_shr_cat]

http://www.cs.cornell.edu/courses/cs6787/2017fa/Lecture11.pd...

[godelmachine → che_shr_cat]

Thanks for this.

There was also a PDF published by some university on Hardware required on DL operations. Must be somewhere in my submitted posts here

[jaimex2 → che_shr_cat]

Awesome! Looking forward to it.