Though, I feel that ANNs' number of neurons are under-represented is some ways:
- In the example case of a CNN, the total weights of a kernel is of [Input Number of Channels * Input Kernel Size * Number of Filters], which can be a pretty small amount when it comes to for example a 3x3 kernel with 3 channels with 128 filters coming to a total of 3,456 parameters (3 * 3 * 3 * 128), however in the case of an ANN the same filter is strided across the entire 2D input feature map (or 3D for 3D CNNs). If the input image is of HD resolution of 1280 * 720 and the stride is 2 across both dimensions, then the number of strides is 230,400. The effective number of parameter activations is 796,262,400 (3,456 * 230,400). The reason for this example is that it is sort of a known thing for likely decades now that CNNs are inspired in part by the human visual cortex [0]. For the human visual cortex which needs to be fast, there cannot be parameter sharing across a single kernel, and likely the weights would need to be parallelized to an extent, which would theoretically imply duplicating the weights across the human brain. Thus, the advantage out here lies with ANNs.
- The neurons in the human brain would have to have a certain level of redundancy in place due to the constant cellular repair work.
- The neurons in the human brain can seemingly only be updated by Hebbian learning rather than direct updates which is in the case of the computer memory of ANNs.
- Finally, a significant part of the human brain is for non-logical but environmental reasons, such as movement and touch, and non-logical things such as fear, jealousy, lust, etc; parts which ANNs do not need to possess in the same way (eg: the fight-or-flight response of the amygdala part of the brain).
- In the example case of a CNN, the total weights of a kernel is of [Input Number of Channels * Input Kernel Size * Number of Filters], which can be a pretty small amount when it comes to for example a 3x3 kernel with 3 channels with 128 filters coming to a total of 3,456 parameters (3 * 3 * 3 * 128), however in the case of an ANN the same filter is strided across the entire 2D input feature map (or 3D for 3D CNNs). If the input image is of HD resolution of 1280 * 720 and the stride is 2 across both dimensions, then the number of strides is 230,400. The effective number of parameter activations is 796,262,400 (3,456 * 230,400). The reason for this example is that it is sort of a known thing for likely decades now that CNNs are inspired in part by the human visual cortex [0]. For the human visual cortex which needs to be fast, there cannot be parameter sharing across a single kernel, and likely the weights would need to be parallelized to an extent, which would theoretically imply duplicating the weights across the human brain. Thus, the advantage out here lies with ANNs.
- The neurons in the human brain would have to have a certain level of redundancy in place due to the constant cellular repair work.
- The neurons in the human brain can seemingly only be updated by Hebbian learning rather than direct updates which is in the case of the computer memory of ANNs.
- Finally, a significant part of the human brain is for non-logical but environmental reasons, such as movement and touch, and non-logical things such as fear, jealousy, lust, etc; parts which ANNs do not need to possess in the same way (eg: the fight-or-flight response of the amygdala part of the brain).
[0] https://msail.github.io/post/cnn_human_visual/