You are right and IDK why you are downvoted. Few units of perceptrons, few nodes in a decision tree, few of anything - they are "interpretable". Billions of the sames - are not interpretable any more. This b/c our understanding of "interpretable" is "an array of symbols that can fit a page or a white board". But there is no reason to think that all the rules of our world would be such that they can be expressed that way. Some maybe, others maybe not. Interpretable is another platitudinous term that seems appealing at 1st sight, only to be found to not be that great after all. We humans are not interpretable, we can't explain how we come up with the actions we take, yet we don't say "now don't move, do nothing, until you are interpretable". So - much ado about little.
