The majority of the Linux kernel's source code is device drivers. The overwhelming majority of that is not included in the kernel image by default, but instead made available as kernel modules you can enable as needed. E.g., your thermostat probably doesn't need support for an obscure game controller, so doesn't have those drivers, but it could if you were so inclined.
As someone who knows nothing about anything: that doesn’t mean the tcp/ip stuff is half the source code of the whole kernel, does it?