Cool project! Two suggestions, one serious, one frivolous:
- I wonder if you could get more accurate results by using TCP or UDP instead of ICMP. I think traditional traceroute has an option to use UDP, mtr [1] can use TCP or UDP, and tcptraceroute [2] can use TCP.
- This would be a perfect fit for some Talking Heads references. "And you may ask yourself, well, how did I get here?" [3]
1. ICMP seemed a lot faster / more reliable, but you're right that UDP does sometimes bring out better results. Something I want to do is make it use a hybrid approach — sending both ICMP and UDP packets and using the best of both.
ICMP isn't as reliable for trace routes. ICMP typically flows through the management plane instead of the data plane. The management plane is often driven by low-power MIPS processors, which can appear as additional latency in the traceroute results.
When a network device is busy, ICMP may be dropped entirely.
ICMP is a great tool to establish baseline connectivity, assuming the device responds to it. TCP will provide more accurate results (or UDP given the device responds to UDP packets) if you know a specific port is open.
traceroute uses UDP by default. tracert.exe only uses ICMP.
While ICMP isn't as reliable, I think the reasons are a little different. Its not that ICMP gets punted to the management plane in general - i.e. when you "ICMP echo request" ping a host, that packet doesn't go through the management plane of all the network devices in between. You know that as ping doesn't have the latency/loss problems traceroute does.
Rather what is going on is most network device data planes are going to punt _all_ IP packet's "time-to-live exceeded" case to the management plane, and so it will have the problems you describe (latency, throttling).
So why is ICMP still worse? First of all it misses having the UDP/TCP "port" information that allows it to be flow hashed through different routes. Secondly if the route hits any firewalls, they often have very different configuration for ICMP then they do for the TCP/UDP that they are configured to let through.
In IPv4, ICMP is used to send TTL exceeded messages regardless of what upper level protocol was used in the packet that expired in transit. UDP, TCP, ICMP, doesn’t matter.
The slides you linked to are correct, in general, about router slow path behavior, but that isn’t what makes UDP ping “more reliable”. It’s “more reliable”, theoretically, because UDP is less likely to be subject to indiscriminate filtering than ICMP.
Additionally, generally only ICMP sent to or generated from a router goes through the slow path. An IP packet being sent through a router generally goes through that device’s fast path, regardless of the payload type.
We're not disagreeing with the presentation, you're misunderstanding it.
No matter which protocol you use the packet only has one destination, the destination IP, so it does not go through the slow path on any hop on the way just because its ICMP. When a packet hits the TTL limit on a hop it will go through the slow path as the CPU will generate the response ICMP. The protocol used for traceroute makes no difference here.
Interesting, I didn't know traceroute used UDP by default. Just did some testing with the best traceroute tool, mtr[0], and using the -u flag for UDP seemed to give the worst results. -t for TCP seemed OK. I never thought about the difference between how TCP and ICMP packets are handled.
You can use the -P flag to set the protocol (TCP, UDP, ICMP Other, GRE) and -p to set the port number, if applicable.
Microsoft should have stuck to the BSD traceroute. I don't know why they "invented" tracert.exe (which I believe is _based_ on BSD traceroute, like most of those early network tools in Windows). I wonder if this was a Winsock (or STREAMS[0]!) limitation.
I assume that tracert using ICMP is not that much an WinSock limitation but limitation of the underlying NT security architecture without suid binaries, which was worked around with ICMP-only non-privileged “raw sockets”. But that is mostly an educated guess.
This is a limitation that dates back to NT 3.1 where SOCK_RAW was callable by standard User accounts. Only starting with Windows 2000 [0] does creating a RAW socket require local administrator rights.
"Reliable" may have been crufty wording but I did get more responsive results using it. MTR uses ICMP by default. Definitely want to do a hybrid thing in the future though!
Traffic to vs through a router; both UDP and ICMP probes to a given hop will both go to the control plane which can be rate limited or handled by a general purpose CPU.
Probes to hops beyond will virtually always go through the fast data plane.
- I wonder if you could get more accurate results by using TCP or UDP instead of ICMP. I think traditional traceroute has an option to use UDP, mtr [1] can use TCP or UDP, and tcptraceroute [2] can use TCP.
- This would be a perfect fit for some Talking Heads references. "And you may ask yourself, well, how did I get here?" [3]
[1] https://github.com/traviscross/mtr
[2] https://linux.die.net/man/1/tcptraceroute
[3] https://en.wikipedia.org/wiki/Once_in_a_Lifetime_(Talking_He...