contain nitrogen. (TATP is not used as a military explosive because it will blow up if you look at it the wrong way.) In a lot of cases that is going to come from nitric acid which in turn is made from ammonia which could otherwise be used to make fertilizer.
As you point out, Ammonium Nitrate is a powerful explosive which is commonly used in mining, road construction and other peaceful purposes. Ammonium Nitrate is also popular among terrorists because it is easy to get and highly effective in the form of a truck bomb.
What's the ratio of "nitrates input" for fertilizer vs explosives? A ton of ammonia is fertilizer all by itself but processed might only be 100kg of TNT (totally ignoring all the other inputs).