The United States is not the entirety of the world. There are many other countries that are federations, and some of them have English as one of their national languages. I can assure you that in those countries, colloquial use of "feds" does not refer to the American federal government.