Capturing and punishing thieves has a deterring effect on other thieves. Police spending resources to get "your $200 headphones back" saves more than just the $200 headphones, since usually thieves are repeat offenders and it discourages would-be thieves from becoming thieves. If you think about it, if a police officer makes $100/h you could conceivably make the case that catching a single thief could potentially save local taxpayers who had stuff stolen the officer's full time salary for an entire week. I'd say that's worth it.
Having a working law enforcement system that people can trust also means people can rely on the law to be enforced to protect them, instead of taking the law into their own hands.
Take a look at this phone thief at a concert [1].
[1]: https://www.youtube.com/watch?v=zBb33bKG1hQ