You typically get throttled by the exchange when you send too many messages. When your messages are rejected, you monotonically lose money until you can message again. For this reason, I highly doubt anyone is sending an uncontrolled amount of messages to any exchange on purpose because surpassing messaging constraints is unprofitable, which would violate everyone's primary motive.
The orders not meant to be hit is a separate issue from the messaging. These orders are there because of
1) quoting requirements designed to mitigate volatility,
2) to give traders the feeling of depth (illusory or not) because market participants tend to interact with exchanges that look like they have thicker books,
3) to gain queue spot due to any FIFO component of the exchange's matching algorithm in case the price moves to the level where these orders could get executed,
4) to gain order allocation due to any pro rata component of the exchange's matching algorithm.
All of these reasons are controlled by the exchange and to some extent the SEC. The first 2 reasons are the result of the exchange trying to make money by attracting participants. The last 2 reasons are the result of participants rationally reacting to their incentives as dictated by the matching algorithms designed by the exchange.
Right. My point is it is illegal to enter an order into the market with the intent of goading someone into trading or to manipulate the price unless it is a bona-fide offer to buy or sell. Intent matters with regard to illegality, and I don't think anybody questions the 'intent' of most HFT shops.
Creating the illusion of volume where none exists has long been illegal - people used to paint the tape long before HFT exists to achieve the same thing.
You've provided a good description of why HFT do what they do, and one could argue that laws need to be changed to allow this market behavior. (I would disagree) Much of what they do is illegal by present law, but none of the big market players want the law enforced, so the SEC looks the other way.
The illegal activity you're referring to are the bluffs? I'm under the impression most people have not successfully automated bluffs although I have heard of people manually doing it and using algorithms to exit or hedge. In my experience, most HFT does not intend to manipulate markets, maybe solely because this is difficult to automate and there's a lot of lower hanging fruit (not because they are altruistically abstaining from such behavior). It's difficult to automate bluffs because most HFT relies on backtesting and it's very hard to backtest high market impact strategies. In contrast, it is my impression manual traders can get very good at market manipulation.
There is a lot of shady stuff that goes on in trading due to conflicts of interest. Anyone intelligent or informed enough to know what's going on is financially incentivized to be secretive about it. I haven't gotten the impression the SEC is intentionally being incompetent- they really ARE just incompetent because anyone smart enough to realize what's happening does not join or remain in the SEC. Incompetence is the simplest explanation for the crazy rules and fines they have enforced, and the rules obvious to actual traders that they have ignored.
The orders not meant to be hit is a separate issue from the messaging. These orders are there because of
1) quoting requirements designed to mitigate volatility,
2) to give traders the feeling of depth (illusory or not) because market participants tend to interact with exchanges that look like they have thicker books,
3) to gain queue spot due to any FIFO component of the exchange's matching algorithm in case the price moves to the level where these orders could get executed,
4) to gain order allocation due to any pro rata component of the exchange's matching algorithm.
All of these reasons are controlled by the exchange and to some extent the SEC. The first 2 reasons are the result of the exchange trying to make money by attracting participants. The last 2 reasons are the result of participants rationally reacting to their incentives as dictated by the matching algorithms designed by the exchange.