The problem Boeing had was that there is a massive amount of 737:s, and a correspondingly massive pool of crew qualified for those planes. Customers really wanted not just a plane that was compatible with something, but specifically compatible with the 737. And this was a problem because the 737 is too old of a design.
If Boeing attempted to sell these customers a new type, even if there is some compatibility with some other plane, it creates an opportunity for them to freely compete bids between Airbus and Boeing, and even when Boeing wins that bid, it's a loss to them. (Because they are going to win at a lower price point than if the customer has to account for the switchover cost only for Airbus planes.)
If Boeing attempted to sell these customers a new type, even if there is some compatibility with some other plane, it creates an opportunity for them to freely compete bids between Airbus and Boeing, and even when Boeing wins that bid, it's a loss to them. (Because they are going to win at a lower price point than if the customer has to account for the switchover cost only for Airbus planes.)