Can you form the company in e.g. Luxembourg and then operate in France? I feel that this is the sort of thing the Single Market was supposed to make easy.
France will force the hell out of you to open a branch in France. I have a company in Estonia and France is the only country that asks me to do such thing in the EU. Luckily I live in France, you'll need an address here, and have fun opening a business bank account with papers in, say, Estonian, or even English for that matter. Spent thousands in translations, whereas creating the company in Estonia was over the Internet and under 300 EUR. France will do all it can to drag you into its bureaucratic hell.