Have Tax Questions? Ask a Tax Expert for Answers ASAP
It really doesn't matter where you have your bank account as far as tax is concerned. There is no tax reason why you should not have your private pension paid into a UK bank account
If you are not a UK tax resident and you are a tax resident of France, you should only be paying tax on your state and private pensions in France. The UK state pension is paid without deduction of tax at source. The private pension should have a NT (No Tax) code operated on it by the pension payer. You would need to contact the UK tax office on the number here to have that code issued if it has not been issued already. They may ask you to complete a P85. See Article 18 of the UK/France double tax treaty here.
However, if your private pension is in fact an occupational pension derived from working in a government service job (see here) such as the military, the civil service or in some cases as a teacher then that pension will remain taxable in the UK only unless you are not a UK national and are a French national and a resident of France in which case it will be taxable only in France. See Article 19, paragraph 2 of the UK/France double tax treaty here. If that was the case, you would be entitled to a personal allowance to use against it.
You might be interested in an article in the Daily Telegraph from July 2013 which suggests that France has stopped taxing UK nationals on their UK pensions which as far as I can see completely contradicts the tax treaty. There is a more recent article here. The author of the article has confused a government service pension with a state pension but apart from that he confirms the authority of the tax treaty.
I hope this helps but let me know if you have any further questions.