The finance company owns the car here. You do have consumer rights which have been breached. As the car is not of satisfactory quality, this is in breach of the finance agreement. You should make the payment - the finance company should refund you once this is resolved.
You are covered under the Consumer Rights Act 2015 here which means you have a right to expect a service (from the finance company) carried out with reasonable care and skill. Furthermore, the car should be “fit for purpose”, "free from defects" and “as described”. If it isn’t then you can allege breach of your consumer rights and breach of contract.
You should now make a formal complaint and if that does not resolve the matter ask them for a "deadlock letter", which is a letter giving their final response.
Once you have the deadlock letter the next step would be to escalate this to the Ombudsman - they will investigate and liaise with the finance company which would hopefully result in a refund. The finance company should arrange to return the car to the dealer or to sell it to another company. They can order the finance company to make a financial award for inconvenience if they have acted poorly. The ombudsman scheme is free to use and quicker than court action.
Once you have their final response, you can make the complaint here : www.financial-ombudsman.org.uk/consumers/how-to-complain
Based on the circumstances, assuming the finance company does not uphold your complaint, I am sure the Ombudsman will do. The Ombudsman will look at this case independently and will make a decision based on what happened. Their decision is binding on the finance company, but not you. So you would not be bound in law to follow their decision. If the Ombudsman did not find in your favour, therefore, you then have the option of suing the finance company for breach of your consumer rights and you have up to 6 years to bring a claim to the court (the limitation period is 6 years from your date of loss, to when you need to issue a claim).
If you wanted a law firm to do the above, any of these could help you further :