I don’t think that the failure to submit an application to building control would be sufficient to rescind the contract. It’s easy enough to make the submission to put that right.
You say you could have got in trouble for it, all they would do is say that you needed to make the application. He is saying that you thrown off because the window delivery took longer than expected, something outside his control.
It appears that you have paid GBP3400 from a total cost of GBP4900 leaving, somewhat obviously, GBP1500 outstanding. But he has only done 50% of the work.
I think you need to explain that to him in words of one syllable. You need to say exactly that.
However he cannot keep pestering you for money otherwise that’s harassment.
Harassment is defined as a course of action (more than once) which is intended to harass, does harass, is reckless as to whether it harasses, or if the average bystander would look at the conduct and think that it was indeed harassment.
Harassment is both a civil matter and a criminal matter.
If a person is being harassed they should go to the police and ask them to warn the alleged offender under the Protection from Harassment Act. Some police forces take that very seriously and others not so and therefore, a person may need to speak to a senior officer or put a complaint in writing to the Chief Constable.
It’s also a breach of the provisions of section 40 of the Administration of Justice Act in respect of unlawful harassment of debtors.
If you are not going to pay him, he’s going to have to take you to court and you are going to have to defend it.