I would suggest that you inform the agents that you refuse to pay as your consent was not obtained. Further, you should inform the agent that unless they make good any damage caused by their contractors, you will pursue the matter through the courts.
The agent cannot make you pay. If you fail to agree, either party will have the right to issue proceedings through the courts.
I hope this answers your questions. If so, kindly rate the answer and provide feedback.
If you have any further questions, please do not hesitate to ask.
There is no specific legislation and it is based on common law, that is established caselaw over the years.
In any event, if you refuse to meet the agents demands, they will have to issue proceedings against you to recover the sum demanded. You will have an opportunity to submit a defence and counterclaim and it will then be to a county court judge to make the final decision.