Hello Ian my name is ***** ***** I will help you with this.
Just to be clear, the fence is yours? Is the fence on your land or joint?
If it is yours or a joint fence then the answer is no.
This would be your property and permission would be needed. Otherwise it would be criminal damage as it belongs to you.
So you could get the Police involved who could investigate and prosecute.
Can I clarify anything for you about this today please?
All the best.