What is the problem?
I need further details before I can assist.
Did you agree to the panels being removed?
Is it your fence or shared?
Unfortunately, in law, there are no obligations to fence a boundary UNLESS specifically stated in the title deeds. I would say that specific obligations in deeds relating to fencing are uncommon. Do check the deeds though, just in case.
This means, that unless there is a fencing obligation in the deeds, there is very little you can do legally if your neighbour does not reinstate the fence.
Your only option is to erect a fence on your side of the boundary.
I am sorry if this is not considered helpful but it is the legal position in your situation.
Any questions or queries?
Please can I kindly request that you accept / positive rate my reply otherwise the website will not pay me for assisting you.