If this is a documented right-of-way in the deeds, either created by deed or acquired through prescription/longer use, then it is virtually impossible to get it removed unless the dominant and the servient land both come into common ownership or both owners agree to extinguish the right.
The right does not cease to exist through lack of use regardless of the period of time and it does not cease to exist and cannot be removed because it is no longer required for example if there is an alternative.
Anyone exercising the right must not cause nuisance or obstruction and both of those would be actionable separately but not to the extent of removing the right.
The question is very common.
I do not know whether it’s the answer you wanted or not.
Can I clarify anything else for you?
Please rate the service positive. It’s an important part of the process by which experts get paid.
We can still exchange emails.
If it’s your field, you can remove the gate and do what you like with it.
If you want to fix it in place so that it will not come off the hinges and he cannot remove it, you are also entitled to do that. You would not be entitled to lock it because that would obstruct his right-of-way.
If he has damaged it, you can ask him for the money to repair and if he will not pay up, you can take him to the Small Claims Court.
If its yours you can make it unremoveable