There is no key. It is 2 separate farm gates. It is an old buiding split into 4 dwellings and there is a share right and responsibility for upkeep of the grounds and house and splitting bills that is governed by a trust.
The exact wording of the relevant restrictions on the covenant say
Restriction 4. "Nothing shall be done or omitted or suffered to be done in any dwelling or any part of the land attached to any dwelling which may be or become or tend to be or become a nuisance or annoyance or damage to the owners or occupiers of any of the others of the three dwellings or the owners or occupiers of any neighbouring property"Restriction 9 "No fences or boundaries are to be erected in any part of the garden other than those existing save with the prior agreements of all such owners"
So the problem is really the erection of the boundary without consent.
I had hoped that if that could be construed as breach of the peace then I could involve the police.