We have the right to allow only those we wish to onto our property save where the organisation has a right to enter, police etc.
As such the person using your property is trespassing and committing a nuisance on your property and you can threaten legal action. You would need to know who they are and where they live.
A cheaper option may be to put gates up to physically prevent it.
I hope this helps, if there are any further points please reply