They should not have been on your property without permission but under the Access to Neighbouring Land Act if this was work required to preserve next-door’s property, (replace the fence), you have to give them access. They could apply to court to compel you to give it but they mustn’t cause any damage and if they do, they are liable to put it right.
The reality I’m afraid to tell you is that as this has now been and gone, there is very little that you can do about it, except to say that they must ask you if they want permission to do anything like this again. The practicality however is that if they don’t, you are forcibly making an application to court for a court order for them to for consent, consent which ultimately you are going to have to give them anyway. Nonetheless, it is trespass although trespass is not generally actionable unless there is damage or nuisance on an ongoing basis.
With regard to the gap, I haven’t seen it but if they have created a trap, they would be liable for any injuries resulting from the trap. Give them notice that if anyone is injured as a result of getting their foot caught in this trap, you will hold them liable but as you are now aware of the potential trap it also put a duty on you to make sure that your children are kept safe from harm by being aware of the potential from this.
I know that the majority of this is not what you wanted to hear but that is the legal situation.
Can I clarify anything for you? Please rate the service positive so that I get paid. We can still exchange emails. Best wishes. FES.