Okay, I see you have rated the answer as good for me, which ended the chat, so I wanted to answer more fully here for you.
In short, if they have rights to get to their property via the pathway of their neighbour, then they will be able to do this and you will not be able to prevent them from carrying the scaffolding this way to do what they intend to do. Also, as you say, they could take tye scaffolding through their house if needs be. You will not be able to prevent this happening.
However, you might be able to stop them taking it through your garden via the accessway, but this depends on the terms of the right of way, which might have been expressly granted for the benefit of all properties to use equally.
Also, even if his neighbour tried to prevent using the accessway, he may have rights under the Access to Neighbouring Land Act 1992 which would require the entitlement to enter through his land to erect scaffolding to maintain an existing structure.