thank you. My apologies for being across purposes of this point and this paints a very different picture. From what you describe, what has occurred, is at some point, an individual has taken it upon themselves to lay some form of surface on land which was neither owned by them and from what you describe, was also probably adopted highway verge.
There is no legal right to lay a surface on land which is not owned by you and as such, the individual in question is guilty of trespass in this respect. In addition, there would be no right of way either public or private over that area by vehicle is no vehicular public right of way exists over highway verge.
if you use this as a means of access, you would need to show evidence of 20 years of use in order to establish what is known as a prescriptive right of way. In the absence of this, the local authority can require that you no longer use the area for vehicular access.
Unless they can demonstrate it was you that they'd the surface, notwithstanding the above, I cannot see that they can insist that you remove the surface as you can simply say that you are not responsible for laying the surface. If the local authority cannot identify who laid the surface then it falls to them to restore the area to the condition in which they require it to be maintained