Hello. Provided the boundary hedge is yours and not on a party boundary, you can remove it and replace it with a fence. You might need to check your deeds to see if the boundary is a party boundary.
I am the one that put the hedge in first, but what is a party boundary ?
It is a boundary feature which is jointly owned by the neighbours. I doubt this arises here, so by all means, remove the hedge and put in your fence. Provided it is entirely on your side, your neighbour can't complain. Best, WB