Hello and thanks for using Just Answer.
My name is ***** ***** am happy to assist you with your enquiry.
Provided no work has been carried out by you in administering the Estate, you can step down as Executor if you feel you do not wish to act. To do this, you will need to sign an official Form, called a Deed of Renunciation.
A local Wills and Probate Solicitor will need to prepare this Deed.
If you do decide to do this, then that just leaves your Brother a sole Executor, and the only person entitled to administer your Father's Estate in accordance with his Will.
I hope this assists.
Can I help any further?