Hello. I don't think it will be the fuel pump.
If it runs ok when you do get it started and has no power loss then the pump is probably ok.
The poor starting is more likely to be due to low cylinder compression.
This could be caused by leaking injectors which washes away oil residue from the piston rings and so lowers the compression.
Once the engine does start this compression returns and will not cause any power loss whilst driving. It also accounts for the engine then starting each time you try it because the leaking injectors take time to wash the oil away.