Women with premature ovarian failure do need more estrogen than women undergoing usual menopause, just as you have said. The reasons seem to be related to the fact that women normally have higher estrogen levels than what is given for menopause, and they have more estrogen receptors.
So it's likely that you need more Vagifem than the average menopausal woman.
It's a little unusual that your vaginal symptoms came back so quickly and so strongly within 24 hours of stopping the Vagifem. But the fact that Vagifem did relieve your symptoms seems good evidence that the symptoms are due to low estrogen.
I would recommend taking the Vagifem daily and continuously. I think that this is what your consultant would also recommend. He/she recognized that you need more by giving the Vagifem for 28 days initially.
I think that many women with POF would be OK on Vagifem twice weekly after the initial daily dose, but it seems that you're not one of them. Best wishes.