If you sell the property for £650,000 in September 2013 having paid £312,500 for it in September 1999 and spent £50,000 improving it, you will make a gain of £287,500, £143,750 for each of you.
Of the total period of ownership, you will each be entitled to CGT exemptions for the following periods:
September 1999 to May 2006 Main residence relief 81 months.
June 2006 to July 2008 Main residence relief due to absence abroad for work purposes 26 months.
August 2008 to May 2009 Main residence relief due to absence in the UK for work purposes 10 months.
June 2009 to September 2010 Letting relief 16 months.
October 2010 to September 2013 Relief for the last 36 months of ownership or three years for any reason.
As you will see, the whole period of ownership of the property is covered by reliefs of one sort or another and you should have no Capital Gains Tax to pay.
You will only be entitled to relief for the period you worked abroad and in the UK too far from your home to live in it if it was your employer's decision to have you work in those places as opposed to your choosing to take a job in those places. I'm also assuming that you did not buy a property to live in whilst abroad. Take a look at the HMRC helpsheet HS283
(CG65013 to CG65068) for more information on main residence reliefs and CGT.
I hope this helps but let me know if you have any further questions.