Hope you can help, me, I've got

Hello, hope you can help, me, I’ve got an idling problem on my 2004 GDI 3.5litre V6 petrol Shogun Elegance. When I start the vehicle from cold, that is after she has been parked overnight outside my home, the engine revs to about 1500 to 2000rpm then the idle speed (with no throttle applied) quickly settles down to about 1000rpm. After a few minutes driving the idle reduces until at normal operating temperature it settles at or around 500rpm to 600rpm.
I have a thirty minute drive to work and when I arrive the idle is steady at 500rpm to 600rpm.
When I start the car for my journey home the engine revs again about 1500rpm to 2000rpm. The idle then starts to fall but then jumps up to about 2000rpm briefly and then drops again. This idle surging continues for a number of minutes, gradually reducing, until the engine reaches operating temperature and the idle settles down to the steady 500rpm to 600rpm.
The surging has nearly cause a problem while trying to exit a parking space in a car park, the car surged back and I just caught her on the brakes avoiding a collision with a parked vehicle behind.
The idle surging did not occur on a recent trip of about 50miles, parked for about 90minutes and then returned home.
I have spoken to a couple of garages and they thought the problem could be either a partially blocked fuel filter or a faulty fuel pump. The vehicle shows no sign of petrol starvation such as missing or pulling back when driven hard.
Could you help me identify the issue.
Many thanks for your help.
John Mason