Toyota Repair Problems? Ask a Mechanic for Answers ASAP
If the engine is stone cold you can briefly run it with no belts to see if the noise changes or has gone away, this will help show if an ancillary is the issue
Examine carefully the condition of the bottom / crank pulley as these are composite construction with a rubber element and its possible this rubber is breaking up and allowing the two metal portions to move.
Another possibility is the variable valve timing system as this does rely on oil pressure to correctly synchronise your vvti pulley on the end of the camshaft. If the valve has become blocked up then its possible the pulley is rattling about and producing the noise.
If all of this is OK then its possible the issue is with the dual mass flywheel on the other end of the engine, these are constructed in the same way I mentioned for the front pulley but are more prone to problems due to greater heat and larger forces.
If your gearstick is rattling this is another pointer to this being the problem.