Plumbing Questions? Ask a Plumber for Answers ASAP
Hi Liz, if its not leaking when off it suggests its a faulty valve/gasket, as when on and heating water the pressure increases and thats probably allowing water to seep by.
Your best off getting a heating engineer or plumber unless your happy to strip it down yourself and locate where the water appears to be coming from, however based on what your suggesting this doesnt appear to be an expensive fix.
If you would like me to recommend a local engineer please provide your postcode.