Hello I am a solicitor with over 15 years experience. I will try to help you with this.
Generally your lease or other agreement with the freeholder will set out the details of when ground rent is due and what the penalties are for late payment etc. If you have recently purchased this property then your solicitor should have told you all about this in the report on property. You should look at your documents. As a general principle of contract law however if you were late in paying the ground rent and therefore in breach of contract then your landlord is entitled to be compensated for losses he has actually and reasonable incurred. So if you were late he could certainly instruct a solicitor to chase you for the money. I fear that as long as he has not been unreasonably rushed to instruct particularly expensive solicitors you may be liable to pay the fees.
You should check the documents referred to above to start with. I hope this answer is helpful but I expect you will have follow up questions so please feel free to ask.