Hello I am a solicitor with over 15 years experience. I will try to help you with this.
Can you tell me a bit more please. What is the amount borrowed and how much have you paid back? What is the extent of of your Grandad's disability? Did you discuss borrowing the money with anyone else eg family members?
So you didn't borrow the money yourself?
No you are not being charged for the extra questions in this chat.
If your Grandad has the capacity to make decisions like this and made it and half the money is back in his account anyway then I don't see what it is anyone could say you have done wrong.
Have social services been in touch with you about the money?
OK so the suggestion is that you have been taking things from his flat and from his bank account. You suspect the local authority carers have been doing this. You may as well let them see the statements but send them a letter with the statements explaining precisely what happened and your concerns about the carers. What is your grandfather's view on the carers?
I quite understand why you feel saddened and bullied in this situation but if you send a polite but firm letter to Social Services above this should help.