This is a common tactic.
It does not cost £2000.00 to clean a carpet and paint the walls let alone £4000.00 . He ius simply trying to keep all the deposit by frightening you into thinking he can get a further £2000.00 out of you if you press for the return of some of your deposit.
To sue you he will have to justify the amount claimed with receipts.
If the carpet was not new when you moved in he will not be entitled to all the cost of a new carpet. The walls sound like fair wear and tear so you should not be liable for all the walls just repair and touching up of any specific holes.
I would paint it in the same colour it is now and get the carpet cleaned by a specialist carpet cleaner so you can show it was done properly.
If he fails to return any balance of deposit due you can sue him for the balance due back to you.