Please ignore his threats - they have no basis in reality.
The law says that a child is entitled to have time with BOTH parents and when parents separate they are expected to agree between them what time the child should spend with each parent.
This can be done using Family Mediation if necessary.
If no agreement can be reached and mediation has failed then a Court application can be made.
The court will not look at material things that the parent has to offer - they will consider the needs of the child - and the principal needs are for stability and security.
Accordingly if you have been the main carer until now then that is what is most likely to continue, with the child also spending time with his father each week
The fact that he is continuing to be so verbally aggressive could mean that you can get na order excluding your ex from the home whilst negotiations are ongoing - so I suggest you
inform your ex that if he continues then you will report him for the new offence of emotional control and abuse.
Please ask if you need further details