Hi, thank you for the question, and I shall try and assist you as much as I can.
You are right, as your partner is from Mauritius, he can travel to the UK visa-free, However, he must have the intention to return to his country within six months of entry.
There is a strong possibility that he may get refused entry by an immigration officer on the basis that he has previously overstayed and that he has no intention to return to his country. Also, that as his family permit has been refused it may show that he has no intention to return as he wants to be with you in the UK.
If somehow, your partner passes through immigration, then he can apply for a residence card in the UK as long as he meets the criteria under the EEA regulations.
Or, you can re-apply for a family permit from Mauritius and ensure that you provide as much evidence as possible to support the application.
Thank you for your question, and I would be grateful if you would kindly rate my answer.
As you have both have the intention to live in the UK, this is one option. But be aware that an interview can be conducted by the officer at the airport, and you should be given the opportunity to prove that he is your family member. Please note, to prove he is your partner, you need to show that you are either married or that you have been continuously in a relationship for two years.
Please do not forget to rate my answer.