Hi, thank you for the question, and I shall try and assist you as much as I can.
You are right, as your partner is from Mauritius, he can travel to the UK visa-free, However, he must have the intention to return to his country within six months of entry.
There is a strong possibility that he may get refused entry by an immigration officer on the basis that he has previously overstayed and that he has no intention to return to his country. Also, that as his family permit has been refused it may show that he has no intention to return as he wants to be with you in the UK.
If somehow, your partner passes through immigration, then he can apply for a residence card in the UK as long as he meets the criteria under the EEA regulations.
Or, you can re-apply for a family permit from Mauritius and ensure that you provide as much evidence as possible to support the application.
Thank you for your question, and I would be grateful if you would kindly rate my answer.