Asking the client for a reference is not in itself illegal. Maybe immoral but that does not give you any legal grounds on which to challenge it. Legally there is nothing stopping them from doing this, unless there was a specific contractual restriction, which there does not appear to be.
Your only way of challenging them would be by relying on the restriction preventing them from working for a competitor. I must say you have not done yourself any favours by imposing a 2 year period for it because that would very likely make it unreasonable and thus unenforceable. 3 to 6 months is what the courts allow, maybe 12 at a push.
Also you cannot just prevent someone from going to a competitor, they must be damaging your business in some way affecting legitimate business information, such as using very close connections with clients they have made whilst with you, confidential information, trade secrets, etc.
However, even if these can be shown to apply, the fact that it was for 2 years can make it rather difficult to enforce.
Anyhow, your original question’s answer is that they can legally approach a client for a reference if they wanted to as there is nothing stopping them contractually from doing so.
I'm sorry if this is not necessarily the answer you were hoping for, however I do have a duty to be honest and explain the actual legal position. This does mean delivering bad news from time to time. I hope you understand and would be happy to provide any further clarification if needed. If you are still satisfied with the level of service I have provided regardless of the contents of the answer, I would be grateful if you could please take a second to leave a positive rating by selecting 3, 4 or 5 stars above. Thank you