Did you ever apply for permanent residence during your time in the UK? If so, when was it granted?
I am afraid from the limited I formation that you have provided that it would seem that the passport agency us correct.
If either parent has PR then a minor child can obtain a passport following registration of the child.
However, if you are applying on the basis of the WRS then it seems that this would be correct.
I hope this answers your question, if so, kindly rate the answer and provide feedback.
If you have any further questions, please do not hesitate to ask.