Hello and welcome. My name is ***** ***** I'm a biologist. I'm sorry to hear you are having a problem. Unfortunately, there are over 500 species of lice in the world, plus some plant feeding insects that are called lice (and look like them), so it will be nearly impossible to tell you exactly what these insects are. Most are specific to one host - only birds, only humans, etc. However, since they aren't on any warm-blooded animals, I can tell you a way to get rid of them.
Start with a good soaking spray from a hose. Spray the plants in the window boxes, the window, and the surrounding area. When the area is dry, insecticidal soap (available at garden centers and nurseries) applied liberally should kill the insects. I usually treat twice about a week apart. A few days after the second treatment, give the plants another shower to wash away dead pests. Check repeatedly for even a few of these insects, and if you see them, repeat the treatment. Immature forms can live in the soil, and so you may see them popping up again every so often.
If you have more questions, let me know I wish you success in getting rid of the insects.
My goal is to provide you with excellent service – if you feel you have gotten anything less, please reply back, I am happy to address follow-up questions. Please remember to rate my service after you have all the information you need. I will greatly appreciate a positive rating as that is the only way I am compensated. Thank you!