This is a simple, short letter, but one that needs a bit of unpacking.
This first part of the letter, in which Seneca states, "avoid the many, avoid the few, avoid even the individual," speaks of balance. There is a time when the individual should avoid the many - the crowds. There is a time for when the individual should avoid the few. And there is a time for when the individual should avoid solitude.
The concept that Seneca is trying to share here, is one of caution and learning. If someone is uneducated, then spending time by himself will not help him improve. Therefore, it would be prudent for him to find a mentor or teacher. If someone is newly educated, but needs practice, it would be prudent for him to avoid the uneducated crowds, and perhaps spend more time with peers who are also learning. I think this is what Seneca is trying to convey.
He recommends to Lucilius, that he should be OK spending time with himself. Upon hearing Lucilius speak, Seneca thought to himself,
These words did not come from the edge of the lips; these utterances have a solid foundation. This man is not one of the many; he has regard for his real welfare.Seneca seems to judge Lucilius as well-grounded and well educated, and therefore, Lucilius can trust himself to be with himself as he introspects.
The latter part of the letter delves into prayers to the gods. If you are a praying sort of person, then the evaluation of your prayers ought to be conducted. Are you praying to receive indifferents? Or are you praying for something that the gods have already granted you? Seneca offers this advice, "pray for a sound mind and for good health, first of soul and then of body."
His closing quote also deals with desires and prayer.
Know that thou art freed from all desires when thou hast reached such a point that thou prayest to God for nothing except what thou canst pray for openly.This passage reminds me of the quote from Crates of Thebes - practice being in need of only a few things.