Someone despises me? That is his concern. But I will see to it that I am not found guilty of any word or action deserving contempt. Will he hate me? That is his concern. But I will be kind and well-intentioned to all, and ready to show this very person what he is failing to see - not in any criticism or display of tolerance, but with genuine good will, like the famous Phocion (if, that is, he was not speaking ironically). This should be the quality of our inner thoughts, which are open to the gods' eyes: they should see a man not disposed to any complaint and free of self-pity. And what harm can you suffer, if you yourself at this present moment are acting in kind with your own nature and accepting what suits the present purpose of universal nature - a man at full stretch for the achievement, this way or that, of the common good?
Chapter 12 of Book 11 sounds very similar to the sphere of Empedocles, where there is a perfect balance of form - neither collapsing or expanding. Much like many orbs or spheres we see today (planets, suns), they retain their form and are constant (relatively speaking). So too, the soul should remain constant and intent on truth and light.
In the following chapter, he outlines actions that fall in the the category of "things under my control." We do not have any control over what other people think of us. For our part, we do have control over whether we will be kind and well-intentioned to everyone. And to dig even deeper, if our thoughts were to be exposed for all to see, what would they see? Would you be inwardly complaining? Would you have self-pity? To tie the concepts of chapter 11 and 12 together, both your thoughts and actions should be in alignment - which would create balance and harmony between your inner thoughts and outward actions.
(see also Citadel p. 129, 226, 267, 272, 301)