In many major religions today, adultery is considered a sin. Epictetus appears to have the same view and he states his reasons.
"Human beings are born for fidelity, and that anyone who damages it is damaging the distinctive quality of man" (v. 1, p. 77). In other words, humans' nature is to be rational, and one way to demonstrate that, is to be faithful to the spouse you have committed yourself to. Anyone who attempts to destroy that fidelity is not living according to their (rational) nature.
Adultery ruins, destroys integrity and piety. It destroys the good feelings between friends and neighbors. It destroys trust.
An adulterer exhibits bad character and is "useless" (v. 6, p. 77). Epictetus compares them to wasps: people run away from them and kill them if possible; since all they do is sting!
Simply put, adultery is immoral. For Stoics, where virtue is the sole good, adultery is not virtuous, and therefore not Stoic.