Oh you are SO right about there being red flags - add bells and whistles going off too! First, repeatedly drunk is not ok. It would lead one to believe, probably rightly so, that he has a drinking problem. Many, if not most people without drinking problems can and do drink socially without having so many that they get drunk. In fact, in the 'best' social circles, becoming drunk is something considered "low class" and "unacceptable". So by all means expect any man in your life to live up to that higher level. You deserve nothing but the best, right?
Next - even uttering that he "might be" a sex addict hits the bells and whistles. It's like someone saying, "I MIGHT be an alcoholic" - if a person suspects something about themselves - chances are it's true. In my counseling practice if anyone asked, "Am I an alcoholic?" - I had to tell them that just by asking that question it would appear that they are. Obviously there were exceptions, but you get the idea.
On a subconscious level you likely made that statement about not dating anymore because you "Knew" it was best. His not only being agreeable, but rudely ignoring you thereafter tells me you made the right decision
Agonizing over "what if's" is a waste of time. For every minute you spend looking over your shoulder at yesterdays - you loose a minute of your today's - and quite possibly opportunities right in front of you in the way of meeting someone new and more appropriate.
Let that past go - focus on right now and anticipate the promise of tomorrow
Be healthy - be happy - be in the moment and think more highly of yourself in order to get 'better' going forward. This makes much more sense than ruminating, right?