A bothered person talks about some annoyance. Some bothered ones agree, other bothered ones disagree. At the party, those who aren't bothered are divided into two groups of people who are bothered: those who aren't bothered by that annoyance, but by the fact that there are so many bothered people bothering them; and those who aren't bothered by absolutely nothing, with the exception of those who aren't bothered by being bothered by so many bothers, which, it's known, causes them a deep annoyance.
In the end, the only ones who decided not to go to the party weren't bothered – which certainly bothered a lot of people. I mean, that's how it seemed… at least until they realized they weren't even invited.