Well look at it this way. You aren't WITH them because you didn't vote for them. Then, in a sense, you are AGAINST them because you haven't contributed to their efforts to become elected and could in the future actively choose to not be with them rather than just passively so. All subjects can be looked at in such a way.