I would only counter that the difference in griefing and other in-game activities you can choose to do is that griefing intentionally makes another player's game time worse. Granted the players who are flagged for PVP don't have any excuses to complain, though it is quite funny to hear them whine, complain, and get physically upset, when they get ganked, but when you are in group with them, they like nothing more than to kill/gank/grief any opposite faction players they can.
Reminds me of my 5-year old nephew playing console games against his cousins. If he lost he would cry and yell and flop around on the floor, but if he ever wins a game he's in their face pointing in their face laughing at them. I just figure griefers have the same mentality as my 5 year old nephew, so I can understand why some folks assume they have some mental issues given that they are much older and presumably more mature than a 5 year old.