I think you would, because the plummeting in population/players might put the game at the risk of dying, and that would be bad for everyone, that is why i think it is so important.
Would that not be the case, i would agree with you, the dissapearance of a part of the game you do not take part in, would be inconsequential for you.