Aaahhh.. so OP is upset that people they don't think are worthy get upvotes, while they themselves get downvoted by a couple of known people for writing about how they hate elves.
Though as the OP themself points out: even if downvotes were removed the people who don't like your fanfiction will continue to post critique of it whether they like it or not. Unless they also feel that only positive comments should be allowed too?