For one, the Pandaren are immediately hesitant and weary of both the Horde and Alliance. Hell, Taran Zhu, leader of their army (In a simplified nutshell) expresses open hostility toward us.
Second, Lei Shen wasn't put into place by Pandaren society.