Well, even ignoring the numerous problems with the whole Jesus story (which we really should not get too far into for reasons)... there are numerous things in Passion that Gibson just either gets totally wrong or makes up entirely. There's this whole thing where Jesus invents dinner tables for example. We have definitive proof that dinner tables existed before that point in history. Then there are the tools with which Jesus was tortured with which don't seem to have existed at the time.
So, really the Passion has problems both staying true to the Jesus Legend and historical accuracy.
But it's really great torture porn if that's what gets you going.