I think the void lords saw that they were unable to manifest themselves in reality through 'infesting' a grown Titan directly. I don't think that part said it's impossible for them to drive him insane by their scheming. Which, at least in my opinion, they did quite well.