you need to:
A. go back and reread those books.
B. learn basic ins and outs of TIME TRAVEL.
we are taking the dragon soul using it, then nozradormu will return to about 5min after we took it from way back when.
in other words no one will notice it was gone, cause it wasn't gone.