I mean ... it doesn't change anything at all in the game, and the lore behind thoses gems is poor, instead of calling theses gems crystallized dread/horror/terror, couldn't they give a more legendary name ?
Like for example :
Azureus, sapphire of the abyssal depths
Pyron, ruby of the infernal flames
Also, thoses gems are just an overpowered version of "normal" gems, they do not bring anything except stats while the legendary weapons always used to bring special effects... that's not cool... thoses items are only legendary cause of their quality...