I've heard before that ergot can poison your nerves causing them to die, like what bakers would get when making rye bread.
Wouldn't morning glory do that if it contains ergot alkaloids?
Or would you still consider them safe?

It's all dose dependant. You could probably O.D. on morning glory, but you'll have to take a lot more for that than you need to have an LSD-like experience.

HBWR is a lot nastier than morning glory´s or ololuiqui.

Sure, any substance in large amounts can be toxic, but there is a slight misunderstanding in the OP's question. There are several different ergot alkaloids, which are present in Claviceps spp. . There are also some alkaloids of the same chemical family that are present in morning glory, ololuiqui and HBWR seeds.

Nevertheless the alkaloids which are more toxic and the main responsibles for St Anthony's fire, or, ergotism, or ergot poisoning, are not present in those aforementioned seeds. Once thought to be one single alkaloid and named Ergotoxine, it has been found to actually be a mixture of three alkaloids: ergocristine, ergocornine and ergocryptine.

So with that said, consuming LSA-containing seeds in any reasonable dosage is not dangerous in terms of toxicity.