I remove the sludge when I make teas with mimosa. After simmering, I leave it in the fridge overnight, after which it all settles to the bottom and sticks to the container. I still run it through a cottonball to be sure.
I doubt there's any loss in potency, but in my opinion it's worth not having to drink stomach shrapnel even if there is. My tea seems plenty potent anyway.
I don't want comfort. I want God, I want poetry, I want real danger, I want freedom, I want goodness. I want a clever signature.