Why dont you filter your tincture, then try to dissolve whatever is left on the filter in more acetic acid?
If it dissolves, its probably harmalas and you just hadnt added enough, and you can put together with the rest of the filtered tincture. If it doesnt dissolve, its probably some impurity and can be discarded.
You can use warm acetic acid to increase solubility, just in case some of the impurities are somehow 'trapping' harmalas inside and preventing them from dissolving.