I hear ya, for awhile there Cannabis was tripping me out really badly, i had to quit that a couple years ago, finally able to smoke it again without it getting trippy although now i'm much more aware/conscious of my use of it.
As for herbs and supplements...
Lemon Balm is a good anti-anxiety and relaxative herb that raises GABA levels and counteracts Harmaline's GABA-A inverse agonism (thus cleaning up the bodyload and headspace). There's quite a few different herbs i've tried in combination with Harmalas and many different herbs or supplements will change/alter the effects of Harmalas in various ways. I can't really remember what all i've tried, but Lemon Balm was definitely the main one that stuck out as being very useful, but Rosemary for example has similar properties to Lemon Balm but also has it's own medicinal benefits as well. As far as different herbs go, my suggestion would be to just find some common medicinal plants, and try em' out with the Harmalas to see how they interact and what medicinal benefits the mixtures have to offer.
I've also used terpenes like Limonene, Beta-Pinene, and Alpha-Pinene, as far as terpenes go Limonene and Beta-Pinene are two of my top favs, they seem to act as Adenosine A2A agonists which have quite a few medicinal benefits by way of indirect interaction with other receptor systems like reducing Dopamine 2 receptor activity which can be used to quell "hallucinations" like with anti-psychotics except that anti-psychotics block off/antagonize D2 while Adenosine A2A agonists act as negative allosteric modulators at D2, also has similar properties at some other receptors like CB1 and Metabotropic Glutamate 5, and while i like both Limonene and Beta-Pinene, i think i prefer Beta-Pinene more these days, however while i'm pretty sure Beta-Pinene has Adenosine A2A agonism like Limonene does, and iirc i've seen at least one study that said it did, Limonene has been the prime compound for Adenosine A2A agonism and is backed up by multiple studies, Limonene (and potentially Beta-Pinene) also can help with nausea/vomiting (even with Ayahuasca/Harmalas) and can help reduce acid reflux/heartburn and help to heal GERD, at least for me it did, i rarely ever have heartburn or what not anymore.
I've also used certain essential oils like Lemon EO or Ginger EO, i find Ginger EO to be helpful for anti-inflammatory action especially for my skin because i'm prone to like acne break outs or what not, and it also helps with my gut and gives me an anti-depressant effect, Lemon EO on the other hand contains Limonene and Beta-Pinene as well as some other things, but personally i find to be a bit trippy compared to pure/isolated terpenes like Limonene, Beta-Pinene, Alpha-Pinene, etc. Speaking of Alpha-Pinene, it is an Acetylcholinesterase inhibitor and raises levels of Acetylcholine which can be useful for memory and alertness, but Alpha-Pinene can be a little trippy and i think that's one of the things that contributes to Lemon EO's trippiness.
I've also used Mucuna/L-Dopa extract (on it's own, as well as a couple hours before Harmalas), and it increases Dopamine and Noradrenaline and helps to replenish the body with those neurotransmitters, however because it increases Dopamine, Dopamine can activate the D2 receptor and thus Mucuna may end up causing some "hallucinations" to some degree, i mean i've never pushed it, i've only taken 300mgs of the L-Dopa content twice a day for a total of 600mgs, and while i've gotten some mild visual activity from it, it's never made me exactly hallucinate. But, i have noticed that as i mentioned about, with Limonene and Beta-Pinene being Adenosine A2A agonists and A2A agonism reducing activity at the D2 receptor, i've successfully used Limonene, as well as Beta-Pinene, to do that very thing and i can tell it definitely seems to counteract the properties of Dopamine (and Mucuna/L-Dopa) at the D2 receptor, so that can help to quell any hallucinatory activity that may come from Mucuna, imo/ime.
I've also used Caffeine, as well as Theobromine, in conjunction with Harmalas, although gotta be careful and make sure you take a lower dose than usual, at least by half, because Harmalas inhibit CYP1A2 which metabolizes Caffeine/Theobromine and can thus potentiate it and make it more bioavailable, but Caffeine by acting as a non-selective Adenosine antagonist including the A2A receptor (the opposite of Limonene and Beta-Pinene) can help make things a bit more stimulating and clearheaded/alert, and cleans up the headspace and bodyload as well to a certain degree. Theobromine is more of a Phosphodiesterase inhibitor, moreso than Caffeine, and as such produces a good bit of vasodilation and can be a bit heart opening ime, while having weaker Adenosine antagonism than Caffeine. Often times i'll just have a cup of Coffee earlier in the day and take Harmalas at night, or take some Theobromine a few hours before the Harmalas, back in my early days though i'd drink Caffeinated sodas (so pure Caffeine, compared to Coffee or Tea) while on Harmalas and that's how i learned about it's potentiation before i knew about the CYP1A2 inhibition of Harmalas, and so every now and then i'll consume some Caffeine with my Harmalas, but often times i only have some Caffeine after i wake up, though sometimes so of it will still be in my system by the time i consume Harmalas but it does just fine.
Then there's probiotics, those have been a big help for me personally especially with my gut and how i feel overall, plus Harmalas have some anti-microbial properties and as such can kill off certain organisms in the gut (whether good or bad, but i haven't noticed any real negatives in that department from Harmalas) and so replenishing potentially lost good bacteria, as well as helping to maintain good bacteria in the gut, can come in handy with or without dosing Harmalas.
And i've noticed certain vitamins and minerals can be useful, in general and with Harmalas. Like Magnesium for example, plus ime Harmalas seem to use up or deplete Magnesium (though not sure about anything else) and so supplementing with Magnesium if using Harmalas as medicine is a good idea imo. And making sure you're getting enough Vitamin D, Vitamin C, B's, A and so on and so forth.
There's also various Cannabinoids to look into, things like CBD, CBN, CBG, even psychoactive one's like D8-THC and HHC which are milder, not as trippy, and less anxiety-producing than D9. Personally i find Cannabinoids to go very well with Harmalas, even D9/Cannabis which people here seem to frown upon, even though i myself know of the downsides of Cannabis, having gone through them myself, Harmalas just don't feel "right" to me without some CB1 agonism going on, they go hand in hand, but Cannabis especially can be insightful when combined with Harmalas, but Cannabis definitely shouldn't be overused, but other Cannabinoids can also be quite helpful.
As far as something more traditional goes, i find Mapacho to be useful too, although i'm a Tobacco smoker and find Tobacco to be helpful in general, and while i wouldn't necessarily recommend smoking lol, it can have it's benefits, but in the case of Mapacho, specifically the snuff, i've heard good things, i've only smoked Mapacho and haven't yet used it in the form of snuff, but i have had some really old Tobacco snuff on a few occasions that like was my grandparents in an old antique bottle that my mom had put up in a cabinet just because it was an antique bottle, so i took the snuff and left her the bottle lol, and it was definitely still effective but boy does snuff burn, i'd like to get me some Mapacho snuff at some point though. But yeah, Tobacco definitely has a synergy with Harmalas, Nicotine as a pure compound not so much although it can be potentiated a bit by the Acetylcholinesterase inhibition of the Harmalas, but nothing like the potentiation that occurs with Tobacco. I've noticed that both Tobacco, as well as Cannabis, seem to have this effect where they will bring out different aspects of Harmalas, and the Harmalas will bring out different aspects of them, and in the case of Tobacco it has it's place imo, but it can increase the nausea a bit and maybe contribute to purging, but i find Tobacco to be v ery useful for grounding and for clearing the mind.
That's just the basics of what i can remember at the moment, but i've tried a lot of different things and combinations of things, and imo there's a wide range of different herbs and supplements that can be safely mixed with Harmalas for various purposes, and it's a very interesting thing to explore and experiment around with imo.