Yup, it's safe to smoke and quite active. As an added measure of safety, you could evap the xylene, scrape it up once it crystallizes and then dissolve it in a bit of alcohol (iso, ethanol, methanol) and evaporate that.
This will release any traces of xylene that might remain after the initial evap.
And yet another alternative is to salt the xylene with FASA and then freebase it with sodium carbonate which would be the safest bet. More information about that can be found in the FASA forum DMT Extractions.
Convert a melodic element into a rhythmic element...