It should be fine. You really only have to worry about plastics when dealing with organic solvents (i.e. naphtha, xylene, acetone, etc). The coffee filter is mostly cellulose which is harmless if you ingested it. In my experience strong solutions of NaOH will start to soften coffee filters to the point where they tear more easily, but this can be avoided in rue extractions by using sodium carbonate instead.