I'd rather a manga remain obscure with no adaptation any day over it getting a bad anime with more fans. This weird idea I see from people nowadays that a series is only really 'valid' if it has a sizable or notable fan base is so bizarre to me. If you like something, like it! Why do you need other people to give you some sense of validation that it's okay to like something?
I will say though that I am a bit more open to anime adaptations if I know for a fact they'll be done well and serve to enhance or (Very rarely but it does happen sometimes) even surpass the original manga. Take that upcoming Ghost in the Shell anime for example. The studio and team involved in it have a fantastic track record, so I have full confidence it'll be worth watching.
So yeah, I'll always prefer a manga remain obscure over getting a bad anime adaptation but getting more popular, BUT, like many things in life, there are always exceptions.