Well I guess it depends on the open mics you go to. Around here there are two kinds - places that welcome originals and are quieter 'listening room' type places - some of these are actually coffee shops, open mics in church halls. Then there are open mics at bars/restaurants. These tend to be noisier and where cover songs - songs people know, not obscure B sides - are the best to get people's attentions.
Now either of these types of places can have a 'varied' audience. There may be mostly young people or retired guys. 'Jams' with people getting together just there to play a few songs together may be welcome - or not.
Now if you are playing songs and no one is listening - any of the 'wrong songs for the wrong audience' situations may apply. Or you suck!
With any experience under your belt, you should be able to 'read' the audience during your first song and get a good idea what will work and what won't.