I've never heard anyone say, "You didn't vote, so I won't listen to your complaints", let alone actually not listen.

I've heard it. I've never said it -- and never will -- but my response will usually begin with "So what have you done or are you doing about it?" In at least one case, the responder pointed out the massive amount of nonvoting work he does in the community, intended to bring about change, and I acknowledged that what he does is worth far more than his vote. But for many folks, not voting and not doing anything about their complaints is the norm; that combination strikes me as wanting their cake and eating it too. And the complains that follow seem hollow, by comparison.