The problem, as every sentient being knows, is not the majority of Moslems. It is the relatively large proportion of the minority of Moslems that countenances violence, and violence of a cowardly sort - stabbing women in the back etc. And that minority is toxic to non-Moslems, and to other, peaceful Moslems, of course.
When people say "there's a problem with Islam" they are not talking about the peaceful majority. They are talking about the violent ones, and the unbiddable, fundamentalist culture within which they move. And they do, of course, wonder why Islam, as a whole, seems to produce such a minority, and they wonder how the peaceful models of that religion can overwhelm the violent ones.
Folks see what they see, even if not many are willing to say it.