This is a particularly sad mistranslation. Jesus is addressing in individual here about a specific situation and the specific situation is about a "brother," which is either a relative or a friend. If we assume that we only get Jesus's half of the discussion, we have to fill in the blanks logically, according to Jesus's practice, not the practices of Christians when the Bible was first translated to English. Note, four verses from now, Mattthew 18:21, Peter takes this to mean we must forgive our brothers. To see how much has been done to misconstrue the Greek of this verse, look at the word-by-word analysis of its translations below.

First, Jesus doesn't say "if," but "when." This is an expected situation, not an uncommon one. The word translated as "neglect to hear/refuse to listen" is only used by Jesus here, but the Greek word means "overhear,"  "hear incorrectly, or, in this context, "misunderstand." It is translated into stronger language to make the brother's crime greater. The word translated as "church," did not mean that it Jesus's time. It meant "assembly" or as we more commonly say today, a "meeting." However, it is introduced by a definite article, so "the meeting," likely a regular community meeting. 

So if someone misunderstands the argument of those who come to him and of the community meeting, what is to be done? The verb translated as "let him be" is a third-person command form of "to be," which has the sense of "he must be." However, this verb begins the sentence, it could easily be phrased as a question, "Must he be?" Without hearing the words, it is impossible to know.

However, the words "heathen man/pagan, (better translated as "foreigner" and "publican/tax-collector" are uncommon for Jesus. Though he uses these terms disparagingly, in a light-hearted way, he never shunned such people in his life, healing foreigners and dining with tax collectors. The words are joined by an "and" not an "or," and both introduced with articles,  so they are combined in the sense of a "this foreigner, that tax-collector," making the description worse. So Jesus asks the very logical question, "Must a brother be like that foreigner tax collector if he disagrees with you?" He assumes that the obvious answer is "no." We can see this clearly in all the following verses.