Below are some of Augustine's comments on how widely Matthew's authorship of the first gospel was accepted during the earliest generations of church history and the lack of any competing authorship claim. Much more could be said in support of Matthew's authorship, but Augustine makes some good points as far as he goes:
"you [Faustus, a Manichaean] immediately deny that Matthew wrote the narrative, though this is affirmed by the continuous testimony of the whole Church, from the days of apostolic presidency to the bishops of our own time. What authority will you quote against this? Perhaps some book of Manichaeus, where it is denied that Jesus was born of a virgin. As, then, I believe your book to be the production of Manichaeus, since it has been kept and handed down among the disciples of Manichaeus, from the time when he lived to the present time, by a regular succession of your presidents, so I ask you to believe the book which I quote to have been written by Matthew, since it has been handed down from the days of Matthew in the Church, without any break in the connection between that time and the present. The question then is, whether we are to believe the statements of an apostle who was in the company of Christ while He was on earth, or of a man away in Persia [Manichaeus], born long after Christ….the question in this case is, whether we are to yield our belief to a book acknowledged and approved as handed down from the beginning in the Church founded by Christ Himself, and maintained through the apostles and their successors in an unbroken connection all over the world to the present day…We can now answer the question, how we know that these books were written by the apostles. In a word, we know this in the same way that you know that the books whose authority you are so deluded as to prefer were written by Manichaeus. For, suppose some one should raise a question on this point, and should contend, in arguing with you, that the books which you attribute to Manichaeus are not of his authorship; your only reply would be, to ridicule the absurdity of thus gratuitously calling in question a matter confirmed by successive testimonies of such wide extent. As, then, it is certain that these books are the production of Manichaeus, and as it is ridiculous in one born so many years after to start objections of his own, and so raise a discussion on the point; with equal certainty may we pronounce it absurd, or rather pitiable, in Manichaeus or his followers to bring such objections against writings originally well authenticated, and carefully handed down from the times of the apostles to our own day through a constant succession of custodians." (Reply To Faustus The Manichaean, 28:2, 32:21)
The early sources who affirm Matthew's authorship of the gospel include non-Christians. It should also be noted that Faustus apparently wasn't consistent on the matter. He seemingly affirmed Matthean authorship at times (2:1, 7:1, 23:1-2), though he questioned it or denied it on other occasions. Augustine interacts with Faustus at length, even to the point of mentioning how tiresome it is (10:2). Yet, he never cites any significant reason Faustus brought forward for questioning the gospel's authorship. Instead, weak objections are cited, like the gospel's references to Matthew in the third person (17:1), even though authors commonly refer to themselves that way. Faustus mentioned no early source who denied Matthean authorship, much less did he produce any evidence comparable to or better than what Augustine cited on the other side.