Why do so many early sources say nothing about Mary's perpetual virginity while suggesting they rejected it?

Tom Schmidt recently published a book arguing for the authenticity and significance of Josephus' material on Christianity. I've written some posts in recent years about the importance of what Josephus tells us about the baptism of John the Baptist. And Schmidt's book reminded me of the significance of Josephus' comments on James' relationship with Jesus as it pertains to the perpetual virginity of Mary.

Think of how many sources in the earliest centuries address issues relevant to Mary's alleged perpetual virginity and not only don't affirm her perpetual virginity, but even use language that's most naturally taken to contradict it: Matthew, Mark, Luke, John, Paul, Josephus, Hegesippus, Irenaeus, Tertullian, and Hippolytus. Those are just the sources in the earliest generations of church history. There was ongoing opposition to Mary's perpetual virginity well after the concept became popular in later centuries. The opposition continued into the medieval era. In the earliest generations, the belief that Mary gave birth to other children after Jesus seems to have been the dominant view. Josephus, like the other earliest sources, knew of other language he could have used and did use such language in other contexts ("relative", "cousin", etc.). Think of the cumulative effect of Josephus and the other sources involved. It's highly unlikely that so many early sources would believe in Mary's perpetual virginity, yet not only not refer to it, but even use language that seems to contradict the concept (multiple terms in multiple contexts and multiple types of contexts).