If MTF athletes don't have an advantage over cis-female athletes just from the fact that they used to be biologically male presumably you'd see FTM athletes that have been given hormone therapy and legally become male competing at a high level of mens' sport too? Fact is pretty much every trans athlete worthy of note is either MTF and competed in womens' sports after transitioning, or FTM, and competed in womens' sports before transitioning. The only example of a FTM athlete competing with men that I can find is Schuyler Bailar, who was one of the best swimmers in the country in his age group when he was competing against girls, swimming alongside swimming legend Katie Ledecky, and is now an above average college level swimmer in the mens' category. That seems to be the bottom line. Even a potential Olympic gold medallist won't be better than middle of the pack after transitioning to male. Biology is just simply a hell of a drug.
It's a difficult issue because it's going to be unfair on someone no matter how it's handled, and there can't really be any shades of grey based on hormone levels or something like that. It's undeniable that, all other things equal, MTF athletes do have a significant advantage that no amount of hormone therapy will ever take away from them, and even the best cis-female athletes can't do anything legal to level the playing field in that respect. Banning trans athletes from competing in women's events is the least bad option of the two, simply because the other option unfairly affects a much larger group of people.