Oh, there is. Relatively poor record in UCL*, and not a good record vs big European teams.
2 titles, 4 finals, and 7 semi-finals or so in 19 seasons is pretty decent, but not necessarily great. Zidane has already more titles (as has Ancelotti), Guardiola has more semi-finals and so on. We also had our fair share of getting eliminated in group stages, and except in 1999 final (when we got totally outplayed), I don't remember us eliminating a team that was supposed to be better than ours.
Essentially, if I had a match for my life against a stronger team, Fergie wouldn't make my list of managers. If I have a season-long league against as good and potentially stronger teams, then yes, he would be. And if I want to win the league as many times as possible for the next decade, then the list would essentially have only one name.
Don't get me wrong, he was good at it. Just not as good as some others, and not as good as on his other skills.