There is nothing wrong with the refs. The problem is the fans. Most people don't know the laws of the game, including the players and pundits. So they start moaning about a call that the ref was right about and everyone assumes the ref is wrong. Sure, sometimes they miss something, but most of the time their calls are correct. It all comes down to interpretation of the laws. The most confusing one seems to be handling the ball. Everyone is too caught up on the word "deliberate" or "intentional". The thing is, FIFA's definition of those words are different from the dictionary. For FIFA, deliberate basically means your hand was moving towards the ball, and the ball came from far enough away that you had time to move it.
Then there is offside. We are lucky to see so many slow motion replays from all different angles. The AR has one attempt at it. Sometimes they get it wrong. Personally, I would rather see them side with the offense. I would rather see a goal given that was marginally offside, then see a goal taken away that was legit. I also think they should change the law. For me, there should be clear day light between the defender and the attacker to be offside. None of this "his nose was offside" crap. I say give it to the offense and let's see more scoring...