This is a great insight into the world of refereeing. It's not simply about recognising and penalising fouls, it's man management and team-work.
In every refereeing seminar I've been too, we talk about selling decisions. If you're unsure of a decision but give it with full conviction, the players 9 times out of 10 won't question it. It's impossible to be 100% sure on every scenario in every match so you work as a team with your assistants and between you, you should get the vast majority of decisions correct.
How many times do you see a team holding the ball up in the corner seconds before the end of the game? As part of game management, the referee and assistant will give a free kick to the defending team if there is even the slightest foul by the attacker. This keeps the game flowing and there's less likelihood of a mass confrontation breaking out.
Throw ins that you're not sure of will almost always be given to the defending team.
I've heard people say that the Premier League assistants don't really do anything. I know one assistant personally and he has told me numerous times that he will be constantly speaking to the referee through the headsets and will give a foul without waving the flag. He has said that this is bad practice as it doesn't 'sell' the decision