Referees are human. They're not immune to personal bias. Even if they dont support one team in particular. Some fans or players bound to get on their good or bad side along the years. Maybe some manager gave them an earful, it's hard not to take some of it personally. They're human afterall.
But i have no doubt they will try their best to be fair since it's actually very hard to be partial in a split second decision where every mistake can be scrutinized.
The FA rightly will defend their ref in public, no excuse even if said ref made a glaring blunder for authority sake. If they start giving pts for referee that'll end their job on the grand scheme of things opening them up to endless scrutiny. You have to back your ref, or else they'll never command enough authority to officiate a match.
I believe most ref error are due to human eyes incapable of seeing small little details that's happening in real time. Heck i can't even see a foul in multi angle replays let alone seeing them while running in full speed while having to watch many different things at once. I also believe that each one of them actually doesnt see the incident all the time but made a professional guess based on experiences and circumstances. Some incidents are physically impossible to see i really dont believe even the best referee can spot them all. If a team counter attack with long ball for example and the attackers is fouled 50 yards away from the ref. Hard to see that.
We have to remember that they're alone on the pitch, real time with 22 players trying hard to "cheat" him with theatrics.