I just watched again. Right before Pogba gets to him you can see Jose's right arm raised before Carrick pans in front. Pogba shakes but Jose does some Ninja move and brings his arm down quickly.
Your wrong he does shake his hand, look at pogba's hand movement it makes two breaks in motion, before Pogba turns around, because of something Jose says, Pogba responds with watch who your talking to, they move towards the crowd, Jose say's something and Pogba say i haven't done nothing, Pereira then nods his head to acknowledge Pogba, at the same time Rashford walks between Pogba and Mourinho, shakes someone's hand, doesn't shake Pereira hand, shakes the person next to Pereira hand, Pereira moves to the right, and there's a feeble handshake between him and Rashford, then Pereira moves away from the crowd.
More telling is the facial expression of the number 54 and the black guy in front of Pogba, wiitch is motionless, so what was said at the stage of the conversation was not anything damning.
Another thing to watch for is after Mourinho walks away, after the handshake, Pogba and the guy in the bomber jacket, stare at each other.