Well if someone says you cant have a laugh with black friends which I regularly do as they are comfortable with it, its not racism. Racism is defined by the context. Just a quick google search and I got this -
Prejudice, discrimination, or antagonism directed against a person or people on the basis of their membership of a particular racial or ethnic group, typically one that is a minority or marginalized.
The belief that different races possess distinct characteristics, abilities, or qualities, especially so as to distinguish them as inferior or superior to one another.
Basically, words that Cavani used cant be described as hateful or prejudiced, so therefore not at all racist. That's the be all and end of it.
You can absolutely say inappropriate things to black friends you know won't take offence to it, if your friendship works that way.
If you say those inappropriate things on social media though, you are liable to face backlash and (depending on your job) direct punishment. Because at that point you aren't just saying it to your black friend, you're sharing it with the world. Which is what Cavani did.
Hell, even if you just say those things in a room containing black people you don't know, you may well get a negative reaction. For all the talk about context in terms of intent, context in terms of audience matters at least as much.
The idea that something you do/say can't be deemed racist if you didn't intend for it to be hateful is so at odds with the way the world works that it's remarkable for it to still be used as excuse in 2020, after
all the discussion on race we've had this year. If you like you can go into the Blackface thread and argue that it isn't racist if people aren't doing it out of spite, see how that goes.
The reality is that Cavani publicly referred to someone's race in a way that almost certainly contravenes the guidelines he is required to follow as a premier league footballer. Arguments about culture and context might add nuance to the discussion but they don't change the fact that he has done something that will likely be deemed inappropriate and (rightly) result in punishment.