Leaving out British neighbours, as there's more standard rivalry there, I don't think it's bitterness. ABE clearly, but more to do with apprehension at the bragging and endless fanfare if England won.
I lived there for 12 years, support an English club, loved Rio, Scholes, Beckham, Rooney... Been very proud of Luke and would have loved to see the game won 1-0 with his goal. I still have to make an effort and remind myself of the good and isolate the bad.
It all begins with the press rubbing you the wrong way (ultimately they are a reflection of society, like it or not). The whole coming home business is irritating and that takes you to fans mouthing off and the exposure many will have had to the more drunken and violent ones.
The word is aversion, not bitterness. Some of my best mates are English and most people I know/knew clearly are no more than long-suffering fans whom I would have been delighted for, but it doesn't mean I don't see the other irritating side of an England win.
Take Brazil for instance, I was living there in 2002, just a giant party, no bragging, no we are god's gift to earth, nothing, just joy and everyone is welcome to join. Fantastic fans and that's probably why it was most people's "second team"... until Neymar showed up.