What I don't understand is that he seems to be all for the idea of a welfare state(hence his supposed support for Bernie), but then he also seems to hate the idea of a bigger government and more taxes, which doesn't make any sense.
The problem with Joe Rogan is that he agrees with most of his guests when they are present. It's impossible to know exactly what he thinks, but I think the evidence suggests that he's a libertarian with a soft spot for some leftist ideas.
Republicans have pretended for a while that they're the party for a smaller state. They're not, they want to spend just as much money as the Democrats, they just want to spend it on other things and they don't want to pay for it. So they cut welfare, cut taxes and increases the deficit.
Rogan is closer to a Democrat, probably, regarding what he wants to spend it on. But he really, really doesn't want to pay for it. Supporting Bernie is perfect, because Bernie wants a lot of the things Rogan would want if they were free, and Bernie won't ever win so there's no risk.
When it comes to Trump vs Biden, or an R senator vs a D senator, or a governor or whatever, then things might actually happen and he can't fantasize anymore. So he'll openly cheer for Trump and say Texas won't be Texas anymore if run by a Democrat (while living in Austin, the fecking moran), and people will still be shocked and outraged when he's called right wing.