Yeah this is true and I thought of that as well after I wrote it. All it takes is a few weak folks in the center to start believing there is some truth to it all and they ll start taking a bunch more in the center with them.
Still - I do believe a lot more people how have to the time to pay attention to how looney some of these folks are. There are some things that just don't sit well with the vast majority and that's things like bringing kids to rallies like that.. Guns to try and rationalize a point by clearly using intimidation (even many gun owners would not agree with that), and of course showing political symbols like Trump gear and Nazi flags doesn't help support the idea you are there for the people as a whole. I remember we used to all be Americans - with yes still vast differences but Presidents used to generally (at least pretend to) care about all whether red or blue or something else. The "us" vs them mentality is what defined America. Now it's us vs them within our own borders and everyone surrounding this President is at fault for that.
I don't think it will play out like that, this isn't just some random "xxx is communism" dynamic. A lot of people get that these measures are necessary, but I don't think anyone actually enjoys them and given the US's relative lack of a safety net, my guess is that lots of people will be quite desperate to get back to work, even if they understand the threat of the virus. These "protests" have gotten (inter-)national media coverage, they have become the central anti shutdown movement and more and more people will show up there once they have been sufficiently primed by Trump, Fox and financial pressures that they feel the need to take action.
For example in Berlin the police shut down a disorderly protest that had been a weekly occurrence for a month now and united a growing number of people from far left to far right (iirc around 250 this time) and not all of them drooling mouth breathers. Some protesting for "no more dictatorship, fascism and war", some having more moderate objections to the way emergency measures affect basic rights, some of the "it's just four thousand dead people" variation, the obligatory "vaccination terrorists" banner and I also saw picture of a guy with a maga hat.