What Would It Mean to Not Have Cancel Culture?

I saw an article recently that made the argument that free speech warriors who complain about cancel culture are hypocrites because really, they just want to restrict the speech of others. After all, what would a world without cancel culture even look like, asked the author? It would basically be a world in which people are not allowed to criticize celebrities and public figures! And if they’re not allowed to criticize others, it means someone would have to be doing the disallowing. Therefore people who oppose cancel culture are frauds, who don’t care about the right to free speech, just about their own ‘right’ to say bad things without any consequences, and to have that be backed up by a legal regime that punishes their detractors.