When you deny people to talk on your college campus because the speaker disagrees with you--that is threatening free expression.
Well, when you are paying someone to speak, it is up to who is paying. Otherwise, any crackpot can claim something outside of settled science, for example, and demand to be paid to talk about it. Would you consider it a violation of free expression to not pay dropdeadfred to speak?
Anyone can speak for free at any commons area at a university all they want. But they have no right to be paid to speak, if someone wants to select them for paid speech, that is up to the hidden hand of the free market that determines who. I have noticed that when someone gets their temporary employment rescinded, they never waive the fee and speak anyway.
I imagine you are totally outraged that liberal people are almost never selected to speak at Oral Roberts, Liberty University, or any of the bible colleges as well.