I think part of it is that raw brain power can appear anywhere in society, but if one does not have the privilege of wealth in the US... the choices are the military GI bill or becoming an indentured servant for a significant portion of their lives. Not many other places put that burden on their best and brightest. It is all about selfishness of the wealthy.
The idea of doing away with legacy admissions is a good one, but should not have anything to do with making the financial risk of investing in education for the non wealthy any more difficult. What the legacy wealthy hammerlock means is that the best and brightest have such a huge disadvantage that it affects our culture negatively... when potential society advancers are left to selling insurance as opportunity is lost to them when they could be better contribute to society as doctors, scientists, teachers. The science haters have held sway for too long. Making sure all have opportunity based on merit is important, and part of that is reducing the systemic advantages of dumb and less talented wealthy kids.