You supposition fails to take into account that Social Security has always ran a surplus up to this point in time (brings in revenue which offsets any outlay), and instead has been a borrower to help finance larger tax cuts to Corporations and wealthier individuals, which has also raised the interest payments, in the pie chart, that we pay on the national debt. In essence, payroll taxes largely paid for by the middle and lower class wage earners bring them a negative rate of return from the tax system, as opposed to the tax incentives derived in comparison with upper wage earners and Corporations. This difference even expands when you consider competition between the States on tax incentives to land new businesses, and outsourcing of American jobs, and capital. Depleted revenues are the main cause of our lack of greater infrastructure expenditures, which greatly expand our economy and revenues, by bringing us closer to full employment. Also, since middle and lower wage earners spend a greater portion of their income on necessities, this money has greater velocity in the economy, and returns even more revenue back to the government coiffures. That is why for example, analysis has shown that unemployment benefits and the food stamp program actually return more to the economy than they take from it, while large tax cuts have either a small rate of return, or in effect drain government revenues. The question will always come to whether we use Federal tax money to invest in our people and the economy, or if we use it as large piggy bank for those whom influence the ones that pull the levers.
Current and past military spending account for 54% of the Federal budget. That also does not take into account the supplemental war spending in our appropriations since 9/11, for at this point in time, our wars in Iraq and Afghanistan have largely be unpaid for, and will be borne by future generations of wage earners.