I realize that, but without the card, this opens up a whole new way for identity fraud.
Years ago at my first job we would hire people based on the social security number that they gave us. It was a metal refinishing plant so the employee turnover was huge. At the end of the year when the w-2's went out, we found out that many of the social security numbers were bogus.
Without ID and the SS card, who's to know if the number is real or not?