The market maker is not keeping track of which shares go where. They don't have numbers on them and he wouldn't know who bought them and when they sold them.
he just has to has to have as many borrowed as he has short. The person who doesn't have shares is the one who had them borrowed. They have the IOU.
Same reason if you have a margin account you might not get a vote at a corporate meeting. your shares might have borrowed out and the vote goes with it. You also might not get dividends. Usually the broker will cover the dividend out of the money made lending the shares.