It will be affordability. How much long do you think housing prices can keep appreciating before all the demand is gone? What happens when all the demand is gone?
Everyone's house didn't just increase by 10-20%. There just a percentage of the population over paying.
Look around. Everything is going up...except for employment and salaries.