Not entirely true. Google stayed private for six or seven years, by which time it was worth so much, and had so many shareholders, that it was required by law to go public.
And once upon a time, most companies didn't go public till they'd at least established some kind of business. For them, the next step was expanding that business by raising money, which is usually easier for public companies.
Penny stocks are different. Often they nothing more than an idea, and maybe a business plan.