I work for a company based out of Louisville, KY which is extremely well ran, and we are worth close to one billion dollars. Our CEO lives in New Jersey, our COO lives in Connecticut or some place like that. They are hardly ever in the office, but are obviously active in the business.
That's the joy of the internet, Clone! You can literally work from anywhere!
While I do believe leaving the area while your company is fresh is strange, it doesn't throw up any red flags to me.