The U.S. has abnormally cheap natural gas.
From a heating perspective (BTU content to be technical), burning 6 mcf of natural gas is equivalent to burning 1 barrel of oil. 6 mcf of nat. gas in the U.S. costs less than $20, while the equivalent oil barrel is ~$65. So, heating your house with natural gas will always be cheaper than any other form of heating--electricity, oil, or coal.
Coal proponents would have you believe that U.S. coal production is dying as a result of regulation. That's garbage. It's dying because it's more expensive than natural gas in the U.S., since the U.S. is oversupplied as a result of the shale fracing revolution. Why pay more for coal, AND get extra pollution (most of the earth's mercury pollution comes from coal) and CO2?
A lot of U.S. electricity production comes from natural gas. But, that gas has to be burned in a power plant combuster (where the combustion efficiency is much better than your home gas furnace), then it heats steam, which turns a generator. The electricity from the generator output is then sent over power lines, where it loses on average about 50% of the power to transmission losses.
So, strictly from a cost perspective, it's much cheaper to just burn the natural gas in your home (at an unimpressive efficiency ratio), rather than converting it to electricity and transporting it long distances. On the other hand, it's easier to capture CO2 and noxious combustion byproducts in a central power plant.
Electrically-driven geothermal heat pumps (with heat exchanging pipes buried somewhat deep under your house) are approaching natural gas costs. And in warmer, sunnier climates of the southwest and parts of the southeast, where home solar photovoltaic is effective, you can compete with natural gas if you use electricity produced on your roof top to run the heat pump. The heat pump works in reverse in the summer when you need air conditioning.
Of course, insulation and weatherproofing are usually the most cost-effective home heating investments.
