In NYC you have the potential danger of a car going out of control and going into the sidewalk, therefore the man should be closest to the street. However, there is always the danger of getting jumped/mugged from someone hiding in the buildings as you walk, therefore suggesting the man should walk closest to the buildings.
My grandmother was born in the late 1800s and she insisted that when we walked to the store I walk on the curb side and I'd have to switch sides if we crossed a street. I finally asked her why and she said if a horse jumps the curb you'd get killed first.