It's even worse when it's to obtain a shipping address for an order (vs. for security/identification purposes). Then the phrasing suggests that the schmuck on the phone *really* knows where you live, and you're just taking a wild guess.
Thanks, Elmer. Another thing that's going to irritate me from now on... ;)