The point is, even US presidents befriended a person who espoused anti-semitism - they did not ban/disown him.
"...even if he kept it private (if true)" - I presume you are not suggesting the voices on the Nixon tapes may not belong to Nixon and Graham, are you?