If the silence continues, before any serious talk of replacement, it may be prudent to have him surrender his passport for safekeeping. Not sugesting or saying he has ever done anything wrong, ever, just looking for a smooth transition if that ever takes place.