When a ball is replaced on the putting green it's in play even if the ball marker isn't removed (Decision 20-4/1) and there's nothing in the Rules saying that the ball marker must be removed before the ball is played. Several mails from the R&A on the matter confirms this...to me...