cbrad - if you haven't already, check the Board's, " BAN list", of posters - against your favorites column. These BAN posters could be giving temporary marks, withdrawing the marks shortly after the issue.
How can it happen? One way is if Lurkers who are on the Ban List send PMs offering you an exchange. They may or may not list this board as where they saw you. That exchange is very likely negated by them quickly resulting in much of what many go through, temp marks.
Those are the only marks I have removed thus far since I have been on the board. I am thinking they are a major underlying reason for my MM fluctuation.
Likely as not the list will grow based on the practice of giving and taking away marks but the posters will need to be identified first, so let the mods know if you do identify any posters doing it.
Review the ban periodically to insure your favorites are not on that ban list and remove the MMs if they are.
BOL
