Chevy -
I'm not sure if your numbers pre-recession are correct. Are you using pre 2007 numbers or 1987 numbers?
The numbers I have were in 1980 were over 20k, now there are 6,812 as of the end of 2013. My understanding is that they are following a similar model to the credit unions where I have better data:
2007 - 8,332 CU
2008 - 8,215
2009 - 8,066
2010 - 7,710
2011 - 7,442
2012 - 7,165
Back in 2007 85% of credit unions had less than $100M in assets and 1.5% had more than a billion. Its now down to 79.6% under $100M and 2.7% over $1B. Regulations and the shrinking spread in loans are driving it.