I'm a glass half-full kind of guy. Let's look at what SCOTUS did:
They picked one sentence from “incidental power” saying “for the benefit of the GSEs, or agency” and self-added that the Agency represents public interest.
They clarified that the FHFA is a government actor, and acting in the best interests of itself/the public.
My interpretation is the Conservator is now authorised to rob the Conservatee.
They verified that is a governmental taking rather than an illegal exaction.
Both of these comments are very useful to Fairholme's other lawsuits. It's a blessing that SCOTUS included the language "or the public it serves," because you can directly compare the the 5th amendment: "nor shall private property be taken for public use, without just compensation."
While I don't think that SCOTUS got the APA claims right, I believe they are setting us up for a takings win. It's lazy on their part because they didn't want to get involved with second-guessing FHFA"s decision-making. But they are ensuring that there will be consequences for FHFA's actions.
Worth noting that in oral arguments, one judge even says "then why did you not bring a takings claim? Plaintiffs say they have. SCOTUS knows how this is going to end.