Ok, so it seems the problem was because, at one point, I was testing an animation state in the animator with an alpha channel. I tried the fade fsm it on a fresh sprite and it all worked up until I added the Animator Component that contains the state with an alpha animation.
So even if you aren't currently playing the specific animation on the game object, it will still not allow an fsm to control the color. There can't be an Animator component with a color animation state somewhere in that Animation Controller.
This is pretty obscure, but maybe this will help someone else too. I'm marking this as solved.