Ugh, I feel stupid for getting the old position, getting the current position, subtracting them to get the change per frame, and using that as a direction...
Then I moved to simply looking at the old position. (which always faced backwards)
I've been doing that on so many projects...
I could have just used the current velocity! DOH!
EDIT: Here's some screenshots of 'looking at velocity'.