So, a little update with my progress. I couldn't get Get Angle to Target to work properly (I'm not sure if it's more suited for 3D-environment) but I made workaround.
Problem for Float Compare of Mouse X/Y is that it's based of where in screen you're pressing rather than where moveable object is. For example when you press somewhere upper screen, and object is border of upper side, it still play going up animation even if it's actually heading down.
My workaround that works 98% of times is that it won't play animation if transformation of X or Y is small. That way, it won't do moonwalking and it's something that no one probably won't notice.
I mark this solved, but feel free to comment if anyone has some input.