Yea, I actually have a similar issue using Get / Set Position but with a camera - the player has a trail renderer and I notice that using that method to make the camera follow it, there's a noticeable jerk to the trail motion.
I wonder if the addition of a Fixed Update to the action would do the trick?