Hi,
Just want to update on how I resolved this issue incase anyone else has the same problem. So it was the Blend time Option. I left it at the default .3, and changing it to zero solved the problem.
I dont know if its common thing or I setup my animation incorrectly.