Making a controller that takes physics into consideration?

[Designerwatts]

Hi everyone.

So I've been working on making some isometric prototypes. My latest was a 2-week project. The exported game can be accessed here: https://www.dropbox.com/s/x5g0wk2hkcdec6o/2WeekProjectExport.rar?dl=0

So one of the bigger issues of this prototype I'm looking to try and solve for the next prototype is creating a isomeric (Diablo-style) control system that's similar but actually takes hitting physics objects into consideration.

As of right now my movement model is that when I left tap the mouse on the floor it does a ray cast and creates a vector 3 from it. The "Player" object then moves and rotates towards the location via a "I tween move to" https://www.dropbox.com/s/ppyc2er5vo7bvcs/Movement%20Model.PNG?dl=0

The issue at the moment is that any attempt to give these objects ragidbodies turns the movement into jittery madness that quickly ruins the player object.

So, how could I create a similar isometric control style of an object traveling towards a mouse-point I click, but also allowing it to have dynamic physics that will stop when it hits a collider?

Thanks for your time everyone.

Any ideas?