You mean you use the phone's orientation?
The device's camera acts as Unity's main camera I suppose.
Are you doing some augmented reality thing?
The idea likely remains the same, since you orient the Unity camera according to the orientation of the real device.
You need to have your 3D modeled arms follow the Unity camera with a little delay, through using a lerp for example.
No that's not correct, I am not using the phones camera as the main camera.
As you can see in my video above the character is just "holding" the phone.
Is there a specific way to use a lerp? If I make the phone a child of the camera the phone will follow 1:1 with no delay.
Right, I see. You have the phone's position that jitters horribly when it should not.
Yes, making the phone a child of the camera would solve this, it would inherit both position and orientation from the camera. However, it would be extremely rigid.
I suspect the shaking being due to the calculation based on the ever changing position of the camera, essentially restarting the gradual positioning at every frame.
What actions have you used exactly? Please make a capture of them.
There are tricks, like displaying certain elements as a late update (good for graphics), and since you're doing a FPS, you could look into using a secondary camera that would only see the phone; you would need to set the culling so as to filter everything else for that camera and then project that second camera's render into a texture to project into the first camera.
There are FPS tutorials that help you set up the double camera system.
However the phone in cam 2 would still be stiff, you'll need to reuse the accelerations of cam 1 (main cam) to somehow apply some smooth slides on X and Y to your phone so it
appears to be affected by inertia as you move forth and back, spin on your feet, etc.