Thanks, yeah I tried using linecast and it seems to help a little better but there seems to be no way to create a wall proof camera if I purposefully swing the camera really fast.
Update: I made a mistake in the position of the camera which caused it to move through the wall. Maybe I need to re-do the whole thing.