I worked on the issue yesterday, and I found kinda a solution.
When I'm using raycasts for ground dectection, I had the problem of the character often fallen through the ground. The issue was that it wasn't being detected, I had a distance of 0.2 which meant that in almost half the cases it didn't hit the floor with a ray, and just went through it instead. So I increased the ray casts to have a distance of 0.4 which helps, but apparently it detects the floor at different distances, so sometimes my character would be half through the floor or with hes knees into the ground. Which looked bad. So I had to do a quick fix where I offset the character as he lands, I can output a hit distance from the raycast action and then subtract that from my ray distance which gives me the offset I need. Btw I run over 60 fps so it shouldn't be a performance issue.
But I'm still wondering if there is a way to get more solid ray casting, if I turn up quality of physics would that matter?