Firstly, congratulations on a pretty scary soundscape... I wanna play this game! Sounding a bit Amnesia, the scariest game EVER due to the sound!
Now I dont really think that what you are doing SHOULD NOT WORK; so dont have a solution for that. But it seems a bit funny to me to pile all your sounds onto your player character and have them fade in and out.
Why not just place you sounds IN THE SCENE, on game objects, either control the level with careful range limits (completely drag-able in the scene) Or maybe for more control have the trigger box fade up that audio game object? Just reference the Audio Game object instead of your player. Obviously things like the breathing sound are probably good to cart around with you, but incidental sounds that you come across should be placed in scene. Waterfall sounds in waterfalls, cars in cars, and nasty gnashing of spiky teeth in Nasty Spiky toothed Creatures...
Just an idea to try?