I implemented something like this a while ago, albeit in 2D. Without any details what doesn't work in your case, here's a general overview how to do it.
Animator Layers
Normally, one animation clip runs at any one time, and mechanim blends between them as you tell it to do. The layers allow you to run several clips concurrently. A simple example: you have your animation of idle and running. But you want your character to blink the eyes every now and then. You could put the eye blinking into each idle and walking, but that would look unnatural and interfere with a clean walk cycle etc. As a solution, you'll use a new layer, set it to weight 1 (so it's in full effect), and make a clip with just the eye blinking periodically so it looks good. In this new layer, this blinking clip is simply the default state.
Once you got the concept, you have to think about how to slice the mustard. For example, for a weapon system, it could make sense to make an unarmed set, e.g. idle and walking with the arms swinging. But that looks bad with the weapon attached. So you have a a "just the arms" states for idle and running on a separate layer.
Generally, I found it best to not actually blend that much, but still take advantage of layers. Simply animate different parts, and delete everything from the clip that isn't necessary. E.g. in my example here, the eyes aren't animated at all anywhere else. So there is no weirdness from blending. Their animation is simply added.
Change Only Arm Movement
So you could make a new layer, "arms". Don't forget to set the weight to 1. You put an empty state in there as default, since when unarmed, you want nothing overriding the arm movement in the base layer.
Next, equip the character with the weapon (temporarily). If you have an unarmed state, or weapon switch system, you need an extra step. Let's do this first...
Bind Points
Generally, this works by another dedicated (empty) GameObject that serves as the bind/mount for the weapon. And a second one, that is the "grip" of the weapon.
Bind: You likely want to add the bind as a child object to the hand. Make it visble by giving it a small gizmo (inspector upper left corner). Leave it there.
Grip: Has the weapon a proper pivot, where it is held? If not, let's make one. Make another empty game object, the grip, also give it a tiny gizmo and put the weapon as a child into it. Move the weapon around so that the parent's gizmo is where the weapon is held by the hand. Rename the weapon so it's clear it's just an element and not the "the weapon" itself, and name the grip as "the weapon". This whole construction from now on is The Weapon. The point of the exersize is that when you drop it onto the character, and zero it, it'll sit correctly.
Now drop the weapon we just prepared into the bind, we also made a step earlier, i.e. the weapon is child of it. Important! Reset The Weapon's transform (i.e. zero position and rotation). Now, leave the weapon alone, and adjust the bind objectto orient and position such that it looks good.
You need to tweak a bit, but you want that the The Weapon itself (the grip with the weapon mesh) has a zero transform as child of the bind, while also making sure that the grip is at the correct place at the weapon, and the bind is properly in the hand. Make sure this all aligns properly.
Weapon Animations
Let's run through the process for idle. Copy over your idle keyframes to a fresh clip, or duplicate and rename it for the "idle armed" version. In it, only adjust the arm movement to what you want it to look like with the weapone equipped. After you're satisfied: In the animation timeline you see a list of game object that have been changed. Delete everything but the arms / IKs for arms. So you are left with an animation clip where only the arms move in some fashion.
This new clip goes onto the new "arms" layer in mechanim. Connect that to the empty state you created before, and set the conditions there, e.g. a trigger or a bool. First try switching around the idle and idle armed/blended animation and worry about the rest later. But with this method you can make a recoil animation etc. and it's independent of running and idling etc.
Prototype in Playmaker
Create a FSM, with two states. Each with a finished state leading to the next one. In the first state, listen for a "key down" and let's pick "tab", and copy this to the second state. Test this. Hit play. You should be able to tab between the states. We will be throwing this away later on, so it's not super important (use the "key" versions for prototyping, and button for real, because buttons can be mapped).
Now add the set animation actions accordingly into each state. One is for the armed version, one for the unarmed one. When you play, you should now see how the character's arms go from one version to the other, and you can switch back and forth using "tab". If you still have the weapon attached, you can see whether it looks good.
If that still doesn't do the trick, figure out what exactly doesn't work. Rather don't spent on another asset until you know what you're trying to solve.
Hope that helps, cheers.
