If I were you, I would cut the implementation with a knife and grenades - I would leave rifles and pistols - and the bug would go away with animation, and there would be no need to think about all this wisdom with controllers - as I said 2 there, 2 here
+ to all this - imagine that people can still use third-party plugins, hats for example and something else where there is a specific attachment - then there the player will simply swing and dance at lightning speed in idle time)