Normally you won't assign rank perspectives 2, 3, and 4 to a bus group, but if you didn't set a non-zero perspective mix percentage to them, that should have no effect. You can unassign those to avoid confusion in the future.
If I read it right, you are sending all 3 primary buses, each of which has its own outputs, to master mix 1. Then master mix 1 duplicates all 3 signals to outputs 1 and 2, which you don't want. I think you're hearing all ranks from channel 1 and 2 as well as the intended channel pair (1/2, 3/4, or 5/6), and that's making you think all ranks are going to all 6 channels, but I could have missed something.
Please remove the outputs (1 and 2) from master mix 1 and try it.
Also just a check, are primary buses 001 and 002 in one bus group for GT and PD, and primary bus 003 is alone in a bus group for SW? I'm trying to make sure I know what you're trying to do.