According to Straven (the leader of the Phoenix Gang), the original members of the Black Arm Gang were once part of the Phoenix Gang. After a quarrel this faction broke away, split and took half of the Shield of Arrav in one of their battles, and formed the Black Arm Gang.
Players without a gang can join this gang in the Shield of Arrav quest after giving Katrine two Phoenix crossbows. Also, players who have joined the Phoenix Gang will not be able to join this gang. Therefore, if you want to complete the quest, you need a friend who is not a member of your gang.