100,000 of them were willing to join the Union Army and shoot at their former owners.

I think it was more like 180,000, counting those from both North and South. With, I think another 19,00 or so in the US Navy....not sure on that. And you're quite right, they were more than willing to shoot at their former owners.