The "academic" historical works that deal with Canada during the Civil War period tend to portray a general split in public opinion in "Canada" (separate provinces at the time), with it shifting toward the North as the war progressed. There was definitely some unabashed support for Lincoln and anti-slavery, but it seems to me that the extent to which Canadians really supported the Southern cause is now downplayed. One of the arguments for the Canadian public supporting the North is the estimated 50,000 or so Canadian men who served in the Union army. Most of these men however did not leave Canada in order to enlist, rather had usually been long-time residents of the USA. Others had close family ties in the states bordering Canada. In my research I have come across only a minority of men who volunteered to fight for the Union, to end slavery etc. Many more were actually crimped or were bounty jumpers. In terms of men who were crimped, this really enraged Canadian public opinion. After the war Jeff Davis even visited Canada and was met by huge, adoring crowds. Only a few thousand Canadians may actually have fought with the South, some of the examples that I have suggest that a few may have been reluctant about it. One case is documented in a series of articles published in a contemporary Canadian newspaper, which I have compiled as an eBook. I know little about the author and just how "Canadian" he was, but he seems to have had family connections to the Brantford, Ontario area.

http://www.amazon.com/Year-Southern-Rebellion-Thomas-Sinon-ebook/dp/B00EDPDHDU