Here's information previously posted. It is not clear when James was posted to Gen. Ganty.

However, as an ADC, he would serve the General as his most senior personal aide, see:

http://en.wikipedia.org/wiki/Aide-de-camp

James Boykin

Card No. 5 in his file with Lenoir’s Independent Company, Alabama Cavalry, notes

“Vol. ADC to Gen Cantey”

...

James Cantey was appointed a brigadier general to rank from January 8, 1863. Cantey was detached and sent to Mobile, Alabama from January 1863 through April 1864

http://en.wikipedia.org/wiki/James_Cantey

Gen. James Cantey, Commanded the Garrison at Mobile,
The Photographic History of the Civil War, Vol. X