Strange, the James B. Baltimore I find in Co. C, Snead's Company (Fluvanna Artillery) Nelson's Battalion, is recorded as "Absent on 30 days furlough from April to March 6, 1864."
He was captured at Waynesboro, Va. March 2, 1865. His parole documents from Fort Delaware, describe him as a resident of Fluvanna, Va., 5' 6", Bro. hair, Blue eyes, Fair Complexion, being released June 16, 1865.
Records are found in two rolls for men of the Fluvanna Artillery, Capt. Huckstep's Co., Light Artillery and Capt. Sneads. There is no Baltimore listed in Huckstep's Co and only James in Snead's.
[Source, Compiled Service Records]
I find five other Baltimore's in Virginia units, none with the name James or associated with the Fluvanna artillery.