If I have correctly ID'd him as son of (Dr.) Daniel (E.?) and Caroline Huger, he was 13 in the 1860 census, which would make 17 his correct age in 1864.