I looked at the 1850 census again. Looks like he was 42 (maybe 44 -- hard to read) in 1850, so should have been 53 to 55 in 1861, born around 1808. Maybe too old to have served.