The Thomas J Tarpley in the 6th Tenn Inf. was apparently the same Thomas Tarpley listed as living in Madison Co as a single man employed as a saddler in the 1860 census (and born in Tenn). But that also lists him as 23 years old. Could you possibly have the wrong date of birth or could your Thomas Tarpley be another man?
Alan Whitehead