It would appear the family may have relocated to Campbell County, Georgia after the 1860 census.

State of Georgia, Campbell County

Before me Wm B. Jackson, a Justice of the Peace in of or said County personally came Israel Pannell who being duty sworn said that he is the Father of Benjamin F. Pannel deceased & late a Private in Company C, 35th Regt., Georgia Volunteers . . . .
July 18, 1863

M266: Compiled Service Records of Confederate Soldiers Who Served in Organizations from the State of Georgia