Dr. Jeptha Vining Harris's military service records should contain his service in the following units according to the info he provided in his pension application.

Corp. J. V. Harris
Company C
4th. Battalion Mississippi Cavalry
Unit capsule history -
http://mississippiscv.org/MS_Units/4th_Bn_MS_CAV.htm

Private/2nd. Corporal J. V. Harris
Company E - 2nd. Regt. Miss. and Ala. Cavalry*
8th. Confederate Cavalry (Wade's)

*This company was formerly Company C, 4th. Battalion Mississippi Cavalry and Company C, 2nd. (Brewer's) Battalion Mississippi and Alabama Cavalry.
The 8th (Wade's) Regiment Confederate Cavalry (also known as the 2nd. Regiment Mississippi and Alabama Cavalry) was formed in May, 1862, by the addition of three companies, which had formerly served in the 1st. (Beall's) Battalion Alabama Cavalry, to the 2nd. (Brewer's)Battalion Mississippi and Alabama Cavalry.

Also see the information at this site.
http://www.tarleton.edu/~KJones/8csahist.html

In the 1860 census, Dr. Harris was a 21 year old physician living in Columbus, Lowndes County, MS which is where he joined for duty and enrolled November 14, 1861 for 12 months.
Under Remarks from a company muster roll dated August 31, 1862 - 2nd. Corp. until 21 of April (1862) then detailed as A.S. (Assistant Surgeon).
What is not noted in this muster roll is his discharge from this regiment August 3, 1862 which should lead up to his service as Asst. Surgeon in the CS Army and Navy.

His complete military service records are available at this site.
http://history-sites.com/research/index.html