Have done a bit of work on this regiment in the past. This is the story of one, but not necessarily your, A.J. Johnson of Caudill's outfit. During the first weekend of August 1862, Caudill's company, still part of the 5th Kentucky Infantry, CSA, came down Levisa Fork of the Big Sandy River from Tazewell Co., Virginia to recruit in Pike County, Kentucky. At the same time the area was being foraged by a newly-formed regiment of Virginia militia, the 4th Virginia State Line. On the first Sunday of August '62, Caudill's company got into a hot skirmish with some local Home Guards under a Captain Thornsberry (Thornbury) and A.J. Johnson took a bullet through the fleshy part of both buttock cheeks. He was taken in by a local family and nursed back to health until he could rejoin his outfit, which had probably grown to regimental size by that time. When the war was over, Johnson came back to the area and married a daughter of the family who had taken him in, Melinda Ratliff (who was a McCown in the 1860 Pike Co. census), and lived in the area until he died. He is buried in the Ratliff Cemetery at the mouth of Shelby Creek in Pike County. Now, whether this is your Andrew Jackson Johnson is another matter altogether. This old boy was originally from the Wythe County, Virginia area and was several years older than his young bride, Melinda. I got this information years ago from Russell Mullins, irect descendant and historian of Johnson, who has since passed.
If tis doesn't put you on the path to finding your Andrew, it will at least weed out a conflicting possibility. Keep me informed of what you find. I'll be most interested.
Randall Osborne