There is a single record notation for a "A.Herring" (note the different spelling) as a Private of Co.I, 6th La Cavalry Regt. Resident of St.Landry Parrish LA, Surrendered at New Orleans May 26th 1865, Paroled on June 13th 1865 at Alexandria LA.
You can obtain a gravestone via the Veterans Administration free of charge. But the cemetery might have a charge to have it placed/set on the grave. If the cemetery has no records/knowledge of where his respective grave actually is, not sure what can be resolved with that.
Frederick