The name rang a bell for me.
At http://www.mississippiscv.org/mississippi_history.htm, I see no 3rd Batt. listed. I checked the CWS&S and sure enough he is listed in the 3rd Batt.
Checking the cemetery listing, I only saw one A. J. Dunn listed. I believe you are looking for the 3rd Infantry Regt. based on this reference, H. Grady Howells "To Live and Die In Dixie" pages 508-509 listed as buried in the McGavock Cemetery -- A.J. Dunn, Co. E, 3rd Regt.
He is also listed again in the roster section as being in Co. E the "Biloxi Rifles" mustered into Confederate service Oct. 5, 1861 Shieldsborough, Mississippi.(Bay St. Louis)
HTH.
GP