The John Riley buried in Vicksburg is at
http://freepages.history.rootsweb.ancestry.com/~taleese/soldiers_rest_vicksburg/texas/texas_1.htm

Sorry for the long cut and paste URL.

Tallulah is just across the river from Vicksburg, but you knew that. If he is the same John Riley who died on the trip from Tallulah to Vicksburg, it seems they dropped him off at Vicksburg's Fisher Funeral Home. That business him buried the next day, 1 Oct 1862.

He has two grave markers in the cemetery, both pictured on the web page given above.

The old marker is believed to be the original placed by his fellow soldiers. It has the date of death as Sep 30, 1862. Enlarge the photo to make out the death date at the bottom.

The VA marker has the date from the funeral home.

There is also a link under the grave markers for John Riley that will take you to a short info page about him.

------
I am also looking at this second John Riley --
https://www.fold3.com/image/11610117
But I don't know if he is the same man as above. This man signed on in 1861 and then he was discharged soon afterwards.