Checking the Richmond City map ca 1860s, I'd have to believe your ggrandfather was buried in the Hollywood Cemetery.
I find a number of men who died at Jackson at that time were buried at Hollywood. See map at:
http://www.mdgorman.com/images/Hospital_Map.jpg
Click on the lower left corner to get a blow up.
Jackson Hospital was to the SW of Hollywood whereas Oakwood Cemetery is way off to the NW of the city.
George Martin