It may be that Williamson prisoner-of-war records 1) are misfiled, or 2) the set you received is incomplete.

All prisoners who weren't buried on Ship Island receieved transportation up the Mississippi River to a point near Vicksburg MS and released there. Anyone discharged on Ship Island would need to find their own way to the mainland.