I was simply postulating the same idea as Art was suggesting below. The fact is that an army in the field, and its component parts thereof, draws its supplys from somewhere. With Vicksburg and later even Jackson, Mississippi under attack Port Hudson would have become the center of military logistical activity.
There are always people who are "left behind" for the logistical purposes of obtaining needed supplies, adminstration, such as teamsters, blacksmiths and even lack of horses, and yes even sickness. Isn't it true in the army today that for every soldier in the field there are 7 back at base to support them? During this conflict absolutely that level of support was not the case. But we certainly know that free Blacks were enlisted to free the white soldiers doing these jobs to be returned to the ranks.
I just feel that it is a false assumption and generalization that all of these men were mearly in the hospital sick. Especially when there is some question as to why they are there in the first place. Obviously there was no previous mention in their service record thhat they were sick. In fact in studing federal cavalry units, and I am assuming here that the same would be true in Confederate Cavalry units, that because they usually were on the move they tended to have less of a sickness problem that infantry because they were not cooped up in camps all the time. Second the lack of proper mounts usually decreased the effective fighting force of a cavalry unit by as much as 25%. If you don't have a horse, in a cavalry or mounted infantry unit, what do you do?
Just food for thought.