Now that you recall his middle name was "S", that narrows the 6 Mississippi William Fullers down to just two, neither of which show middle initials in my source. The other four had middle initials, but not "S":

Private william Fuller: Cook's Company, (MS Horse Artillary)
Private William Fuller; Hoskins' Battery, Brookhaven Light Artillary

Oh, I also found the following,

Private W. Fuller, Company K, 21st MS Regt. Infantry.

So, now you have three Mississippi possibilities. With this information, you should be able to start Googling and dig out more on these three including what county their units were organized in.

Good luck in your search.