The 51st Virginia Infantry vacated Fort Donelson, Tennessee, under the cover of night across the Cumberland River, after being surrounded by Union forces Union General Grant. They weren't deserters, though there may have been some among them that did desert.

Many of these soldiers reenlisted as soon as they arrived back in Patrick County. My great grand father, Captain David Lee Ross, reenlisted in 21st Virginia Cavalry on his return and served until the end of the war. The 21st & 23rd Virginia Cavalry are noted for raided yankee towns throughout Pennsylvania, Maryland and New Jersey. In short, creating havoc and disrupting Union lines of communication.

I suggest you purchase the book on the 51st Virginia Infantry as I imagine it will shed light on your search.
author James Davis
publisher H. E. Howard, Inc., Lynchburg, Virgina 1984