Please look at the service records for this Elijah Lee of the 4th Cav to make sure I have interpreted the residence as Covington County correctly. You know how difficult it is to read handwriting sometimes.
So this Elijah, son of Steven and buried in Louisiana, had the pension filed for and my Elijah, son of John Riley and buried in Pearl River County, has the headstone. There has evidently been a big mistake. And I'm about ready to conclude that my Elijah didn't fight at all, but rather, it was this other Elijah instead who was in the 4th Cav, simply because of the locations given on the service records. It just doesn't make sense as to why my Elijah enlisted at Port Gibson and gave his residence as Covington County. However, he is not found in either the 1850 or 1860 Covington County census, but he is found in the 1850 Perry County Census listed at 3 years old. So why list Covington County as a place of residence? That doesn't make sense either unless he moved from Perry County to Covington between 1860 when the census was taken and 1864 when he enlisted. However, I have in my notes that his grandfather, Elijah, died in Perry County.
Now, there is this other Elijah.....of course.....whom I did find in Covington County. He is the son of William and I found him in the Covington County 1850 census when he was 14. This is the Elijah who ended up serving in the 7th MS Infantry and dying in the train crash in Alabama in 1862. His wife, Eliza, and he are found in the 1860 census for Jones County. So they moved from Covington County to Jones.
So this just doesn't make sense at all. The Elijah who actually lived in Covington County didn't fight in the unit listed on the service records. And the Elijah listed on the service records didn't live in Covington County but Perry County, which isn't close in proximity to Covington County, which is on the service records. If you put all three Elijah's together, the service record/pension will be correct! But taken separately, something's horribly wrong.