It's a curious phenomenon and deserves study. For instance, how many of these are based on first-hand observation as to a second-hand report from citizens? How many of these date from the first year of the war?
Particularly during the early period of the war, any Confederate camp would have included black cooks, teamsters, musicians and body servants in noteworthy numbers. We may speculate that many of the body servants left when masters were discharged from service, died of disease, or were killed or disabled by wounds. That may explain why most Northern reports of Black Confederates date from 1861 and the first half of 1862, and not later.