It's really the same thing: spelling a number doesn't change its meaning. There might have been a tendency to spell the number in a formal report. Most references use the number.

Most wartime references omit the term "infantry". Never seen this explained, but assume it's because infantry service is assumed unless otherwise stated. So, as seen in George's references, the usual wartime reference is 21st Alabama Regt.