Jim,
Go to this page and click on "Bartlett's Dictionary of Americanisms" 1859. Looks like the term has been around awhile.
http://www.omnilexica.com/?q=hickory+shirt