"I heard somewhere that you didn't really need to wear a vest/waistcoat over your shirt if your shirt was pleated." Back to the original question. In Victorian times, certainly in the 1860's, it would be unacceptable for men to wear a shirt without a coat or vest in the presence of ladies---unless they had some compelling reason and asked permission from the ladies before removing the garments. For soldiers in the field performing pioneering work or simply off duty, these rules would not apply.