The 34th Mississippi Regiment was involved in the Battle of Perryville KY. You will come across some sources for the Battle of Perryville which mention this command and refer to it as the 37th Mississippi. That is not the regiment that you have in mind.
At the time of the Battle of Perryville, your 34th Mississippi was part of General Earl Van Dorn's army in northeast Mississippi, hundreds of miles away.