I believe that maybe be Joseph D Payne, born Oct 30 1816 in Sussex Co, New Jersey, died Dec 24, 1892 at Hickory Creek, Cuivre, Audrain Co, MO.
1860 Census
Name: Joseph Payne
Age: 44
Birth Year: abt 1816
Gender: Male
Birth Place: New York;??
Home in 1860: Cuivre, Audrain, Missouri
Post Office: Hickory Creek
Family Number: 172
Household Members:
Name Age
Joseph Payne 44
Julie Payne 42
Charles Payne 19
Elizabeth Payne 17
John Payne 16
Anna Payne 12
Isabelle Payne 10
Abigal Payne 8
Joseph Payne 5
Barbary Payne 2
A link to a squabble he had with some 3rd Iowa Cav in a billiard hall in Mexico Mo 19th July 1862 is here https://www.fold3.com/image/290383863/
He should not be confused for a Joseph J Payne/Paine that served as a drummer for Capt Musick's Pike Co Home Guard as listed in the Hawkins files. That Joe Payne was also associated with Co A, 27th MO Mtd Infantry.
Place name: Hickory Branch
Description: See Hickory Creek.
Source: Leech, Esther. "Place Names Of Six East Central Counties Of Missouri." M.A. thesis., University of Missouri-Columbia, 1933.
Place name: Hickory Creek
Description: Rises in Cuivre Township, Audrain County, and joins Spencer Creek in Jasper Township, Ralls County. Named from the hickory groves along the bank. Hickory Branch on 1878 atlas. (Mrs. Jennie Alexander, O.M. Montague)
Source: Leech, Esther. "Place Names Of Six East Central Counties Of Missouri." M.A. thesis., University of Missouri-Columbia, 1933.
Place name: Hickory Creek
Description: A settlement and post office in Cuivre Township, established as early as 1853 and discontinued before 1876. Named from its location near Hickory Creek (q.v.). (Hayward, 824; C.B. Kennan)
Source: Leech, Esther. "Place Names Of Six East Central Counties Of Missouri." M.A. thesis., University of Missouri-Columbia, 1933.
Paul Moser lists Hickory Creek as a post office 22 miles east of Mexico Mo also known as Barneyville. It is in the SE corner of Audrain county near the Montgomery county line.
Barneyville Post Office (historical), Missouri
Barneyville Post Office (historical) is a post office located in Audrain County, MO at N39.16900° W91.50149° (NAD83) and at an elevation of 725 ft MSL. Thats essentially at the intersection of W and Audrain CoRd 620.
It can be seen on the USGS 1:24K topographic map Vandalia Lake, MO. Its within easy travel distance down route BB about 5 miles to Middletown MO and Middletown would have been the nearest community of any size and services to it. (BB being the road west from Middletown through Scotts Corner to Mexico MO.)
His tomb (Saxe Cemetery, Audrain Co, MO) is marked with a Union marker as a Pvt Co B 67th MO EM, There is a service record for that unit having been deployed in and around Danville Mo in 1864. The best fit is this card from MO SOS:
http://www.sos.mo.gov/Images/Archives/Military/s00804/s00804_0436.pdf
His son Charles G Payne is in the same unit:
http://www.sos.mo.gov/Images/Archives/Military/s00804/s00804_0433.pdf
UDC has a card on him as Capt. in 3rd Mo Infantry (confederate 1861 - 1862),
I'm only guessing but suspect he played with the "southern states rights" folks that were predominant in that part of Little Dixie early on, but never traveled far from home. His age was against him. He likely did what was necessary to survive later in the war, enrolled and kept his head down... He lived out his years on his farm at Hickory Creek being listed in the 1870 and 1880 census rolls there.
Of interest is this article from the August 27th 1863 Missouri Citizen:
Union Meeting
-----
At a meeting held in Quiver{sic} township, Audrain County, Missouri, on 22d August, AMOS PETTIS, was called to the Chair, and JOSEPH PAYNE, elected secretary.
The object of the meeting being briefly stated by L. C. MUSICK, after which the following resolutions were adopted.
1st. Resolved – That we are in favor of a vigorous prosecution of the war until the last Rebel is subdued, without regard to cost or time.
2nd. Resolved – That we are in favor of immediate emancipation, believing that the life and health of the nation demands such a course.
3rd. Resolved – that we are opposed to the course of Gov. Gamble – believing that it is calculated to strengthen the rebels and weaken the Union cause.
4th. Resolved – That we condemn the course of the convention – because they assumed a power which belonged alone to the people, thereby striking at the very foundation of our Republican institutions.
5th. Resolved – While we believe there are some amendments necessary, both in our Federal and State Constitutions, we are in favor of a strict adherence to law as it is; until those amendments can be legally obtained.
6th. Resolved – That we recommend the Legislature, to call a Convention for the purpose of taking into consideration the subject of emancipation, at the earliest day practicable.
7th. Resolved – That a copy of these proceedings be sent to the St. Louis Evening News, Louisiana Journal and Missouri Citizen, for publication.
The utmost harmony of feeling existed and all seemed to feel the importance of action, to secure fully and maintain firmly, their political rights.
Cuivre township in Audrain county was predominantly Union leaning. The notorious Major Bay / Ham Brown Massacre involved folks from the 67th EMM so it strikes me as being plausible that the Payne's had all sorts of reasons to "switch sides"
FWIW...Audrain County History lists Alpheus Payne as a Capt. in Confederate service