Rufus W. May is listed in the 1850 Census as 9 years of age and son of James H. and Drussilla May. In the 1860 Census he is listed R. W., 18 years of age and son of James H. and Caroline F. May, Living in South Fork township, Montgomery Co. Ar.