Fellows;
The district # is the US Census Enumeration District #. No connection to a political subdivision. Enumeration district maps are available from NARA for 1850 but are not on line. Starting with the 1880 census some of these maps are available on line from the census bureau but tend to be metropolitan areas. Short of ordering from NARA not sure how else to see, but it appears in this case that the closest geographical drill down is at the County level.
John R