In January 1798 the Kentucky Legislature authorized the moving of the Madison County seat from nearby Milford because of the inconvenience of the location.

The move was bitterly opposed by about 300 residents of Milford and a fight to retain the seat resulted between Dave Kennedy, who represented Milford, and William Kearly, who represented the new site. Although history does not record the results of the brawl, the county approved the move in March 1798 and the first Court was held in Miller's barn.

On July 4, 1798, the site was officially named Richmond in honor of Miller’s birthplace, Richmond, Virginia and fifty acres were surveyed.