Originally a four-room house with south and east porches, this home was built about 1870 by carpenter and blacksmith D. H. Harkey. Dr. J. M. Johnson bought the property in 1896 and added the second story and the colonial revival style columns. In 1897 the house was sold to Confederate veteran John Payne, who ran a successful mercantile business in Palmer. Built of cypress, the home remained in the Payne family until 1906.