A part of this house may have existed as early as 1855, when site and improvements were sold to John E. Shelton. He was a master craftsman who built other fine houses prior to 1860. Shelton built the main portion about 1858 for his friend and business partner, Thomas Wesley Smith (1829-1902), who later was a leading cigar manufacturer in this county. Smith and his heirs owned the house until 1924. A granddaughter added the dormers in a 1922 remodeling. The 1924-1970 owners were the Thomas A. Gay family. Mr. and Mrs. K. M. Whisenant now (1976) preserve it.