The structure originated as a one-story limestone dwelling, built between 1869 and 1871 by Charles Denny. Mrs. N. L. Holliday, a widow with six children, purchased the house in1898 and added the second floor in 1906. The residence was later occupied by her daughter, Margaret (d. 1921), a local physician. Margaret and her husband Dr. Simon J. Clark renovated the house about 1920. They added an elaborate entryway on the north side and stuccoed the exterior wall. Recorded Texas Historic Landmark - 1978