According to oral history, Alexander Waller was the first person to be buried at this site. He died shortly after helping his father choose the family burial ground on their land purchased in 1872. John Waller, a veteran of the 15th Texas Cavalry, and his wife Frances (Dyer), had eleven children, several of whom are buried in this cemetery. Other Confederate veterans are interred here: Thomas Cothran, Jesse Long, and Oliver Smoot. Many more Sulphur Branch community families rest here, including two African-American families originally buried outside the main cemetery. Waller Cemetery continues to serve descendants of the early residents of Sulphur Branch.