Originally known as Trinidad, Rosser depended on the Trinity River for trade until 1882, when a line of the Houston and Texas Central Railroad was built here. The company was later renamed the Texas Midland Railroad by Mrs. Hetty Green of New York, who bought the line in 1892. Operated by her son Edward H. R. Green, it had a dramatic impact on the early growth of Kaufman County. Rosser supplied lumber for ties, gravel for roadbeds and water for the locomotives. The brick depot, built here in 1917, replaced an earlier frame depot which burned.