Elm Hill was one of the Roanoke River basin’s early plantation houses. Although unoccupied and in deteriorated condition, the T-shaped house preserved nearly all of its original fabric including bold provincial Federal woodwork. Elm Hill was built ca. 1800 as the residence of Peyton Skipwith, although title to the property was held by his father, Sir Peyton Skipwith, of Prestwould. Elm Hill remained the property of the Skipwiths into the late 19th century. Owned in recent years by the Virginia Commission of Game and Inland Fisheries, title to the house and a small surrounding parcel had been transferred to the Historical Society of Mecklenburg which had intended to undertake a long-term restoration. The house overlooked Lake Gaston and the John H. Kerr Dam.

The house at Elm Hill was destroyed by a fire, likely sparked by a lightning strike, on June 25, 2014.