Oak Grove was built in stages between the 1750s and 1833. Its varied elements capture the gradual evolution of Virginia architecture during that time. It began as a two-story dogtrot log house built by tobacco farmer John Payne. His son, Philip, inherited the property in 1784 and enlarged it. His son, Philip M. Payne, completed it by 1833 with the addition of a classic two-story, three-bay dwelling with a side-hall plan. The house retains most of its interior trim, some of it with significant false-grained finishes. Extant outbuildings include a smokehouse, slave quarter, grain shed, and hay barn.