Originally known as Glengyle, Virginia Manor evolved from a simple vernacular structure into a high-style country estate. It began in 1800 as the two-story log home of John Fleming. In 1856 the log house became the rear wing of the two-story, five-bay frame home of Thomas Burks. In 1890 the property was sold to the Rockbridge Company, a land development company hoping to capitalize on the development of the nearby town of Glasgow. The company president, Gen. Fitzhugh Lee, nephew of Robert E. Lee, set up residence at Glengyle. The development scheme failed, and in 1897 the Rockbridge County property was purchased by Mrs. George W. Stevens, whose husband was president of the Chesapeake and Ohio Railway Co. The Stevenses transformed the property into a summer residence, called Virginia Manor, further remodeling and enlarging the house and adding various outbuildings, including a private railroad station.