Originally built circa 1813 in the Federal style for John Joyner Smith, a Beaufort plantation owner, this house was entered from its Bay Street facade. A Greek Revival era remodeling relocated the entrance to its present location along Wilmington Street and resulted in modifications to the interior plan. During the Civil War, General Stevens, the Federal Military Commander, occupied the house with his staff.