Open only during the summer and for school tours by appointment, the Mather House Museum is a complex of buildings and gardens. The main house was once the home of the Mathers, a family of shipbuilders who built the home around 1840. The Mather House now serves a dual purpose as museum and headquarters for The Historical Society of Port Jefferson.
On the grounds are several buildings that showcase artifacts from the 19th and 20th centuries. Among the collections are a craft house and museum shop, tool shed, country store, marine barn and sail loft and The Spinney Clock Museum. Next to the craft house is The Thomas Jefferson perennial garden, maintained by the Suwassett Garden Club.
The Historical Society's collection on-site includes ship models, historical books, vintage tools, vintage clothing, period furniture, antique toys and taped oral histories. There are also special exhibitions on occasion.