Sag Harbor — One Incorporated Village Split Across Two Towns, Built on a Whaling Fortune

What Actually Makes This Market Different

Sag Harbor is not one town, and it is not simply "the Hamptons" -- it is a single, incorporated village government, formed on March 26, 1846, that happens to straddle two separate Suffolk County towns. Roughly three-fifths of the village lies within the Town of Southampton and roughly two-fifths within the Town of East Hampton, the line running along Division Street. Practically, this means one mayor, one Board of Trustees elected village-wide, and one village police department hold exclusive authority over land use across the whole village -- but a given property still sits within either Southampton or East Hampton for town-level tax assessment, and the two towns' assessors have at times valued similar Sag Harbor properties differently, creating documented, real tax-bill friction between neighbors on opposite sides of Division Street. Harbor jurisdiction is split further still: the Village regulates land use and waters out to 1,500 feet from mean high water, the Southampton Town Trustees regulate docks, moorings, and dredging in town waters and operate two public boat ramps, and the East Hampton Harbormaster provides mutual aid mostly outside village limits. A waterfront or dock project here can genuinely require permits from more than one government.

The village was built on the site of a former Montauk-Indian village called Wegwagonock, with English settlement documented from 1707 and the name "Sag Harbor" first appearing on a bill of lading in 1730. Its most consequential early distinction came in 1789: following the Second Session of Congress on July 31 of that year, Sag Harbor was declared a federal port of entry, one of the first in the new nation, and sources describe it as second in importance to New York City in shipping tonnage at the time. The village's Custom House, built around 1770 and expanded by first customs master Henry Packer Dering (appointed in 1790 under George Washington), still stands and now operates as a museum run by Preservation Long Island. Sag Harbor also reportedly published Long Island's first newspaper, the Herald, in 1791, and had Long Island's first post office around the same year.

Whaling defined the village's 19th-century identity and its architecture. Sag Harbor rivaled New Bedford and Salem, Massachusetts, with the industry peaking in the 1840s before declining as kerosene and then Pennsylvania petroleum displaced whale oil starting in 1859; the last whaling ship out of the village, the Myra, captained by Henry W. Babcock, sailed in 1871. Herman Melville never lived in Sag Harbor, but Moby-Dick references "a sagacious Sag Harbor whaleman" in Chapter 12. James Fenimore Cooper moved to the area in 1818, married into the Delancey/Nicoll family, partnered with Sag Harbor's Charles Dering in the whaling business, wrote The Spy while staying at the Dering home, and based his character Natty Bumppo partly on Sag Harbor whaling captain David Hand. That whaling wealth is why the village's housing stock -- Greek Revival and Victorian-era former sea-captain houses and whaling cottages -- looks the way it does today, and why the entire business district is listed on the National Register of Historic Places as the Sag Harbor Village District. The Sag Harbor Whaling & Historical Museum occupies the Benjamin Huntting II House, an 1845 Greek Revival mansion designed by architect Minard Lafever with a roofline decorated in carved flensing knives and blubber spades; it was later owned by philanthropist Margaret Olivia Slocum ("Mrs. Russell Sage") from 1907 to 1918, became a Masonic Temple in 1920, and was purchased and opened as the whaling museum in 1945, with the upper floor still a working Masonic lodge today.

The 20th century brought a different kind of distinction: a genuine literary and cultural history layered on top of the port and whaling story. John Steinbeck owned a cottage on a point off Sag Harbor from roughly 1955 until his death in 1968, wrote The Winter of Our Discontent there while fictionalizing the town as "New Baytown," and began Travels with Charley from the same cottage. Colson Whitehead's semi-autobiographical novel Sag Harbor (2009) chronicles summers he spent as a child in the Azurest community. Betty Friedan, Lanford Wilson, Nelson Algren, Spalding Gray, and poet Kazimierz Wierzynski are all documented residents or visitors, and Langston Hughes visited in the 1950s while staying with historian William Pickens in the village's Black community. Poet and educator Olivia Ward Bush-Banks was born in Sag Harbor in 1869. Today, Canio's Books remains a long-running independent bookstore on Main Street and a documented literary hub, though its ownership and operating future were reported as uncertain as of 2024. Bay Street Theater, a nonprofit regional theater at 1 Long Wharf founded in 1991 by Sybil Christopher, Emma Walton Hamilton, and Stephen Hamilton, and the annual, free Sag Harbor American Music Festival (which marked its 15th year around 2024) round out the village's cultural calendar.

Sag Harbor's most historically significant modern chapter belongs to Eastville and the SANS communities. Eastville, on the East Hampton side of the village, is the oldest documented African-American community in Sag Harbor, a freedmen's settlement dating to the early 1800s. After World War II, the Azurest, Sag Harbor Hills, and Ninevah Beach subdivisions -- collectively SANS, or the Historically Black Beachfront Communities -- were developed as refuges for middle-class Black families excluded from other beachfront resorts under Jim Crow-era discrimination, beginning with Azurest, conceived in the late 1940s by Brooklyn schoolteacher Maude Terry and subdivided by her sister, architect Amaza Lee Meredith. More than 300 Black families purchased properties across the three subdivisions, drawing owners and visitors including Lena Horne, Harry Belafonte, B. Smith, Allan Houston, Colson Whitehead, Reynold Ruffins, Edward R. Dudley, and Langston Hughes, and by the 1950s-60s the community was compared to Oak Bluffs on Martha's Vineyard, with summer populations swelling to roughly 1,000. The Sag Harbor Hills, Azurest, and Ninevah Beach Subdivisions Historic District was listed on the National Register of Historic Places on July 10, 2019. Since then, a real and ongoing local debate -- documented in 2018 and again in 2022-2023 -- has weighed whether to add a further local historic designation to guard against teardowns as property values rise, a genuinely unresolved question with legitimate preservation and property-rights arguments on both sides.

On population and price, the honest picture involves real source variance rather than one clean number. The 2020 Census is reported as either 2,772 (Wikipedia's infobox) or 2,748 (World Population Review), both citing the same census -- best read as approximately 2,750 to 2,770 residents. That compares to 2,313 in the 2000 Census and approximately 2,169-2,171 in 2010. Median household income estimates swing from $100,900 (a 2015 ACS figure) to $129,821 (a 2024 estimate), a fluctuation more typical of small-population ACS sampling than a confirmed economic trend, so treat it as directional only. Recent ACS estimates put the village at roughly 90% White non-Hispanic, 6% Hispanic, and 3% Black or African American, down from a 7.44% Black share in the 2000 Census. No specific current median sale price or price-per-square-foot figure for Sag Harbor was confirmed in the research behind this page -- for live figures, consult a current Douglas Elliman or Miller Samuel Hamptons market report, or a local licensed agent.