Retiring in Cape May: A Small City You Can Walk, Most of the Year
Cape May's 2020 census population was 2,768 -- a genuinely small, walkable city, and per Wikipedia the only city in the United States wholly designated a national historic district, with more than 600 Victorian-era buildings covered by a National Historic Landmark designation since 1976. That is a real, citable distinction, not marketing copy, and it matters for retirement in a very practical way: the historic downtown most people picture when they think of Cape May is not a curated tourist strip set apart from daily life, it is essentially the whole town. But the honest picture also includes a real seasonal swing -- that same small city sees an estimated 40,000 to 50,000 visitors each summer, a scale shift that changes traffic, crowds, and pace for months at a time. This page tries to hold both of those facts at once, along with a plain note that New Jersey is this site's first state outside Delaware, with a genuinely different tax framework that this page hedges rather than guesses at. No invented healthcare facility names, no invented tax rates, no invented home prices -- just what the research confirmed, and pointers to who to ask for the rest.
A National Historic Landmark You Would Actually Live Inside
Most retirees who consider a historic town are choosing between a genuine old district and a newer development built to look old. Cape May removes that choice: per Wikipedia, the entire city -- not a preserved block, not a museum district carved out from the rest of town -- was listed on the National Register of Historic Places in 1970 and elevated to National Historic Landmark status in 1976, a designation Wikipedia describes as making Cape May "the only city in the U.S. to be wholly designated as a national historic district." More than 600 buildings, most from the Victorian era, sit inside that district, and a National Park Service architectural historian is quoted describing it as "one of the largest collections of late 19th century frame buildings left in the United States."
For someone weighing where to spend retirement, that matters less as a superlative and more as a description of daily life: a walkable downtown of real Victorian houses and storefronts is the town itself, not a themed attraction you visit and then drive home from. This page does not have confirmed, current information on specific walkability scores, sidewalk conditions, or parking availability, and does not invent them -- those specifics are worth confirming with a visit.
How a Fire Built the Streetscape You Would Be Moving Into
Cape May's Victorian look has a specific, dated cause, and it is not simply age. In 1878, what the City of Cape May's own official history page calls "the most devastating fire of all" tore through the city -- the city's own account puts the destruction at approximately 35 acres, while WHYY's own reporting describes the same event more broadly as having leveled most of the town. Rather than force those two figures into one reconciled number, this page presents both, because neither source confirms a single agreed-upon damage total. What both sources agree on is what came after: a substantial rebuilding effort in Queen Anne, Gothic, and American bracketed architectural styles, followed by larger bungalows and mansions built between 1900 and 1920.
That rebuild is the direct reason the town looks the way it does today, even though the settlement underneath it dates back centuries earlier. And the streetscape nearly did not survive a second time: Cape May received a $3.5 million preservation-focused Urban Renewal Grant in 1963, the entire town was listed on the National Register of Historic Places in 1970, and National Historic Landmark status followed in 1976. WHYY's own reporting credits that 1970s preservation push directly with saving and formalizing the Victorian character for the long term, rather than letting it be redeveloped away as happened in many other aging shore towns. For a retiree, that history is a reasonable proxy for a town that has chosen, twice, to protect what it has rather than replace it.
A Resort Older Than Almost Anywhere Else on the Coast
Part of what makes Cape May feel settled rather than newly discovered is that it has been a resort for a very long time. European settlement traces to the 1630s, when the first residents purchased land from the Kechemeche Lenape people and built fishing and whaling industries; by 1766, Cape Island -- Cape May's earlier name -- was already developing, per the city's own history, "as a place where many resort for their health and pleasure." By 1834, six boarding houses were drawing wealthy visitors from major eastern cities, and Congress Hall Hotel opened in 1816 and still operates today, a genuine, dated anchor of the resort's earliest era. The New Atlantic hotel opened in 1842, accommodating 300 guests, and in 1847 Henry Clay spent two weeks in Cape May -- a visit the city's own history credits with marking "the beginning of its reign as the Queen of the Seaside Resorts."
Construction began in 1852 on the ambitious Mount Vernon Hotel, designed for 3,500 patrons, but it burned in 1856 while still under construction -- a smaller, earlier fire that previewed the far larger 1878 disaster. The 1863 completion of the railroad from Philadelphia opened what the city calls the "Cottage Era" of private summer-home construction, laying down much of the building stock the 1878 fire and rebuild would later reshape. For a retiree, the practical takeaway is not any single date but the pattern across all of them: this is a town that has been receiving visitors, building lodging, and rebuilding itself for close to two centuries, not a place discovered by retirees in the last decade or two. This page does not have confirmed, current details on Congress Hall's present-day ownership, room count, or operations beyond its 1816 opening and continued landmark status, and does not invent them.
The Honest Seasonal Swing: 2,768 Residents, Up to 50,000 in Summer
The number that matters most for a retirement decision is the plain demographic one. Cape May's 2020 census population was 2,768, down 23.3% from 3,607 in the 2010 census, spread across a total area of just 2.90 square miles (2.47 square miles of land, 0.42 square miles of water), with a highest point only 14 feet above sea level. That is a genuinely small, low-lying city. Each summer, that same small population is joined by an estimated 40,000 to 50,000 visitors -- not a modest bump, but a scale shift measured in multiples of the year-round population.
Both halves of that fact matter and neither should be hidden from the other. The off-season town is small enough that a walkable, National Historic Landmark downtown is genuinely quiet and livable day to day. The summer town is a different experience entirely -- more traffic, more crowds, longer waits, and a local economy built substantially around that visitor surge. A retiree weighing Cape May should plan for both realities, not just the postcard version of one or the other. This page does not project a current (2026) population figure beyond the confirmed 2020 census count, and does not state a current median home price, active listing count, or specific rental rate -- none of those figures were independently confirmed this research pass, and none are invented here.
New Jersey Taxes: A Genuinely Different Picture, Hedged Honestly
Cape May is this site's first market outside Delaware, and that is not a small distinction. Every market built on this site before it sat in a state with no sales tax; New Jersey does levy a state sales tax, and New Jersey is also generally recognized nationally as a state with comparatively high property taxes -- a genuinely different starting framework from anything else covered on this site so far. That general reputation, though, is not the same thing as a specific number for any given retiree's own situation, and this page does not guess at one.
This research did not confirm current Cape May County or City of Cape May property tax rates, current assessment figures, the current New Jersey state sales tax rate, or how New Jersey currently treats pension income, Social Security benefits, or retirement-account withdrawals for state income tax purposes -- and none of those specifics are invented here. Anyone weighing a move to Cape May for retirement should confirm current property tax rates directly with Cape May County government and the City of Cape May's own finance office, confirm current state tax rules directly with the State of New Jersey, and talk to a New Jersey-licensed tax professional about how the state's rules apply to their own specific retirement income before treating any general reputation, favorable or unfavorable, as a settled financial fact.
What We Could Not Confirm, and Who to Ask Instead
This research pass confirmed Cape May's 2020 census population of 2,768, its 2.90-square-mile total area, and its estimated 40,000-to-50,000 summer visitor swell, but it did not confirm a current (2026) population figure, current median home prices, active listing counts, or specific current rental rates -- none of those numbers are invented here. It also did not confirm current Cape May County or City of Cape May property tax rates, the current New Jersey state sales tax rate, specific current flood-zone designations for individual parcels, or any 2025-2026 storm damage, erosion event, or beach-replenishment project specific to Cape May. It did not find and does not invent specific current healthcare facility names, distances to any hospital, or specific medical services available in or near Cape May -- anyone weighing a move for health reasons should research current healthcare access directly rather than assume any particular facility or distance.
For the specifics that actually drive a retirement decision, go to the sources that can confirm them directly: Cape May County government and the City of Cape May's own finance office for current property tax rates and assessment figures, the State of New Jersey for current state tax rules, FEMA's Flood Map Service Center for current flood-zone designations, a New Jersey-licensed tax professional for how state tax treatment applies to your own retirement income, and a local real estate agent who works in Cape May specifically for current housing stock, prices, and healthcare access. Given the strength of the town's underlying character -- a genuinely walkable National Historic Landmark downtown with a resort history stretching back to 1816 -- that conversation is likely to be a worthwhile one.
Ready to talk to a local Cape May agent?
Get a Free Agent Referral →Independent research. No ads. No sponsored listings. Data sourced from Wikipedia's own Cape May Historic District and Cape May, New Jersey entries (2020 census population of 2,768, down 23.3% from 3,607 in 2010; total area of 2.90 square miles; highest point of 14 feet above sea level; estimated summer population increase of 40,000 to 50,000; the National Historic Landmark designation making Cape May, in Wikipedia's own words, "the only city in the U.S. to be wholly designated as a national historic district"), the City of Cape May's own official history page (the 1630s European settlement and purchase of land from the Kechemeche Lenape people; Cape Island's 1766 development "as a place where many resort for their health and pleasure"; the 1816 opening of Congress Hall Hotel; the 1842 New Atlantic hotel; Henry Clay's 1847 visit; the Mount Vernon Hotel's 1852 construction and 1856 fire; the 1863 completion of the railroad from Philadelphia and the resulting "Cottage Era"; the 1878 fire's approximately 35-acre damage estimate; the 1963 preservation-focused Urban Renewal Grant), and WHYY's own reporting on the 1878 fire (described more broadly as having leveled most of the town) and the 1970s preservation movement that led to the 1970 National Register listing and 1976 National Historic Landmark designation. This is this site's first New Jersey market, following eight Delaware markets already built here -- New Jersey's state sales tax and its property tax structure are genuinely different from Delaware's no-sales-tax system, and current rates and retirement-income tax treatment were not independently confirmed this research pass and are not invented here. Also not stated as confirmed fact: any current (2026) population figure beyond the 2020 census count, current median home prices, active listing counts, specific current rental rates, current Cape May County or City of Cape May property tax rates, specific current FEMA flood-zone designations, any 2025-2026 storm damage or erosion event specific to Cape May, and any specific healthcare facility name, distance, or service. Confirm current details directly with Cape May County government, the City of Cape May's own finance office, the State of New Jersey, FEMA's Flood Map Service Center, a New Jersey-licensed tax professional, and a local real estate agent. Nothing on this page is legal, tax, medical, or financial advice.