Fern

Support centerSubscribe to updates
Powered by
Privacy policy

·

Terms of service
Write-up
Docs DNS outage
Full outage
View the incident
Summary

On October 12, 2026, every Fern-hosted docs site on a custom subdomain was unreachable for 56 minutes, from 1:03 to 1:59 PM ET. Vercel's Trust & Safety team blocked Fern's production hosting project by mistake. They were acting on a third-party abuse report about a single subdomain.

We had already been moving customers to new static infrastructure that doesn't depend on Vercel. During the outage, we offered an emergency DNS switch as a faster way to restore service. Now that Vercel has lifted the block, there's no rush: each customer can switch at their own pace.

Current status: all sites are up. Vercel has sent us their written root cause analysis. It's summarized below.

Impact

Customers on a custom subdomain still routed through Vercel lost their docs for 56 minutes.

Timeline (ET)

Time

Event

1:03 PM

Vercel blocks Fern's production project. Every Fern docs site served through Vercel goes down.

1:06 PM

First customer report: "This deployment is unavailable."

1:08 PM

Fern reports the outage to Vercel in our shared Slack channel.

1:11 PM

Vercel support responds. They're closed for the holiday, so they escalate to whoever is online.

1:11–1:55 PM

Vercel's internal escalation goes to the wrong team first, because their automated notice called the action an account suspension.

1:17 PM

Fern opens a public status-page incident.

1:30 PM

Fern ships a dashboard flow for customers to move their custom domains directly to the new infrastructure.

1:32 PM

Vercel opens an urgent Trust & Safety review.

1:34 PM

Fern points app.buildwithfern.com directly at Cloudflare, which brings back all subpath and reverse-proxy sites.

1:57 PM

Fern publishes self-serve DNS-switch instructions for subdomain sites. At the same time, Vercel's Trust & Safety team picks up the review.

1:59 PM

Vercel removes the block. All sites serve traffic again.

2:23 PM

Fern resolves the status-page incident.

  • Fern escalated to Vercel within 5 minutes of the block, 2 minutes after the first customer report.

  • Time to first customer notice: 3 minutes.

  • Most of the 56 minutes went to Vercel routing the case internally, on a day their support was closed for the holiday.

  • We migrated our monitoring from Checkly to Postman Monitors. While these were more accurate, they were misconfigured and only ran hourly. The monitor ran one minute before the block was enforced, so it missed the whole incident.

Root cause

Vercel's Trust & Safety team blocked Fern's entire production project after a third-party brand-protection service reported one subdomain on it. That subdomain was served through the same Vercel project as every customer's docs, so blocking it took all of them down.

Vercel has safeguards meant to stop this. Enterprise accounts need a second review from a Trust & Safety lead, and reports about a platform's end customers are supposed to go to the platform first. Neither one applied here:

  • A bug in Vercel's tooling skipped the Enterprise warning.

  • Vercel's tools didn't mark Fern as a platform that hosts many customers.

As a result, the Vercel's team thought the block would affect only the one reported site.

What Vercel has changed

  • Fern is now marked as a platform. Reports about our customers' sites come to us first, and Vercel acts on a single site instead of the whole project.

  • Blocking a project on an Enterprise account now needs a Trust & Safety lead.

  • Vercel fixed the bug so the Enterprise warning always appears.

  • Vercel's tools now detect wildcard and multi-tenant domains and remove the option to block the whole project.

  • By October 16, Vercel's on-call reviewer can be paged directly, and blocking notices will say exactly what was blocked.

What we're changing
  • Contain each customer's site. A block or failure affecting one customer's site shouldn't take down any other customer's site.

  • Increase the frequency of our alerting. Our monitoring cron job will run every 5 minutes moving forward.