Incident Report: Documentation Site Performance Degradation Date: June 2–3, 2026 Duration: 30 hours Severity: High Status: Resolved
Summary
Between June 2 and June 3, 2026, documentation sites hosted on the Fern platform experienced intermittent slow page loads. Pages that normally load instantly were instead taking several seconds to render. The issue was resolved on June 3, and all sites have been fully recovered.
In the spirit of never washing a crisis, the fixes we made have improved cache hits and page load speeds across sites.
What Happened
We were making improvements to our cache revalidation pipeline — the system that ensures your documentation pages are updated quickly after you publish changes. As part of this work, we changed our invalidation strategy to be more aggressive: instead of marking cached pages as "stale" (which still serves fast cached content while refreshing in the background), we switched to fully deleting cached pages (which forces a fresh render on the next visit).
This change exposed a bug where our automated health checks accidentally cleared all customer caches every 30 minutes. As a result, visitors frequently experienced slow page loads due to uncached content. During the incident, our metrics showed cache hits dropping to 60%, leaving a 40% cache miss rate.
Why It Was Difficult to Diagnose
During the weeks leading up to this incident, we were actively improving our documentation rendering performance. We shipped over 20 improvements to page rendering speed, the caching infrastructure, and cache-warming logic.
Each fix appeared to work — pages would become fast again — but within 30 minutes, the monitoring job would inadvertently wipe all caches again. This created a recurring pattern in which improvements seemed to "regress" shortly after deployment, leading us to continue investigating rendering-layer issues rather than the cache invalidation path.
The bug was invisible before June 2 because our previous "stale-while-revalidate" strategy still served fast cached content to visitors even when the underlying cache was being incorrectly invalidated. The switch to hard-delete removed that safety net, making the issue immediately visible.
Resolution
On June 3, once the root cause was identified:
• Reverted the aggressive invalidation strategy — restored the "stale-while-revalidate" approach so pages would continue to be served fast from cache even during invalidation.
• Fixed the scoping bug — corrected the invalidation logic so that cache clearing for one site only affects that specific site, not all sites on the platform.
• Backfilled all caches — ran a full cache warming pass across all customer documentation sites to immediately restore fast page loads.
All documentation sites recovered and have remained fast since the fix was deployed.
Positive Outcomes
While investigating this issue, we made substantial improvements to our documentation rendering infrastructure:
• Faster cold-start renders: Even when pages are uncached, they now render significantly faster than before.
• Improved cache warming: Pages are now warmed asynchronously after publication, reducing the window during which visitors might hit an uncached page.
• Better caching architecture: Added an additional immutable caching layer for content serialization, reducing redundant processing.
• Removed performance bottlenecks: Eliminated slow dependencies from the critical render path.
Preventive Measures
Action
Status
Fixed the cache invalidation scoping bug
In Progress: 6/8/26
Restored safe invalidation strategy
Complete
Backfilled all customer site caches
Complete
Adding automated cache-hit-rate verification across customer sites
Complete
We sincerely apologize for the impact to your documentation sites. The performance improvements we shipped during the investigation period mean that your docs are now faster than before this incident began, and the fixes we've put in place prevent this class of issue from recurring.
Resolved
Incident Report: Documentation Site Performance Degradation Date: June 2–3, 2026 Duration: 30 hours Severity: High Status: Resolved
Summary
Between June 2 and June 3, 2026, documentation sites hosted on the Fern platform experienced intermittent slow page loads. Pages that normally load instantly were instead taking several seconds to render. The issue was resolved on June 3, and all sites have been fully recovered.
In the spirit of never washing a crisis, the fixes we made have improved cache hits and page load speeds across sites.
What Happened
We were making improvements to our cache revalidation pipeline — the system that ensures your documentation pages are updated quickly after you publish changes. As part of this work, we changed our invalidation strategy to be more aggressive: instead of marking cached pages as "stale" (which still serves fast cached content while refreshing in the background), we switched to fully deleting cached pages (which forces a fresh render on the next visit).
This change exposed a bug where our automated health checks accidentally cleared all customer caches every 30 minutes. As a result, visitors frequently experienced slow page loads due to uncached content. During the incident, our metrics showed cache hits dropping to 60%, leaving a 40% cache miss rate.
Why It Was Difficult to Diagnose
During the weeks leading up to this incident, we were actively improving our documentation rendering performance. We shipped over 20 improvements to page rendering speed, the caching infrastructure, and cache-warming logic.
Each fix appeared to work — pages would become fast again — but within 30 minutes, the monitoring job would inadvertently wipe all caches again. This created a recurring pattern in which improvements seemed to "regress" shortly after deployment, leading us to continue investigating rendering-layer issues rather than the cache invalidation path.
The bug was invisible before June 2 because our previous "stale-while-revalidate" strategy still served fast cached content to visitors even when the underlying cache was being incorrectly invalidated. The switch to hard-delete removed that safety net, making the issue immediately visible.
Resolution
On June 3, once the root cause was identified:
• Reverted the aggressive invalidation strategy — restored the "stale-while-revalidate" approach so pages would continue to be served fast from cache even during invalidation.
• Fixed the scoping bug — corrected the invalidation logic so that cache clearing for one site only affects that specific site, not all sites on the platform.
• Backfilled all caches — ran a full cache warming pass across all customer documentation sites to immediately restore fast page loads.
All documentation sites recovered and have remained fast since the fix was deployed.
Positive Outcomes
While investigating this issue, we made substantial improvements to our documentation rendering infrastructure:
• Faster cold-start renders: Even when pages are uncached, they now render significantly faster than before.
• Improved cache warming: Pages are now warmed asynchronously after publication, reducing the window during which visitors might hit an uncached page.
• Better caching architecture: Added an additional immutable caching layer for content serialization, reducing redundant processing.
• Removed performance bottlenecks: Eliminated slow dependencies from the critical render path.
Preventive Measures
Action
Status
Fixed the cache invalidation scoping bug
In Progress: 6/8/26
Restored safe invalidation strategy
Complete
Backfilled all customer site caches
Complete
Adding automated cache-hit-rate verification across customer sites
Complete
We sincerely apologize for the impact to your documentation sites. The performance improvements we shipped during the investigation period mean that your docs are now faster than before this incident began, and the fixes we've put in place prevent this class of issue from recurring.
Resolved
Page load speeds and CDN cache hit rates have returned to normal (well, actually better).
Monitoring
We have applied a fix to our cache invalidation and fill strategy. We are backfilling the cache and are monitoring load times across sites.
Identified
We identified that first-time page loads were experiencing slower than usual response times. Once cached, subsequent visits returned nominal load times.
Working with the Vercel Enterprise team, we've applied a configuration change to our caching behavior. Early signal shows an increase in cache hit rates for returning visitors, which should reduce the frequency of slow initial loads over time.
We are continuing to monitor cache performance and page load times. No action is required from customers at this time.
Identified
We've deployed a fix to optimize our caching layer and are actively monitoring the rollout. Early indicators show improved cache hit rates across affected sites.
Investigating
We identified the root cause of elevated page load times: our caching layer is performing too many sequential cache lookups per page render, and cache invalidation is causing temporary cold-cache conditions across affected sites. We are actively working with Vercel Enterprise Support to optimize our caching strategy and reduce cache-miss penalties.
Intermittent slowness is expected until improvements are rolled out.
Investigating
We are currently investigating reports of elevated page load times affecting Fern Docs sites for a subset of customers. Certain sections may be slower than expected to render.