Data Processing Delays - Replay Affected

Incident Report for PostHog

Resolved

We have processed our backlog and are back to realtime for session replay. We have a few followups to perform from this one to ensure that it does not happen again, but overall we are in a good state.
Posted Sep 21, 2023 - 04:16 UTC

Update

We're still catching up, we expect to be fully caught up in about 5-6 hours. It's likely that during the period where we experienced high error rates we dropped a small percentage of recordings, or some portions of recordings were corrupted. We will investigate this tomorrow.
Posted Sep 20, 2023 - 23:36 UTC

Monitoring

We're recovering from the spike in load and are continuing to monitor.
Posted Sep 20, 2023 - 22:54 UTC

Identified

Our Replay data processing infrastructure is running behind which is causing recent recordings to be unavailable. No data has been lost and the system should be caught up shortly.
Posted Sep 20, 2023 - 15:05 UTC
This incident affected: US Cloud 🇺🇸 (App, Event and Data Ingestion Lag).