Resolved -
We've spotted that something has gone wrong. We're currently investigating the issue, and will provide an update soon.
May 30, 17:23 UTC
Update -
A small portion of the traffic is still experiencing ingestion delay, so we'll keep this status page open until we're full caught up.
May 30, 15:38 UTC
Update -
We processed all of the events we stashed away this morning, all analytics events should now be processed with nominal latency.
We will monitor the situation for a bit longer before closing this incident.
May 30, 13:15 UTC
Monitoring -
Events submitted in the last 30 minutes should be ingested without delay or duplication, some events from earlier have been stashed in a separate topic and are slowly being backfilled.
May 30, 11:45 UTC
Identified -
We detected a significant amount of duplicated events going to ClickHouse. While these events will be deduplicated, they affect insight quality on recent events.
Mitigation work is ongoing, and as part of it are currently delaying part of the traffic. No data is lost.
May 30, 10:38 UTC