we're monitoring the systems after fixes were applied and they are stable.
We're processing the delayed data which should be completed soon.
Web app and feature flags are fully operational again for a while now.
We update this once the delayed data is fully processed.
Posted Jun 25, 2025 - 13:59 UTC
Monitoring
We applied some more fixes and seeing signs of recovery. We're monitoring the situation.
We still need some time to crunch through the data. We will send an update.
No data has been lost in all of this.
Posted Jun 25, 2025 - 12:09 UTC
Identified
We identified the cause for the increased latency. The error rates for feature flags have decreased (from 1.5%) and the ingestion lag is coming down.
Posted Jun 25, 2025 - 11:49 UTC
Update
We're investigating increased error rates in feature flags and an increase in latency in the ingestion pipeline.
There is no data loss, but we're currently processing events with a delay.
Posted Jun 25, 2025 - 09:30 UTC
Update
We're still investigating but also discovered that the web app is showing increased latency.
Posted Jun 25, 2025 - 08:27 UTC
Investigating
We've spotted elevated error rates in feature flags. We're currently investigating the issue, and will provide an update soon.
Posted Jun 25, 2025 - 07:24 UTC
This incident affected: EU Cloud 🇪🇺 (App, Event and Data Ingestion Lag, Feature Flags and Experiments) and US Cloud 🇺🇸 (Event and Data Ingestion Lag).