Event counts off (low) by 10%
Incident Report for PostHog
We have finished recovering all missing events and have resolved the underlying issue. We apologize for any inconvenience this may have caused.
Posted Jun 29, 2024 - 05:42 UTC
Event recovering is still in progress. Due to the volume of data being handled, we expect the process to finish between today and tomorrow EOD.
Posted Jun 27, 2024 - 13:30 UTC
We have identified the partitions that were not consumed correctly into CH and have sequestered them for re-ingestion. The vast majority of missing events will be restored by EOD tomorrow.

The issue initially started on Thursday ~16:00Z when we brought a new CH replica online. We've removed that replica from the Kafka consumer group and all data is now being consumed successfully.
Posted Jun 25, 2024 - 20:04 UTC
There may be fewer events reported than actual for the last few days. We are investigating the root cause. We have already pushed a fix that has fixed the issue going forward, but are working to identify impacted events in the past.
Posted Jun 25, 2024 - 16:40 UTC