Empty Lookalike Segments

Incident Report for Piano Software

Postmortem

RCA report for the incident can be found at this link.

Posted Apr 11, 2024 - 11:25 UTC

Resolved

On March 18th at 10:44 UTC, a change was pushed to increase the flexibility of lookalike segments. This resulted in empty lists of lookalikes being stored in the CX Platform without any associated errors. The root cause was identified March 20th at 13:00 UTC, and an initial fix was deployed at 13:25 UTC. Lookalike modeling jobs were then scheduled to run again.

Fixes for associated errors with small sociodemographic segments were fully deployed on March 21st at 12:10 UTC. The associated jobs were prioritized on March 22 at 7:37 UTC, and the remaining issues were resolved for the majority of clients within the next two hours.

Posted Mar 18, 2024 - 10:30 UTC