Serraview Detailed Root Cause Analysis – Severity 2 – January 30, 2024 

 US Regions Automated Imports Degraded Performance 

We are truly grateful for your continued support and loyalty. We value your feedback and appreciate your patience as we worked to resolve this incident.  

Description: 

On January 30, 2024, internal and external customers began to report Automated imports were not processing.  

Type of Event: 

Degraded Performance 

Services\Modules Impacted: 

Production – Automated Imports via SFTP 

Timeline: On January 30, 2024, at approximately 11:59am EST, multiple customers began to report their automated imports were not processing. Our support team immediately raised this issue with our internal teams and notified all customers that we are treating this situation as an Severy 2 via Status Page. At 6:49pm EST, internal teams had identified and implemented the fix. The status page was updated from investigating to monitoring. As customers began to confirm resolution, monitoring was updated to a resolved status at 10:11pm EST 

Root Cause Analysis: 

Imported files were not getting through the parallel mode, which is an effective way to process the way we receive Automated reports.  

Remediation: 

To fix this situation, internal teams changed the automated import from parallel to sequential mode, so that the stacked files could get imported first. This is done to get through the backlog of imported files. Once this process is completed, the team has switched back to parallel mode.  

Preventative Action:  

There is ongoing investigation, to enhance monitoring for this specific service by engineering and cloud ops teams.