Restoring control during abnormal traffic against customer services
The Challenge
A regional communications provider operates a suite of consumer self-service, billing, and enterprise service portals, all of which are supported by a combination of internal network teams, cloud services, and outsourced operational teams. These portals are essential for customer interaction and engagement, allowing users to manage their accounts, access billing information, and utilize various enterprise services seamlessly. However, the complexity of this multi-faceted support structure often leads to challenges in maintaining consistent service quality and security across all platforms.
Threat Surfaced
Recently, short bursts of abnormal traffic were detected, which adversely affected public endpoints. This surge in traffic posed a significant risk to the integrity and availability of the services being offered. While the security and network teams were engaged in their respective investigations, they operated from disparate views and lacked a unified perspective on the situation. Compounding the issue, several high-impact support accounts were found to lack consistent session ownership, leading to confusion and potential vulnerabilities that could be exploited by malicious actors.
Service-delivery
The interruptions caused by this abnormal traffic resulted in increased portal latency and intermittent time-outs, which in turn drove a higher demand for service-desk support. Although the teams involved could identify the symptoms of the problem, they struggled to share a common threshold for what constituted normal versus abnormal activity. Furthermore, there was no clear activation path for addressing the issues, nor was there a designated accountable decision owner to facilitate timely and effective responses to the growing crisis. This lack of coordination hindered the ability to resolve issues swiftly and efficiently, resulting in a frustrating experience for end-users.
OUR Approach
In response to these challenges, ONESECURE took a comprehensive approach to remedy the situation. The team began by mapping the protected services, traffic paths, normal operating ranges, and relevant provider contacts. This mapping process was crucial for understanding the landscape of the network and identifying potential vulnerabilities. Next, the agreed-upon network, cloud, and endpoint telemetry were connected to a Managed Extended Detection and Response (XDR) system. This integration allowed for real-time monitoring and analysis of the traffic patterns and security events. Additionally, priority was given to identifying privileged accounts that were critical for mitigation efforts and any necessary platform changes, ensuring that the most sensitive access points were adequately safeguarded.
Rectification & Accountability
Analysts conducted thorough validations of the affected services and gathered supporting evidence to understand the full scope of the issue. Findings were escalated through a named incident owner responsible for coordinating the response efforts. This structured escalation process ensured that only pre-authorized provider actions were taken, thereby minimizing the risk of further complications. Moreover, all privileged sessions were subjected to stringent approval processes, recording, and review requirements, which enhanced the accountability and traceability of actions taken during the incident response.
The Outcome
The engagement culminated in the development of a tested DDoS decision runbook, which serves as a critical resource for future incidents. This runbook outlines the steps to be taken in the event of similar threats, ensuring that the organization is better prepared to respond swiftly and effectively. Additionally, an inventory of in-scope services and authorized contacts was created, along with a prioritized incident queue to streamline response efforts. A meticulous, time-stamped record of each decision made during the incident was established, providing a clear audit trail. Any claims of reduced downtime now require substantiation through customer-approved incident data, reinforcing the importance of transparency and accountability in service delivery.
Solutions
The comprehensive solutions implemented include DDoS Protection, Managed XDR, and Identity & Access Governance. These measures work synergistically to enhance the overall security posture, ensuring that the communications provider can effectively defend against future threats while maintaining high service availability for their customers.
