How undetected DDoS cut town from internet

Outgoing DDoS stopped before making havoc by saturating uplink for the whole network segment

Companies: Any network operator such as provider of internet, communication services, web hosting, etc.

Roles: Network administrator

Use case: Beneš @ DobruskaNet

Network administrator was notified by an alert that one particular radio uplink becomes saturated in the middle of the day. This network segment represented a remote town. Consequently all homes and institutions in there experienced severe connection problems.

SNMP data showed the saturation of uplink in regular intervals. But nothing apparent was going on. Behind this link ISP didn’t have detailed information per customer, hence, the network administrator was able to find out whether there is one particular endpoint responsible for the upload.

What to do?

Network administrator analyzed anomaly using netflow data – traffic telemetry.

• ISP had netflow export on the perimeter 
• Netflow was stored in the central collector with FLOWCUTTER software.
• With FLOWCUTTER, any admin can easily do a fast drill-down analysis of netflow and other data sources.
• In addition, FLOWCUTTER allows to set up “out of the box” detection of various volumetric DDoS attacks

That helped identify the nature of the DDoS attack and revealed information used to effectively mitigate it.

The traffic analysis in FLOWCUTTER showed the nature of the anomaly. By plotting Top N source ports by number of flows, the administrator correctly identified that responses to DNS queries were contributing most to the uplink saturation. Incoming DNS packets stayed hidden in the overall volume of traffic, but the outgoing DNS responses were clearly visible.

Focusing on just DNS responses meant easy operation (filtering src.port=53 only) that took only 2 seconds. This is where FLOWCUTTER excels above all competitive solutions.

The next step was to compare behavior before and during the anomaly. It showed that normally there were 1.000 talkers, but suddenly raised to 23.000 corresponding to the distributed nature of the attack.

Even though the administrator wasn’t able to find a particular end customer responsible, because there were many customers hidden behind each public IPv4 address, he identified this reflexive DDoS using DNS. 

Moreover, analysis revealed information (combination of Src/Dest. ports = 53/24335) that he could use to mitigate the attack using BGP FlowSpec (RFC 5575). Use of FlowSpec is similar to BGP Remotely trigger black hole, but it provides more granular control (ports, packet length, TCP flags, ICMP code, …) and shape these flows. 

• Netflow analysis in Grafana
• Open ports scan
• SNMP vs Flow telemetry

ISP detected uplink saturation.

1. Within netflow, the administrator find root cause in outgoing reflexive DDoS attack using DNS.
2. He was able to mitigate attack using BGP FlowSpec

For the future, ISP used FLOWCUTTER’s ability to monitor and alert on traffic anomalies – admin can set up “out of the box” detection of various volumetric DDoS attacks. And next time be alerted even faster.