Title
ISP with a repeatedly blacklisted IP, shared by several customers, used FLOWCUTTER to trace the cause to botnets on its own backbone devices and at a customer, even though flow volumes looked normal.
Who
Companies: ISPs offering NATed or shared public IP addresses to multiple customers.
Roles: Network administrators, abuse and security teams.
Situation
One of the ISP's public IP addresses kept losing reputation and ending up on blacklists. The address was NATed and shared by several customers, so one blacklisted IP affected all of them.
The only short-term fix was to keep rotating IPs for the affected customers. That disrupted service and did not solve anything.
Challenge
NetFlow data showed no traffic spike or volumetric anomaly on that address. Nothing in the flow volumes explained the recurring blacklisting, so there was no obvious lead to follow.
Solution
The Vulnerability Scan in FLOWCUTTER flagged something unusual: port 53 (DNS) open on 10 devices behind several addresses. The first hypothesis was a reflective DNS attack, but it did not hold up on closer inspection.
The team then checked the IP reputation lookup built into FLOWCUTTER (AbuseIPDB integration). The address had been reported for attacking WordPress websites.
Drilling into all outbound conversations on port 443, the team traced the traffic back to devices with Telnet (port 23) open to the internet, 8 devices in total. These devices were not customer CPEs. They were part of the ISP's own backbone infrastructure.
The exposed Telnet port had been used to break in and plant a script. The script turned the equipment into a botnet attacking WordPress sites over HTTPS.
The team then applied the same approach to a second, separate case. A customer-side device, believed to be an IP camera, turned out to be infected and part of a botnet as well.
Results
The root cause of the blacklisted IP was identified. Backbone devices with exposed Telnet had been compromised and joined a botnet. The botnet attacked WordPress sites over port 443, which got the shared IP blacklisted again and again.
The backbone devices were cleaned up and the reputation issue was resolved. In parallel, the infected IP camera was identified and the customer was notified.
Flow volume alone did not show either problem. It took open-port findings, IP reputation data and a flow drill-down by destination port to uncover both botnets.
Resources
- Vulnerability Scan (open ports)
- IP reputation lookup (AbuseIPDB integration)
- NetFlow drill-down by destination port
Takeaway
A normal-looking traffic graph does not mean an IP is clean. Exposed management ports such as Telnet are a risk for customer CPEs and for an ISP's own backbone equipment alike. Combining open-port findings with reputation data and flow drill-down revealed two separate botnets that traffic volume alone would never have flagged.