Beaconing is one of the most critical signals in cybersecurity: traffic sent at regular intervals from the internal network to an infrastructure controlled by an adversary, a sign of a malware infection or an ongoing data exfiltration. The problem: this signal is drowned in massive volumes of proxy logs, impossible to detect manually. Security teams had no tool that could automatically sort through the millions of daily connections to isolate suspicious behavior and report it back to experts for verification.
Beaconing detection
A large group's security team needed to identify beaconing signals, regular network traffic sent by potentially compromised machines to servers controlled by attackers. The volume of logs made manual detection impossible. We built the anomaly detection system capable of processing this data on a large scale.
Problem
Solution
What we built
We deployed 2 Data Scientists to design a machine learning beaconing detection system, capable of processing a massive volume of network data.
Step 1 — Extraction and feature engineering. Parsing and cleaning proxy logs to extract usable features. Two levels of granularity: daily aggregations by client, host, and date, and aggregations over a historical period by host to capture the patterns of regularity characteristic of beaconing.
Step 2 — Modeling by detecting anomalies. The features were used as training data for several anomaly detection models. The unsupervised approach was necessary: by definition, beaconing cases are not labeled in historical data.
Step 3 — Realistic evaluation system. Implementation of an evaluation protocol simulating real conditions of use by the security team, to measure the relevance of alerts in an operational context.
Step 4 — Continuous improvement with experts. Iterative work with security experts to refine feature engineering and reduce the rate of false positives, so that the volume of alerts remains humanly treatable.
Projects in the same category
