The Problem with Threat Hunting Metrics
Threat hunting is a massive commitment of time, resources, team members, and technology. Any investment that impactful would normally be one that was carefully measured to ensure it was driving sufficient value for the team. The thing is, there’s no established benchmark of “success” in threat hunting. Measuring the ROI of a cyber threat hunting program can be challenging because it often involves qualitative benefits and preventative measures, which are hard to quantify in monetary terms.
No hits can mean that your environment is secure – OR threats are present, but other variables are at play. A lack of hunt findings can indicate that hunters are simply too inexperienced to find the sophisticated threats, they don’t have access to the right information, and a number of other possibilities. On the other hand, identifying many hits confirms that you’ve caught malicious actors in your network…but that can also mean that the network is vulnerable and your security strategy needs to be adjusted.
As a result, security teams struggle to measure ROI – threat hunting just isn’t that black and white. There are other key indicators that threat hunters and their leadership can look to when trying to understand the success of their program, AND of individual hunts.
Building a Strong Threat Hunting Foundation
- Do I have the right hypothesis?
- Is the threat I’m looking for plausible?
- Do I have the right behaviors?
- Am I looking for the right things?
- Am I looking for everything that could possibly indicate the presence of this threat?
- Do I have the right data?
- Am I looking in the right places?
- Am I looking at the right logs, systems, and tools in my tech stack?
- Are we collecting the necessary data?
- Have we kept data long enough to find something? Are we cleaning house too frequently?
- Was I thorough enough in my analysis to confidently say “it’s not here”?
Measuring Threat Hunting Success: Key Metrics
1. Measuring Success of the Threat Hunting Program
Analysis of the threat hunting program can be separated into two categories: the activity (hunts) and results (hits).
Activity metrics refer to the frequency of threat hunts, and they’re valuable for organizations to understand the amount of time, effort, and resources going into their threat hunting program. The second key component of their program, the result metrics, refer to what the hunters are able to find. Coupled together, teams can contextualize and understand the efficiency of the hunts they’re conducting.
Activity Metrics
The frequency of hunts might be limited due to challenges like:
- They don’t have enough skilled personnel present, or skilled personnel are busy due to resource constraints
- They have an unfocused security strategy with a reliance on headlines and emergent threats (instead of the ones that are most likely to impact the organization)
- The organization lacks threat prioritization and doesn’t know where to start
- Threat research takes a significant amount of time
- They’re operating with a siloed, disconnected tech stack that blocks hunters from finding the data they need
Ideally, threat hunting activities complement their detection strategy and fill in the gaps where necessary.
Result Metrics
Breadth / Relevance of Threats: When measuring threat hunting results, hunters need to know that the threats they’re investigating are the ones that actually matter most to their organization. Just because they’re hunting many different threats, that doesn’t mean they’re being thorough – at least, not in the right way. They need to measure the breadth of threats hunted against their threat profile (or another tool used to identify the threats that are most relevant and impactful to their organization).
When assessing the breadth and relevancy of threats at hand, hunters should ask themselves:
- Am I hunting the threats that matter?
- How much of my threat profile have I hunted?
- Do we have the data? Can we hunt for these threats?
- Do we know the behaviors and indicators?
Therefore, when analyzing the success of the threat hunting program, teams must understand how well their detection methods and alerts are really working. Through detection validation, security teams can test and tune their rules as needed to ensure they’ll fire when it matters most. The core question here is, “Of everything we’re hunting, how much are we actually alerting on?” In some circumstances, this can actually help take some of the work off the hunter’s plate; the more confident they are in their detections, the less they’ll need to include in their hunt program.
Mean-Time-to-Detect: Mean-time-to-detect is a key metric not just for threat hunters, but for the SOC as a whole to understand the effectiveness of their detection strategy. It’s a measurement of how long certain threats were present in the network before the team identified them. A related measurement is dwell time – the amount of time a threat actor is present before the threat is remediated. Though slightly different, they’re both metrics that indicate how long adversaries can go undiscovered – and undisturbed – in an organization’s system.
Incident Reduction and Cost Avoidance: When the average cost of a security incident is $4.45 million, it’s no question that a key driver behind threat hunting is to prevent incidents (and the financial fallout that accompanies them). If a threat hunt can be linked to preventing a potential incident, teams can determine their savings, and therefore the ROI of their threat hunting program.
Coverage: “Coverage” might sound like a vague, hard-to-quantify metric in the realm of threat hunting. The threat landscape is so vast, how can threat hunters fully understand the breadth of their coverage? Essentially, threat hunters need to know that their approach is comprehensive through the lens of the threats that are relevant to them.
2. Measuring Success of Individual Threat Hunt Outcomes
Hunt Duration: Organizations should measure the time taken to complete each threat hunt. Because there is no industry-wide framework for threat hunting, security teams might not know whether their hunts are efficient.
Hit Rate: Perhaps the most obvious metric that hunters should keep track of is the hit rate – the proportion of hunts that identify threats. Less obvious, however, is how they’re supposed to interpret that number.
Measuring Threat Hunting Success
Whether you’re building a threat hunting program from the ground up, trying to make your current approach more effective, or just trying to determine where you stand, SnapAttack’s platform and threat hunting maturity assessments can get you to the next level. Get in touch today.
About SnapAttack: SnapAttack is an innovator in proactive, threat-informed security solutions. The SnapAttack platform helps organizations answer their most pressing question: “Are we protected against the threats that matter?”
By rolling threat intelligence, adversary emulation, detection engineering, threat hunting, and purple teaming into a single, easy-to-use product with a no-code interface, SnapAttack enables companies to get more from their tools and more from their teams so they can finally stay ahead of the threat.