A performance indicator or key performance indicator (KPI) is a type of performance measurement. KPIs evaluate the success of an organisation or of a particular activity in which it engages. Key Performance Indicators are the metrics that help to measure how well an organisation is performing in achieving the objectives of the critical initiative that they started. The following article will cover how the concepts of KPIs can be adopted to help evaluate the effectiveness of an organisation’s Threat Intelligence program.
When an organisation starts a Threat Intelligence program, they have some objectives in mind that they want to achieve. Every organisation wants to get maximum value out of their Threat Intelligence program as they are spending so much effort and resources on simply deriving the program. Every successful security program requires key performance indicators to report back to the organisation.
Key Performance Indicators and Metrics
While defining the metrics for key performance indicators, one should be very careful of falling in love with the wrong metrics. This invariably creates high levels of bias within the Threat Intel Program and can cause great harm to your organisation. Some Metrics look very catchy while looking at them, but they provide little value. In worst case scenarios they don’t provide any value but are bad in some way or the other. For instance they may not take into account modern threat vectors or insider threats at all presenting no indicators of a threat.
Conversely, merely producing a high number of certain indicators and attributes is not considered to define a good metrics stable because when defining a Threat Intelligence program, quality matters over quantity. A company should have a mechanism to filter out feeds for the indicators that they are certain provide accurate data as to the reality faced by the organisation.
A good Threat Intelligence program should produce an absolute minimum of false positives. Most organisations, while defining a Threat Intelligence program, develop some whitelisting services that help them filter feed for indicators that produce false positives. Apart from whitelisting services that are based on heuristics, some pre-compiled whitelisted indicators help organisations remove false positives. These whitelists can be generic as well as specific to specialised functions of the organisation.
Threat Intelligence is usually meant to be distributed to different stakeholders like Security Operations Center (SOC) analysts, incident responders, and forensic specialists. What follows are some of the KPIs necessary to provide value to SOC analysts within the Threat Intelligence program.
A SOC Analysts have the difficult job of monitoring alerts during long shifts, often when the rest of the world is asleep. A well derived Threat Intelligence program is an asset to SOC Analysts. In the case of bulk alerts, logically a Threat Analyst needs to respond to high-priority alerts first, a policy known as triaging. A high-priority alert could be the one that is imposing more threats to an organisation. How could an analyst know about the alerts that he needs to respond to first? Put differently how does the analyst determine what is high-priority amongst the noise of alerts?
This is where Threat Intel comes in. A score attribute can be associated with each indicator that is coming from a source. Some heuristics to provide a score to an indicator are following:
- An indicator that is coming from a highly reputed source can be given a score based on the reputation of the source. Those with higher reputations can be trusted more than others but never trusted completely.
- An indicator that pops up in multiple feed sources can be given a high score that indicates it may be a threat requiring investigation.
- An indicator with malicious enriched context can be given a score that fits the level of malicious content discovered.
Timely response to Alerts:
An important part of a SOC analyst’s job is to respond to an alert in as little time as possible. This can only be possible if Threat Intelligence provides enough enriched context about an indicator to warrant the analyst’s attention. The analyst should have enough information about an indicator that can help them provide a verdict as to the safety or danger of an alert.
One can provide enriched context about indicators by integrating tools designed to provide enriching information like the source of an alert while collecting indicators. Let’s take the example of “who is” information. While collecting domain indicators, enriching “who is” information can help analysts in a variety of scenarios. “Who is” information can help analyst provide quick verdict about domain indicator. For example, if a domain is newly registered then there are chances it is being registered by hacker for malicious purpose. Alternatively, using a quick online search can reveal the domain to be linked to a known threat group infrastructure allowing the analyst to quickly ascertain the threat level.
Attributions are a key part of the Threat Intelligence program. Attributions can help prioritise feeds for different organisations, even at the country and state levels. While collecting threat intelligence, if somehow, we can attribute indicators to countries or states from where they are originating and which areas they are targeting, we can prioritise that Feed for that area. Data is of no use for someone, until the data is given meaning by an indicator. Similarly, if an indicator is attributed to a malware type it can help prioritise feeds on an industrial level, that being the level at which the malware is known to target and operate within. For example, if we can mark an IP address as being associated with the Mirai botnet, then the IoT industry is a relevant stakeholder for that indicator. Likewise, an Emotet feed can be prioritised for the Banking industry.
Feed Source Reputation:
It is very important to have some metrics to define the reputation of the feed sources. It can help a lot in improving quality over quantity. Following are some of the Metrics that someone needs to track to get maximum value out of the Threat Intelligence Program.
- Number of unique indicators provided by a source
- Percentage of unique indicators provided by source
- Average Indicator of Compromise (IoC) age in a feed
- The sources that have some mechanism to age IoC holds a high reputation
- Number of false positives reported
Defining metrics for key performance indicators is necessary for every initiative an organisation takes, security initiatives included. That is the only way for an organisation to track progress in real-time. Key performance indicators should be defined after considering all the necessary metrics that will help any initiative by the company to add value for customers. In the case of Threat Intelligence merely counting indicator volume is not a good practice, rather one should consider the metrics mentioned holistically.
In the following LMNTRIX XDR demo you can see how we aggregate over 300 threat feeds and apply context, prioritisation and a risk scoring to allow analysts the ability to gain intelligence about any IOC from anywhere during their investigations.