Category Archives: Security Monitoring

Extract and use Indicators of Compromise from Security Reports

apt-reports-1Every now and then there is a new security report released and with it comes a a wealth  of information about different threat actors offering insight about the attacker’s tactics, techniques and procedures. For example, in this article I wrote back in 2014, you have a short summary about some of the reports that were released publicly throughout the year. These reports allow the security companies to advertise their capabilities but on the other hand they are a great resource for network defenders. Some reports are more structured than others and they might contain different technical data. Nonetheless, almost all of them have IOC’s (Indicators of compromise).

For those who never heard about indicators of compromise let me give a brief summary. IOC’s are pieces of information that can be used to search and identify compromised systems. These pieces of information have been around since ages but the security industry is now using them in a more structural and consistent fashion. All types of enterprises are moving from the traditional way of handling security incidents – Wait for an alert to come in and then respond to it – to a more proactive approach which consists in taking the necessary steps to hunt for evil in order to defend their networks. In this new strategy, the IOCs are a key technical component. When someone compromises a system, they leave evidence behind.  That evidence, artifact or remnant piece of information left by an attacker can be used to identify the intrusion, threat group or the malicious actor. Examples of IOCs are IP addresses, domain names, URLs, email addresses, file hashes, HTTP user agents, registry keys, a service configuration change, a file is deleted, etc. With this information one could sweep the network/endpoints and look for indicators that the system might have been compromised. For more background about it you can read Lenny Zeltzer summary. Will Gragido from RSA explained it well in is 3 parts blog herehere and here. Mandiant also has this and this great articles about it.

So, if almost all the reports published contain IOC’s, is there a central place that contains all reports that were released by the different security organizations? Kiran Bandla created a repository on GitHub named APTnotes where he does exactly that. He maintains a list of all security reports released by the different vendors – “APTnotes is a repository of publicly-available papers and blogs (sorted by year) related to malicious campaigns/activity/software that have been associated with vendor-defined APT (Advanced Persistent Threat) groups and/or tool-sets.” Kiran, among other methods relies on the community who share with him when a report X or Y was released and he adds it to the APTnotes repository. The list of reports is available in CSV and JSON format.

So, what can you do with all this reports? How can we as network defenders use this information?

Three example use cases.  The first use case is more likely and common, where the other two might be more common in larger organizations with higher budget and bigger security appetite.

  • An organization that has a Security Operations Center in place could use the IOC’s to augment their existing monitoring and detection capabilities.
  • An organization that has in-house CSIRT capabilities could leverage the IOC’s in a proactive manner in order to have a higher probability of discovering something bad and, as such, reduce the business impact that a security incident might have in the organization.
  • An organization that has a Cyber Threat Intelligence capability in-house, could collect, process, exploit and analyze these reports. Then disseminate actionable information to the threat intelligence consumers throughout the organization.

In a simple manner, the process for the first scenario would look something like the following diagram:


How would this work in practical terms? Normally, you could split the IOC’s in host based or network based. For example, a DNS name or IP addresses will be more effective to search across your network infrastructure. However, a Registry Key or a MD5 will be more likely to be searched across the endpoint.

For this article, I will focus on MD5’s. Some reports offer file hashes using SHA-1 or SHA-256 but not many organizations have the capability to search for this. MD5 is more common. Noteworthy that the value of MD5 hashes about a malicious file might be considered low.  A great article about the value of IOC’s and TTP’s was written back in 2014 from David Bianco title The Pyramid of Pain. Following that there is an article from Harlan Carvey with additional thoughts about it. Another point to take into consideration about the MD5’s from the reports is that some might be from legitimate files due to the usage of DLL hijacking or they are windows built-in commands used by threat actors. In addition is likely that malware used by the different threat actors is only used once and you might not see a MD5 hash a second time. Nonetheless, the MD5’s is a starting point.

So, how can we collect the reports, extract the IOC’s and convert them and use them?

First you can use a python script to download all the reports in a central place separated per year. Then you can use the tool IOC parser written by Armin Buescher. This tool will be able to parse PDF reports and extract IOC’s into CSV format. From here you can extract the relevant IOC’s. For this example, I want to extract the MD5’s and then use IOC writer to create IOC’s in OpenIOC 1.0/1.1  format which could be used with a tool such as Redline.  IOC writer is a python library written by William Gibb that allows you to manipulate IOC’s in OpenIOC 1.1 and 1.0 format and was released on BlackHat 2013.

The steps necessary to perform this are illustrated below – for sure there are  other better ways to perform this but this was a quick way to do the job -.


With this we created a series of files with  .ioc extension that can be further edited with the ioc_writer Python library. However, not many tools support OpenIOC, so you might just use the MD5’s and feed that into whatever tool or format you are using. You can download below the excel with all the MD5’s separated per year.  If you use them, please bare in mind you might have false positives and you will need to go back to the csv files to understand from which report did the MD5 came from. For example I  already removed half dozen of MD5’s that are legitimate files but seen in the reports, and also the MD5 for an empty file “d41d8cd98f00b204e9800998ecf8427e”

That’t it, with this you can create a custom IOC set that contain MD5’s of different tools, malware families and files that was compiled by extracting the MD5’s from the public reports about targeted attacks. From here you can start building more complex IOC’s with different artifacts based on a specific report or threat actor. Maybe you get lucky and you find evil on your network!

The year 2010 contains 81 unique MD5’s.
The year 2011 contains 96 unique MD5’s.
The year 2012 contains 718 unique MD5’s.
The year 2013 contains 2149 unique MD5’s
The year 2014 contains 1306 unique MD5’s
The year 2015 contains 1553 unique MD5’s
The year 2016 contains 1173 unique MD5’s

Excel : md5-aptnotes-2010-2016


Neutrino Exploit Kit

[First of all Happy New Year to all the readers!
In this summary I would like to briefly describe one component that is part of many parts which form the Internet malware industrial complex. The Exploit Kits (EK’s) which impact users and corporations of all sizes on a daily basis. The
 article below was possible due to the insights and knowledge received from @Kafeine. ~LR]

The EK’s are powerful and modular weapons that deliver malware in an automated fashion to the endpoint by taking advantage of client side vulnerabilities.

Exploit Kits are not new and have been around at least for the past 10 years or so. Nonetheless, they have evolved and are now more sophisticated than ever. The malware authors behind them enforce sophisticated capabilities that evade detection, thwart analysis and deliver reliable exploits. Basically, by introducing malicious code in a web server an attacker can turn a legitimate web server into a mechanism to deliver malicious code by taking advantaged of client-side vulnerabilities against unpatched browsers and applications. This attack vector is known as watering hole or strategic web compromise when it targets a trustworthy web site. In recent years the Exploit Kits have evolved and became very sophisticated weapons and profitable business for the ones involved. The malware authors continue to develop sophisticated capabilities to prevent detection, analysis and deploy exploits for new vulnerabilities in a very reliable manner.

In the last days I had the chance to look at one recent campaign of drive by download that leverages the Neutrino Exploit Kit to infect systems with CryptoWall. The diagram below illustrates the different components of the Neutrino Exploit Kit and how they work together.



  • User browses to the compromised web server.
  • Web server contacts the backend infrastructure in order perform various check and to generate malicious java script code. These checks include things like verification of victim IP address and its Geo-location. Furthermore within the malicious JavaScript code there are new domain names and URLs that are generated dynamically by the backend.
  • The browser processes and decodes the malicious JS. In the observed infection the malicious JavaScript checks the browser version and if it matches the desired version, it stores a cookie and processes a HTML iframe tag.
  • The iframe tag triggers the browser to perform a request to another URL which is the Neutrino Exploit Kit landing page.
  • The landing page is hosted in a randomly generated host using DGA which needs to be resolved via DNS. The authoritative domain to answer these domains are owned by the threat actor. The answers received by the DNS server have a time to live (TTL) of a few seconds. The domains are registered on freely available country code top level domains (ccTLD).
  • The victim then lands in the exploit kit landing page which by its turn delivers a small HTML page with an object tag defined in its body. This object tag directs the browser to load Adobe Flash Player and then use it to play the SWF file specified in the URL. In case the victim does not have Adobe Flash player installed, the browser is instructed to download it.
  • The browser as instructed by the object tag, downloads the malicious Flash file.
  • The obfuscated and encrypted SWF file is played by the Flash Player and exploits are triggered based on available vulnerabilities. The Flash file contains exploits for CVE-2013-2551, CVE-2014-6332, CVE-2015-2419 affecting Internet Explorer and CVE-2014-0569, CVE-2015-7645 affecting Adobe Flash.
  • If the exploitation is successful, shellcode is executed and the malware is downloaded and launched. In this case we observed that the malware delivered has been CryptoWall.

The threat actors behind Neutrino are finding vulnerable websites in order to host their malicious JS  content globally in a repeatable and automated fashion. Furthermore, In the last few days Neutrino has been abusing the registration of free domains registered inside the country code top level domains (ccTLD) such as  .top, .pw, .xyz, .ml, .space and others. The different landing pages have been pointing to a server hosted in Germany and in another cases in Netherlands. In another blog post I will go into more details about it.


References: kit.pdf

Tagged , , , , , ,

Intro to cyber threat intelligence

knowyourenemyThe traditional security monitoring and incident response (IR) capability that has being used across the enterprises in the last decade has fallen behind. It is consensus across the IT security industry that we need a more robust, capable and efficient security monitoring and IR framework. The new framework should enable us to combine security and intelligence functions. An intelligence driven security that allows us to plan for, manage, detect and respond to all categories of threats even as they become more frequent and severe. In other words we want to maximize the organization effectiveness and efficiency to block, detect and respond to attacks. How? By introducing into the traditional security stack the threat intelligence security function we can do more and better.

Following the last post about about what intelligence means and what is the 5 steps of the intelligence cycle below an introduction to  Cyber Threat Intelligence topic.  A quick summary on what is threat intelligence, what is its value and what are the sources to consume or produce intel. More about this topic will follow in future posts.

What is Cyber Threat Intelligence?
Threat intelligence is a recent paradigm in the IT security field that continues to gain a lot of traction due to a change of focus in the risk equation from the vulnerability into the threat. Tracking threats that are specific to your industry, organization or region is key to minimize damage that can caused by an attack.

On the one hand we have strategic threat intelligence. A capability that needs processes, tools and people to leverage an understanding about the attacker’s capabilities and intents. Is normally delivered through reports that are produced by humans and consumed by humans and is the most expensive and hardest to produce. It produces information to support well informed decisions of long-lasting importance such as which policies and processes should change. Or what new changes one should accommodate in the security infrastructure to adapt to the new threat landscape.From a well-established and mature strategic threat intelligence practice you should be able to get answers to questions like: Who is your potential adversary? What is the adversary’s capability to cause you harm? Do they have the intent to cause harm? Where are you vulnerable? How could anyone harm your organization if they wanted to do so?

On the other hand, we have tactical threat intelligence. A capability that aids the prevention, detection and response competencies with real time threat data that is consumed across different systems and functions. Data such as IP addresses, domain names, URLs, email addresses, hashes values, HTTP user agents, registry keys, etc. Remnant pieces of information left by an attacker that can be used to identify threats or malicious actors. These pieces of information are nowadays called indicators of compromise and can, for example, be used to search and identify compromised systems.  This thread data is tactical threat intelligence and is of limited life span. Tactical threat intelligence should be disseminated, integrated and consumed in an automated fashion.  This type of threat intelligence is the cheapest and easiest to create.

What is the value of Cyber Threat Intelligence?
At the strategic level, the value proposition of threat intelligence might include:

  • Make well informed decisions on where you are spending your security dollars.
  • Create comprehensive insight about the threats by developing facts, findings and forecasts about threat actor’s capabilities, motives and tradecraft.
  • Create recommended courses of action on how to adapt to the evolving threat landscape in order to reduce and mitigate risks.
  • Being able to plan for, manage and respond to all categories of threats – even as they become more frequent and more severe.
  • Develop situational awareness about capabilities and intents of your adversaries.
  • Know your adversary and what are they looking for.

At the tactical level, the value proposition of threat intelligence might include:

  • Minimize the risk of attacks that could result in lost revenue, public embarrassment, and regulatory penalties.
  • Improve the effectiveness and efficiency of security monitoring capabilities by integrating and matching threat intel data.
  • Augment security operations and incident response functions with actionable threat data.
  • Reduce the number false positives by adding threat intel data into security operations.
  • Accelerate Incident Response actions and remediation priorities based on targeted information.

What are the sources of Cyber Threat Intelligence?
The sources might vary depending if you are a consumer or a producer of threat intelligence. From a consumer perspective – where the majority of the organizations fit in – they mainly fall into two categories. The open source ones that are free and can be retrieved by anyone. And the closed sources that are commercial or with restricted access. These ones often need a payed subscriptions or being member of a closed circle of trust. Either one, they fall under tactical threat intel when data is delivered to the consumer trough feeds with indicators of compromise. Or they fall under strategic threat intel when the deliverables is a report about capabilities and intents of malicious actors.

From a producer perspective the sources are even broader and using different disciplines. Normally, if you are a service provider there is the incentive to produce it using the most variety of sources, methods and disciplines. Mainly due to the fact service providers do it for a profit. For example, iSight Partners, Dell SecureWorks, Mandiant or CrowdStrike are good examples of service providers that create strategic and tactical threat intelligence combined together. They have dedicated teams of researches that perform all kinds of activities, some of which might be almost considered under intel agencies or law enforcement umbrella. Examples of sources used across producers are honeypots and spam traps that are used to gather intelligence trough the knowledge and information that one could obtain by observing, analyzing and investigating the attacker that are lured to it. Another source could be the output of doing static and dynamic malware analysis.


How to Collect, Refine, Utilize and Create Threat Intelligence by Anton Chuvakin
Security Science by Clifton Smith; David J Brooks
Intelligence-Based Security in Private Industry by Thomas A. Trier

Tagged , ,

The 5 steps of the Intelligence cycle

intelligencecycleBack in 2011, market research companies like IDC, Forrester and Frost & Sullivan were making market analysis about the growth of cyber threat intelligence services and alike. Their analysis stated a double digit growth year of year. Their projections seem reasonable and their current estimations continue in this trajectory.  Nowadays, cyber threat intelligence continues to gain a lot of traction and hype across IT security. However, as many other cases in the IT security, the industry is adopting the jargon used across government agencies and military forces. That being said I wanted to write about cyber threat intelligence. But I thought would be good to first read and understand what intelligence means across the intelligence agencies and military domains in order to have good foundation before applying it to cyber. Below short summary I made on what intelligence is and what the 5 steps of the intelligence are.

What is Intelligence?

Intelligence is the product that results from a set of actions that are performed to information.  Traditionally, used across governmental organizations for the purpose of national security.  The actions are collect, analyze, integrate, interpret and disseminate. The final product of intelligence gives value-added, tailed information that provides an organization or its adversary, the ability to make conclusions. For the enterprise the information product might be to seek information about the threat actors means, motive and capabilities. On the other hand the adversary might want to seek information about intellectual property (patents, copyrights, trademarks, trade secrets, etc) from your company in order to gain economical advantage or to subvert its interests. In any of the cases the information produced gives an edge, a competitive advantage to you or to your adversary.

The information produced contains facts, findings and forecasts that supports yours or the adversary goals.  There are two categories of Intelligence. One is strategic and the other is operational. Strategic intelligence means information produced to support well informed decisions of long-lasting importance. Strategic intelligence is broader and often requires information concerning different fields.  Operational intelligence is of limited life span and it to be used rapidly and is concerned with current events and capability.

What are the 5 steps of the Intelligence cycle?

Planning and direction – This is the first step. It’s here were the requirements and priorities are set. The capabilities to produce Intel are limited as any other resource which means we want to maximize its production with a constant number of resources.  Among others, a methodology to define the requirements might be using the “Five W’s”. It’s also in this step where we define which areas the intelligence produced will have the most impact and make to most contribution.  During the planning is fundamental to specify which categories of Intelligence will be gathered i.e. OSINT (Open Source Intelligence). In addition, the processes, people and technology to support the different steps in the cycle need to be established with clear roles and responsibilities.

Collection – The second step includes all the different activities, mainly research, that involves the collection of data to satisfy the requirements that were defined. The collection can be done either via technical or human means and involves gathering data from a variety of sources.  In the military and intelligence community the sources normally used are people, objects, emanations, records. These sources span the different collection disciplines named as HUMINT, IMINT, MASINT, SIGNT, OSINT and others. Once collected, information is correlated and forwarded for processing and production.

Processing and exploitation – Third step, the collected raw data starts to be interpreted, translated and converted into a form suitable for the consumers of the intelligence. The raw data becomes information.

Analysis and production – The refinement of the information that was produced in the previous step.  The fusion of the different information that was processed from the different intelligence disciplines. These are key tasks performed during this step. The analysis  consists of facts, findings and forecasts that describe the element of study and allow the estimation and anticipation of events and outcomes. The analysis should be objective, timely, and most importantly accurate.  To produce intelligence objectively, the analysts apply four basic types of reasoning. Induction, deduction, abduction and the scientific method. Furthermore, because bias and misperceptions can influence the analysis the analyst should be aware of the different analytical pitfalls. The outcome is value-added actionable information tailored to a specific need. For example,  in the United States, creating finished intelligence for national and military purposes is the role of the CIA.

Dissemination and Integration – Essentially, this step consists in delivering the finished product to the consumers who requested the information. This can be done using a wide range of formats and in a manual or automated manner.

JP 2-0, Joint Intelligence
Operations Security – Intelligence Threat Handbook
USAF Intelligence Targeting Guide
Intelligence Essentials for Everyone


Tagged , , , ,

Thoughts on Measuring Security Monitoring

Measuring-StickWith increasing frequency on security incidents and greater than ever focus on the security posture of organizations, you as a manager or as an engineer, are being called to be able to measure the impact of your security controls. In other words, there is a high demand for being able to report, measure and provide situational awareness about your people, process and technologies that are related to the security business. Essentially, being able to measure something that would produce actionable insights that will result in more effective and efficient services.

That being written, below some random thoughts about this topic. My background is engineering, not economy or business management but is common sense that in any operational business metrics are key in order to improve management and delivery of the services. Security should not be an exception. However, if you consider your organization end-to-end security stack (e.g. from the endpoint AV and operating system patches to the DDoS detection and mitigation controls) there is a huge variety of data sources that don’t have a common interface. This means whenever you want to measure something it will be manually and labor intensive i.e., the metrics are not cheap to gather. Now, if you put on top of that the people and processes aspect it will be even harder to measure.

Anyway, you also don’t want to measure just something just because you can. Metrics should support your mission and help you having a clear picture on how well are you performing. In addition as stated by Andrew Jaquity on his book “Security Metrics – Replacing Fear, Uncertainty, and Doubt”, good metrics should be consistently measured, gathered in an automated way, expressed in a number or percentage and expressed using a unit of measure like hours or dollars – I really recommend his book – .

As industry matures, we are getting better and better at measuring the different processes and different security controls. There are many well defined metrics and the book mentioned previously is a great resource. But let’s consider a practical example. A security monitoring function, maybe within a Security Operations Center.  What do we want to measure and report? It depends on the size, scope and maturity level of the organization but below some reporting goals that one might chose in order to support the overall mission:

  • Provide end to end effectiveness metrics about operational readiness to detect and block threats.
  • Periodic benchmark about operational readiness to detect and block threats.
  • Minimize the risk of attacks that could result in lost revenue, public embarrassment, and/or regulatory penalties.

So, if you stated your reporting goals what would you achieve with those? what might be the outcomes?

  • Enable informed business decisions by producing actionable intelligence and situational awareness.
  • Being able to plan for, manage and respond to all categories of threats – even as they become more frequent and more severe.
  • Minimize false positives and harmless security incidents focusing on valuable and meaningful incidents.
  • Facilitate strategic decisions to improve the security monitoring service by looking at the big picture.
  • Guide the resource allocation.
  • Help diagnose problems.

Now that we might have the goals and the benefits of reaching those goals let’s consider how we do it. First might be a manual approach by gathering the different data sources, extracting the data, consolidate the information, produce the dashboards/graphs, analyze the data and provide the actionable recommendations. Then you improve from there in an incremental approach by producing a mature and consistent process that generate the metrics in an automated fashion.  The content of your report or dashboard might contain:

  • Total number of devices being monitored.
  • Volume of events, incidents and tickets that were handled (both the number and the type).
  • Resolution times (a measure of the length of time from when the incident/ticket was received, the length of time from when the incident/ticket was dispatched, etc.).
  • Number of employees (e.g. Cost).
  • Headcount to Ticket ratio (e.g.,Improve the per capita security incidents ratio enabling a close monitor of the workload that might contributes to a better/worse work environment).
  • Number of employee certifications ( how well-trained and well-equipped is your team?).
  • Events/Incidents generated per region, device, signature (top talkers, outliers, trends..).

With these metrics we start to have answers to basic questions that business leaders might have such as:

  • Are security incidents going up or down?
  • Does the security operations team respond quickly to the different incidents?
  • Does the regional SOC ranks favorably when compared with other regional SOC (both in volume and per capita security incidents rate).
  • How effective is the SOC?
  • Is the SOC running better off than it was last month/quarter/year?


Tagged , , , ,
gb_master's /dev/null

... and I said, "Hello, Satan. I believe it's time to go."

Source Code Auditing, Reversing, Web Security

Finding Hidden codes in the software

BruteForce Lab's Blog

security, programming, devops, visualization, the cloud

Count Upon Security

Increase security awareness. Promote, reinforce and learn security skills.

Naked Security

Computer Security News, Advice and Research

Didier Stevens

(blog \'DidierStevens)


Adventures in double-clicking malware / by Anuj Soni

Rational Survivability

Hoff's Ramblings about Information Survivability, Information Centricity, Risk Management and Disruptive Innovation.

SANS Internet Storm Center, InfoCON: green

Increase security awareness. Promote, reinforce and learn security skills.


Increase security awareness. Promote, reinforce and learn security skills.

Schneier on Security

Increase security awareness. Promote, reinforce and learn security skills. Blog

Increase security awareness. Promote, reinforce and learn security skills.

Lenny Zeltser

Increase security awareness. Promote, reinforce and learn security skills.

Krebs on Security

In-depth security news and investigation