Category Archives: Digital Forensics and Incident Response

Analysis of a Master Boot Record – EternalPetya

NoPetya or EternalPetya has kept the security community pretty busy last week. A malware specimen that uses a combined arms approach and maximizes its capabilities by using different techniques to sabotage business operations. One aspect of the malware that raised my interest was the ability to overwrite the Master Boot Record (MBR) and launch a custom bootloader. This article shows my approach to extract the MBR using digital forensic techniques and then analyze the MBR using Bochs. Before we roll up our sleeves let’s do a quick review on how the MBR is used by today’s computers during the boot process.

The computers that rely on BIOS flash memory instead of the new EFI standard, when they boot, the BIOS code is executed and, among other things, the code performs a series of routines that perform hardware checks i.e., Power-On-Self-Tests (POST). Then, the BIOS attempts to find a bootable device. If the bootable device is a hard drive, the BIOS reads the sector 1, track 0, head 0 and if contains a valid MBR (valid means that the sector ends with bytes 0xAA55) it will load that sector into a fixed memory location. By convention the code will be loaded into the real-mode address 0000:7c00. Then, the instruction pointer register is transferred into that memory location and the CPU will start executing the MBR code. What happens next is dependable on the MBR implementation code i.e., different operating systems have different MBR code  Nonetheless, the code needs to fit in the 512-bytes available at disk sector. The MBR follows a standard and its structure contains executable code, the partition table (64-bytes) with the locations of the primary partitions and finally 2-bytes with 0xAA55 signature. This means only 446-bytes are available to implement a MBR. In the Microsoft world, when the MBR code is executed, its role is to find an active partition, read its first sector, which contains the VBR code, load it into memory and transfer execution into it. The VBR is a bootloader program that will find the Windows BootMgr program and executes it All this happens in 16-bits real-mode.

Now, that we have a brief overview about the boot process, how can we extract and analyze the MBR? More specifically the MBR that is used by EternalPetya? Well, we infect a victim machine on a controlled and isolated environment. We know that EternalPetya main component is a DLL and we can launch it and infect a Windows machine by running “rundll32.exe petya.dll, #1 10”. Our setup consisted of 2 Virtual Machines. One running with Windows 7 and another running REMnux. We created a snapshot of the victim machine before the infection. Then executed the malware. Following that, we waited 10 minutes for the infection to complete. The scheduled task created by the malware restarted the operating system and a ransom note appeared. Then, I shutdown the Windows 7 virtual machine and used vmware-vdiskmanager.exe utility to create a single VMDK file from the disk state before and after the infection. Next, I moved the VMDK files to a Linux machine where I used QEMU to convert the VMDK images to RAW format.

Following that I could start the analysis and look at the MBR differences. The picture below illustrates the difference between the original MBR and the EternalPetya MBR. On the right side you have the EternalPetya MBR, the first 147 bytes (0x00 through 0x92) contain executable code.  The last 66 bytes (0x1be through 0x1fd) contain the partition table and are equal to the original MBR.

So, we are interested in the code execution instructions. We can start by extracting the MBR into a binary file and convert it to assembly instructions. This can be done using radare, objdump or ndisasm. Now, this is the hard part of the analysis. Read the assembly instructions and understand what it does. We can look at the instructions and perform static analysis but we can also perform dynamic analysis by running the MBR code, combining both worlds we will have better understanding – or at least we can try.

To perform dynamic analysis of the MBR code we will use Bochs. Bochs is an open source, fully fledged x86 emulator. Originally written by Kevin Lawton in 1994 is still being actively maintained today and last April version 2.6.9 was released. Bochs brings a CLI and GUI debugger and is very useful to debug our MBR code.  In addition, Bochs can be integrated with IDA PRO and Radare.  You can download Bochs from here.  In our case, we want to use Bochs to dynamically debug our MBR code. For that we need a configuration file called Bochsrc which is used to specify different options such as the disk image and also the CHS parameters for the disk. This article from Hex-Rays contains a how-to on how to integrate Bochs with IDA PRO. At the end of the article there is the mbr_Bochs.zip file that you can download. We will use these files to use Bochs standalone or combined with IDA PRO in case you have it. The Bochsrc file that comes with the ZIP file contains options that are deprecated on the newer Bochs version. The picture below shows the Bochsrc options I used. The Bochs user guide documents this file well.

Then you can try your configuration setup and launch Bochs. If you have IDA PRO then you can use this guide starting from step 6 in order to integrate it with IDA PRO. If all is set up, the debugging session will open and stop at the first instruction from its own BIOS code at memory address F000:FFF0.  Why this address? You can read this and many other low level things in the outstanding work from Daniel B. Sedory.

Uncomment the last line from the Bochsrc configuration file, to tell Bochs to use the Enhanced Debugger. For further references, you can read the “How to DEBUG System Code using The Bochs Emulator on a Windows™ PC” article. Start Bochs again and the GUI will show up. You can load the stack view by pressing F2. Then set a breakpoint where the MBR code will be loaded by issuing the command “lb 0x7c00” and then the “c” to continue the debugging session.

Now we can look at the code, step into the instructions, inspect the registers and stack. After some back and forth with the debugger I created the following listing with my interpretation of some of the instructions.

Bottom line, the MBR code will perform a loop that uses BIOS interrupt 0x13 function 0x42 to read data starting at sector 1 of the hard-drive. The loop copies 8880 (0x22af) bytes of data into memory location 0x8000. When the copy is done, the execution is transferred to the address 0x8000 by performing a far jump and the malicious bootloader is executed. The malicious bootloader code has been uploaded by Matthieu Suiche to Virus Total here. You can also extract it from the hard drive by extracting the sector 1 through 18 or better using the commands from the following picture. Then you can perform static and dynamic analysis.

The 16-bits bootloader code is harder to analyze than the MBR code but it is based on the Petya ransomware code from 2016. In this great article, from Hasherezade, she analyzes both Petya and EternalPetya bootloader using IDA PRO. When you use Bochs integrated with IDA PRO disassembler and debugger, the analysis is more accessible due to the powerful combination.

That’s it for today – Entering the world of real-mode execution on x86 is quite interesting. Analyzing code that relies on BIOS services such as software interrupts to perform different operations like reading from disk or writing to screen or, accessing the memory through segments is revealing. What we learned today might be a starting point to start looking at bootkits that are beneath the operating system and subvert the MBR boot sequence.

Have fun!

References:

Windows Internals, Sixth Edition, Part 2 By: Mark E. Russinovich, David A. Solomon, and Alex Ionescu

Various articles written by Daniel B. Sedory : http://starman.vertcomp.com/index.html

http://standa-note.blogspot.ch/2014/11/debugging-early-boot-stages-of-windows.html

Tagged , , ,

Threat Hunting in the Enterprise with AppCompatProcessor

Last April, at the SANS Threat Hunting and IR Summit, among other things, there was a new tool and technique released by Matias Bevilacqua. Matias’s presentation was titled “ShimCache and AmCache enterprise-wide hunting, evolving beyond grep” and he released the tool AppCompatProcessor. Matias also wrote a blog post “Evolving Analytics for Execution Trace Data” with more details.

In this article, I want to go over a quick exercise on how to use this tool and expand the existing signatures. First, let me write that, in case you have a security incident and you are doing enterprise incident response or you are performing threat hunting as part of your security operations duties, this is a fantastic tool that you should become familiar with and have on your toolkit. Why? Because it allows the security teams to digest, parse and analyze, at scale, two forensic artifacts that are very useful. The forensic artifacts are part of the Windows Application Experience and Compatibility features and are known as ShimCache and the AMCache.

To give you more context, the ShimCache can be obtained from the registry and from it we can obtain information about all executable binaries that have been executed in the system since it was rebooted. Furthermore, it tracks its size and the last modified date. In addition, the ShimCache tracks executables that have not been executed but were browsed for example through explorer.exe. This makes a valuable source of evidence for example to track executables that were on the system but weren’t executed – consider an attacker that used a directory on a system to move around his toolkit. The AMCache is stored on a file and from it we can retrieve information for every executable that run on the system such as the PATH, last modification time and created, SHA1 and PE properties. You can read more about those 2 artifacts in the article I wrote last year.

So, I won’t go over on how to acquire this data at scale – feel free to share you technique in the comments – but, AppCompatProcessor digests data that has been acquired by ShimCacheParser.py, Redline and MIR but also consumes raw ShimCache and AMCache registry hives. I will go directly to the features.At the time of this writing the tool version is 0.8 and one of the features I would like to focus today is the search module. This module allows us to search for known bad using regex expressions. The search module was coded with performance in mind, which means the regex searches are quite fast. By default, the tool includes more than 70 regex signatures for all kinds of interesting things an analyst will look for when perform threat hunting. Signatures include searching for dual usage tools like psexec , looking for binaries in places where they shouldn’t normally be, commonly named credential dumpers, etc. The great thing is that you can easily include your own signatures. Just add a regex line with your signature!

For this exercise, I want to use the search module to search for binaries that are commonly used by the PlugX backdoor family and friends. This backdoor is commonly used by different threat groups on targeted attacks. PlugX is also refered as KORPLUG, SOGU, DestroyRAT and is a modular backdoor that is designed to rely on the execution of signed and legitimated executables to load malicious code. PlugX, normally has three main components, a DLL, an encrypted binary file and a legitimated executable that is used to load the malware using a technique known as DLL search order. I won’t go discuss the details about PlugX in this article but you can read the White Paper “PlugX – Payload Extraction” done by Kevin O’Reilly from Context, the presentation about Plugx at Black Hat ASIA in 2014 given by Takahiro Haruyama and Hiroshi Suzuki, the analysis done by the Computer Incident Response Center Luxembourg and the Ahnlab threat report. With this and other reports you could start compiling information about different PlugX payloads. However, Adam Blaszczyk from Hexacorn, already did that job and wrote an article where he outlines different PlugX payloads seen in the wild.

Ok, with this information, we start creating the PlugX regex signatures. Essentially we will be looking for the signed and legitimate executables but in places where they won’t normaly be. The syntax to create a new regex signature is simple and you can add your own signatures to the existing AppCompatSearch.txt file or just create a new file called AppCompatSearch-PlugX.txt which will be consumed automatically by the tool. The figure below shows the different signatures that I produced. . At the time of this writing, this is still work in progress but is a starting point.

Next step, launch AppCompatProcessor against our data set using the newly created signatures. The following picture shows how the output of the search module looks like. In this particular case the search produced 25 hits and a nicely presented summary of the hits is displayed on a histogram. The raw dumps of the hits are saved on the file called Output.txt.  As an analyst or investigator, you would look at the results and verify which ones would be worth to further investigate and which ones are false positives. For this exercise, there was a hit that triggered on the file “c:\Temp\MsMpEng.exe”. This file is part of the Windows Defender suite but could be used by PlugX as part of DLL search order hijack technique. Basically, the attacker will craft a malicious DLL named MpSvc.dll and will place that in the same directory as the MsMpEng.exe file and execute MsMpEng.exe. The DLL would need to be crafted in a special way but that is what PlugX specializes in. This will load the attacker code.

Following these findings, we would want to look at the system that triggered the signature and view all the entries. The picture below shows this step where we use the dump module. The output shows all the ShimCache entries for this particular system. The entries are normally sorted in order of execution from bottom to top, and in this case, adjacent to the “c:\Temp\MsMpEng.exe” file there are several windows built-in commands that were executed and a file named “c:\Temp\m64.exe”. This is what Matias calls a strong temporal execution correlation. This is indicative that an attacker obtained access to the system, executed several windows built-in commands and and executed a file called “m64.exe” which likely is Mimikatz or a cousin. 

Following those leads, you might want to obtain those binaries from the system and perform malware analysis in order to extract indicators of compromise such as the C&C address, look at other artifacts such Windows Event Logs, UsnJournal, memory, etc.. and have additional leads. In addition, you might want to further use AppCompatProcessor to search for the “m64.exe” file and also use the tstack module, to search across all the data set for binaries that match the date of those two binaries. With these findings, among other things, you would need to scope the incident by understanding which systems the attacker accessed, find new investigation leads and pivot on the findings. AppCompatProcessor is a tool that helps doing that. This kind of finding would definitely trigger your incident response processes and procedures.

That’s it, hopefully, AppCompatProcessor will reduce the entry barrier for your security operations center or incident response teams to start performing threat hunting in your environment and produce actionable results. If you find this useful, contribute with your threat hunting signatures in AppCompatProcessor GitHub repo and Happy Hunting!

 

Tagged , , , , , ,

Intro to Linux Forensics

This article is a quick exercise and a small introduction to the world of Linux forensics.  Below, I perform a series of steps in order to analyze a disk that was obtained from a compromised system that was running a Red Hat operating system. I start by recognizing the file system, mounting the different partitions, creating a super timeline and a file system timeline. I also take a quick look at the artifacts and then unmount the different partitions. The process of how to obtain the disk will be skipped but here are some old but good notes on how to obtain a disk image from a VMware ESX host.

When obtaining the different disk files from the ESX host, you will need the VMDK files. Then you move them to your Lab which could be simple as your laptop running a VM with SIFT workstation. To analyze the VMDK files you could use the “libvmdk-utils” package that contain tools to access data store in VMDK files. However, another approach would be to convert the VMDK file format into RAW format. In this way, it will be easier to run the different tools such as the tools from The Sleuth Kit – which will be heavily used – against the image. To perform the conversion, you could use the QEMU disk image utility. The picture below shows this step.

Following that, you could list the partition table from the disk image and obtain information about where each partition starts (sectors) using the “mmls” utility. Then, use the starting sector and query the details associated with the file system using the “fsstat” utility. As you could see in the image, the “mmls” and “fsstat” utilities are able to identify the first partition “/boot” which is of type 0x83 (ext4). However, “fsstat” does not recognize the second partition that starts on sector 1050624.

This is due to the fact that this partition is of type 0x8e (Logical Volume Manager). Nowadays, many Linux distributions use LVM (Logical Volume Manager) scheme as default. The LVM uses an abstraction layer that allows a hard drive or a set of hard drives to be allocated to a physical volume. The physical volumes are combined into logical volume groups which by its turn can be divided into logical volumes which have mount points and have a file system type like ext4.

With the “dd” utility you could easily see that you are in the presence of LVM2 volumes. To make them usable for our different forensic tools we will need to create device maps from the LVM partition table. To perform this operation, we start with “kpartx” which will automate the creation of the partition devices by creating loopback devices and mapping them. Then, we use the different utilities that manage LVM volumes such as “pvs”, “vgscan” abd “vgchange”. The figure below illustrates the necessary steps to perform this operation.

After activating the LVM volume group, we have six devices that map to six mount points that make the file system structure for this disk. Next step, mount the different volumes as read-only as we would mount a normal device for forensic analysis. For this is important to create a folder structure that will match the partition scheme.

After mounting the disk, you normally start your forensics analysis and investigation by creating a timeline. This is a crucial step and very useful because it includes information about files that were modified, accessed, changed and created in a human readable format, known as MAC time evidence (Modified, Accessed, Changed). This activity helps finding the particular time an event took place and in which order.

Before we create our timeline, noteworthy, that on Linux file systems like ext2 and ext3 there is no timestamp about the creation/birth time of a file. There are only 3 timestamps. The creation timestamp was introduced on ext4. The book “The Forensic Discovery 1st Edition”from Dan Farmer and Wietse Venema outlines the different timestamps:

  • Last Modification time. For directories, the last time an entry was added, renamed or removed. For other file types, the last time the file was written to.
  • Last Access (read) time. For directories, the last time it was searched. For other file types, the last time the file was read.
  • Last status Change. Examples of status change are: change of owner, change of access permission, change of hard link count, or an explicit change of any of the MACtimes.
  • Deletion time. Ext2fs and Ext3fs record the time a file was deleted in the dtime stamp.the filesystem layer but not all tools support it.
  • Creation time: Ext4fs records the time the file was created in crtime stamp but not all tools support it.

The different timestamps are stored in the metadata contained in the inodes. Inodes are the equivalent of MFT entry number on a Windows world. One way to read the file metadata on a Linux system is to first get the inode number using, for example, the command “ls -i file” and then you use “istat” against the partition device and specify the inode number. This will show you the different metadata attributes which include the time stamps, the file size, owners group and user id, permissions and the blocks that contains the actual data.

Ok, so, let’s start by creating a super timeline. We will use Plaso to create it. For contextualization  Plaso is a Python-based rewrite of the Perl-based log2timeline initially created by Kristinn Gudjonsson and enhanced by others. The creation of a super timeline is an easy process and it applies to different operating systems. However, the interpretation is hard. The last version of Plaso engine is able to parse the EXT version 4 and also parse different type of artifacts such as syslog messages, audit, utmp and others. To create the Super timeline we will launch log2timeline against the mounted disk folder and use the Linux parsers. This process will take some time and when its finished you will have a timeline with the different artifacts in plaso database format. Then you can convert them to CSV format using “psort.py” utility. The figure below outlines the steps necessary to perform this operation.

Before you start looking at the super timeline which combines different artifacts, you can also create a traditional timeline for the ext file system layer containing data about allocated and deleted files and unallocated inodes. This is done is two steps. First you generate a body file using “fls” tool from TSK. Then you use “mactime” to sort its contents and present the results in human readable format. You can perform this operation against each one of the device maps that were created with “kpartx”. For sake of brevity the image below only shows this step for the “/” partition. You will need to do it for each one of the other mapped devices.

Before we start the analysis, is important to mention that on Linux systems there is a wide range of files and logs that would be relevant for an investigation. The amount of data available to collect and investigate might vary depending on the configured settings and also on the function/role performed by the system. Also, the different flavors of Linux operating systems follow a filesystem structure that arranges the different files and directories in a common standard. This is known as the Filesystem Hierarchy Standart (FHS) and is maintained here. Its beneficial to be familiar with this structure in order to spot anomalies. There would be too much things to cover in terms of things to look but one thing you might want to run is the “chkrootkit” tool against the mounted disk. Chrootkit is a collection of scripts created by Nelson Murilo  and Klaus Steding-Jessen that allow you to check the disk for presence of any known  kernel-mode and user-mode rootkits. The last version is 0.52 and contains an extensive list of known bad files.

Now, with the supertimeline and timeline produced we can start the analysis. In this case, we go directly to timeline analysis and we have a hint that something might have happened in the first days of April.

During the analysis, it helps to be meticulous, patience and it facilitates if you have comprehensive file systems and operating system artifacts knowledge. One thing that helps the analysis of a (super)timeline is to have some kind of lead about when the event did happen. In this case, we got a hint that something might have happened in the beginning of April. With this information, we start to reduce the time frame of the (super)timeline and narrowing it down. Essentially, we will be looking for artifacts of interest that have a temporal proximity with the date.  The goal is to be able to recreate what happen based on the different artifacts.

After back and forth with the timelines, we found some suspicious activity. The figure below illustrates the timeline output that was produced using “fls” and “mactime”. Someone deleted a folder named “/tmp/k” and renamed common binaries such as “ping” and “ls” and files with the same name were placed in the “/usr/bin” folder.

This needs to be looked further. Looking at the timeline we can see that the output of “fls” shows that the entry has been deleted. Because the inode wasn’t reallocated we can try to see if a backup of the file still resides in the journaling. The journaling concept was introduced on ext3 file system. On ext4, by default, journaling is enabled and uses the mode “data=ordered”. You can see the different modes here. In this case, we could also check the options used to mount the file system. To do this just look at “/etc/fstab”. In this case we could see that the defaults were used. This means we might have a chance of recovering data from deleted files in case the gap between the time when the directory was deleted and the image was obtained is short. File system journaling is a complex topic but well explained in books like “File system forensics” from Brian Carrier. This SANS GCFA paper from Gregorio Narváez also covers it well.  One way you could  attempt to recover deleted data is using the tool “extundelete”.  The image below shows this step.

The recovered files would be very useful to understand more about what happened and further help the investigation. We can compute the files MD5’s, verify its contents and if they are known to the NSLR database or Virustotal. If it’s a binary we can do strings against the binary and deduce functionality with tools like “objdump” and “readelf”. Moving on, we also obtain and look at the different files that were created on “/usr/sbin” as seen in the timeline. Checking its MD5, we found that they are legitimate operating system files distributed with Red Hat. However, the files in “/bin” folder such as “ping” and “ls” are not, and they match the MD5 of the files recovered from “/tmp/k”.  Because some of the files are ELF binaries, we copy them to an isolated system in order to perform quick analysis. The topic of Linux ELF binary analysis would be for other time but we can easily launch the binary using “ltrace -i” and “strace -i” which will intercept and record the different function/system calls. Looking at the output we can easily spot that something is wrong. This binary doesn’t look the normal “ping” command, It calls the fopen() function to read the file “/usr/include/a.h” and writes to a file in /tmp folder where the name is generated with tmpnam(). Finally, it generates a segmentation fault. The figure below shows this behavior.

Provided with this information, we go back and see that this file “/usr/include/a.h” was modified moments before the file “ping” was moved/deleted. So, we can check when this “a.h” file was created – new timestamp of ext4 file system – using the “stat” command. By default, the “stat” doesn’t show the crtime timestamp but you can use it in conjunction with “debugfs” to get it. We also checked that the contents of this strange file are gibberish.

So, now we know that someone created this “a.h” file on April 8, 2017 at 16:34 and we were able to recover several other files that were deleted. In addition we found that some system binaries seem to be misplaced and at least the “ping” command expects to read from something from this “a.h” file. With this information we go back and look at the super timeline in order to find other events that might have happened around this time.  As I did mention, super timeline is able to parse different artifacts from Linux operating system. In this case, after some cleanup we could see that we have artifacts from audit.log and WTMP at the time of interest. The Linux audit.log tracks security-relevant information on Red Hat systems. Based on pre-configured rules, Audit daemon generates log entries to record as much information about the events that are happening on your system as possible. The WTMP records information about the logins and logouts to the system.

The logs shows that someone logged into the system from the IP 213.30.114.42 (fake IP) using root credentials moments before the file “a.h” was created and the “ping” and “ls” binaries misplaced.

And now we have a network indicator. Next step we would be to start looking at our proxy and firewall logs for traces about that IP address. In parallel, we could continue our timeline analysis to find additional artifacts of interest and also perform in-depth binary analysis of the files found, create IOC’s and, for example, Yara signatures which will help find more compromised systems in the environment.

That’s it for today. After you finish the analysis and forensic work you can umount the partitions, deactivate the volume group and delete the device mappers. The below picture shows this steps.

Linux forensics is a different and fascinating world compared with Microsoft Windows forensics. The interesting part (investigation) is to get familiar with Linux system artifacts. Install a pristine Linux system, obtain the disk and look at the different artifacts. Then compromise the machine using some tool/exploit and obtain the disk and analyze it again. This allows you to get practice. Practice these kind of skills, share your experiences, get feedback, repeat the practice, and improve until you are satisfied with your performance. If you want to look further into this topic, you can read “The Law Enforcement and Forensic Examiner’s Introduction to Linux” written by Barry J. Grundy. This is not being updated anymore but is a good overview. In addition, Hal Pomeranz has several presentations here and a series of articles written in the SANS blog, specially the 5 articles written about EXT4.

 

References:
Carrier, Brian (2005) File System Forensic Analysis
Nikkel, Bruce (2016) Practical Forensic Imaging

Tagged , , , ,

Extract and use Indicators of Compromise from Security Reports

apt-reports-1Every now and then there is a new security report released and with it comes a a wealth  of information about different threat actors offering insight about the attacker’s tactics, techniques and procedures. For example, in this article I wrote back in 2014, you have a short summary about some of the reports that were released publicly throughout the year. These reports allow the security companies to advertise their capabilities but on the other hand they are a great resource for network defenders. Some reports are more structured than others and they might contain different technical data. Nonetheless, almost all of them have IOC’s (Indicators of compromise).

For those who never heard about indicators of compromise let me give a brief summary. IOC’s are pieces of information that can be used to search and identify compromised systems. These pieces of information have been around since ages but the security industry is now using them in a more structural and consistent fashion. All types of enterprises are moving from the traditional way of handling security incidents – Wait for an alert to come in and then respond to it – to a more proactive approach which consists in taking the necessary steps to hunt for evil in order to defend their networks. In this new strategy, the IOCs are a key technical component. When someone compromises a system, they leave evidence behind.  That evidence, artifact or remnant piece of information left by an attacker can be used to identify the intrusion, threat group or the malicious actor. Examples of IOCs are IP addresses, domain names, URLs, email addresses, file hashes, HTTP user agents, registry keys, a service configuration change, a file is deleted, etc. With this information one could sweep the network/endpoints and look for indicators that the system might have been compromised. For more background about it you can read Lenny Zeltzer summary. Will Gragido from RSA explained it well in is 3 parts blog herehere and here. Mandiant also has this and this great articles about it.

So, if almost all the reports published contain IOC’s, is there a central place that contains all reports that were released by the different security organizations? Kiran Bandla created a repository on GitHub named APTnotes where he does exactly that. He maintains a list of all security reports released by the different vendors – “APTnotes is a repository of publicly-available papers and blogs (sorted by year) related to malicious campaigns/activity/software that have been associated with vendor-defined APT (Advanced Persistent Threat) groups and/or tool-sets.” Kiran, among other methods relies on the community who share with him when a report X or Y was released and he adds it to the APTnotes repository. The list of reports is available in CSV and JSON format.

So, what can you do with all this reports? How can we as network defenders use this information?

Three example use cases.  The first use case is more likely and common, where the other two might be more common in larger organizations with higher budget and bigger security appetite.

  • An organization that has a Security Operations Center in place could use the IOC’s to augment their existing monitoring and detection capabilities.
  • An organization that has in-house CSIRT capabilities could leverage the IOC’s in a proactive manner in order to have a higher probability of discovering something bad and, as such, reduce the business impact that a security incident might have in the organization.
  • An organization that has a Cyber Threat Intelligence capability in-house, could collect, process, exploit and analyze these reports. Then disseminate actionable information to the threat intelligence consumers throughout the organization.

In a simple manner, the process for the first scenario would look something like the following diagram:

ioc-loop

How would this work in practical terms? Normally, you could split the IOC’s in host based or network based. For example, a DNS name or IP addresses will be more effective to search across your network infrastructure. However, a Registry Key or a MD5 will be more likely to be searched across the endpoint.

For this article, I will focus on MD5’s. Some reports offer file hashes using SHA-1 or SHA-256 but not many organizations have the capability to search for this. MD5 is more common. Noteworthy that the value of MD5 hashes about a malicious file might be considered low.  A great article about the value of IOC’s and TTP’s was written back in 2014 from David Bianco title The Pyramid of Pain. Following that there is an article from Harlan Carvey with additional thoughts about it. Another point to take into consideration about the MD5’s from the reports is that some might be from legitimate files due to the usage of DLL hijacking or they are windows built-in commands used by threat actors. In addition is likely that malware used by the different threat actors is only used once and you might not see a MD5 hash a second time. Nonetheless, the MD5’s is a starting point.

So, how can we collect the reports, extract the IOC’s and convert them and use them?

First you can use a python script to download all the reports in a central place separated per year. Then you can use the tool IOC parser written by Armin Buescher. This tool will be able to parse PDF reports and extract IOC’s into CSV format. From here you can extract the relevant IOC’s. For this example, I want to extract the MD5’s and then use IOC writer to create IOC’s in OpenIOC 1.0/1.1  format which could be used with a tool such as Redline.  IOC writer is a python library written by William Gibb that allows you to manipulate IOC’s in OpenIOC 1.1 and 1.0 format and was released on BlackHat 2013.

The steps necessary to perform this are illustrated below – for sure there are  other better ways to perform this but this was a quick way to do the job -.

extract-create-ioc

With this we created a series of files with  .ioc extension that can be further edited with the ioc_writer Python library. However, not many tools support OpenIOC, so you might just use the MD5’s and feed that into whatever tool or format you are using. You can download below the excel with all the MD5’s separated per year.  If you use them, please bare in mind you might have false positives and you will need to go back to the csv files to understand from which report did the MD5 came from. For example I  already removed half dozen of MD5’s that are legitimate files but seen in the reports, and also the MD5 for an empty file “d41d8cd98f00b204e9800998ecf8427e”

That’t it, with this you can create a custom IOC set that contain MD5’s of different tools, malware families and files that was compiled by extracting the MD5’s from the public reports about targeted attacks. From here you can start building more complex IOC’s with different artifacts based on a specific report or threat actor. Maybe you get lucky and you find evil on your network!

The year 2010 contains 81 unique MD5’s.
The year 2011 contains 96 unique MD5’s.
The year 2012 contains 718 unique MD5’s.
The year 2013 contains 2149 unique MD5’s
The year 2014 contains 1306 unique MD5’s
The year 2015 contains 1553 unique MD5’s
The year 2016 contains 1173 unique MD5’s

Excel : md5-aptnotes-2010-2016

Tagged
gb_master's /dev/null

... and I said, "Hello, Satan. I believe it's time to go."

Source Code Auditing, Reversing, Web Security

Finding Hidden codes in the software

BruteForce Lab

security, programming, devops, visualization, the cloud

Count Upon Security

Increase security awareness. Promote, reinforce and learn security skills.

Naked Security

Computer Security News, Advice and Research

Didier Stevens

(blog \'DidierStevens)

malwology

Adventures in double-clicking malware / by Anuj Soni

Rational Survivability

Hoff's Ramblings about Information Survivability, Information Centricity, Risk Management and Disruptive Innovation.

SANS Internet Storm Center, InfoCON: green

Increase security awareness. Promote, reinforce and learn security skills.

TaoSecurity

Increase security awareness. Promote, reinforce and learn security skills.

Schneier on Security

Increase security awareness. Promote, reinforce and learn security skills.

Technicalinfo.net Blog

Increase security awareness. Promote, reinforce and learn security skills.

Lenny Zeltser

Increase security awareness. Promote, reinforce and learn security skills.

Krebs on Security

In-depth security news and investigation