Digital Forensics – PlugX and Artifacts left behind

When an attacker conducts an intrusion using A, B or C technique, some of his actions leave artifact X, Y or Z behind. So, based on the scenario from the last article about PlugX, I collected a disk image and memory image from the domain controller. Over the past years I wrote several articles on how to perform acquisition, mounting and processing of such images and analyze them by creating super timelines, look at different artifacts like Event Logs, Prefetch, ShimCache, AMCache, etc., or analyze NTFS metadata or look for artifacts related to interactive sessions. Today, I’m not going to perform analysis but I’m going to list a quick overview about some of the Windows endpoint artifacts that might give us evidence about the actions that were executed in the previous scenario and help us produce a meaningful timeline. In addition, I list some tools that could be used to analyze those artifacts.

Scenario 1: The attacker placed the filename “kas.exe” on the folder “c:\PerfLogs\Admin”. Which artifacts could record evidence about this action?

    • Description: The Master File Table (MFT) is a special system file that resides on the root of every NTFS partition. The file is named $MFT and is not accessible via user mode API’s but can been seen when you have raw access to the disk e.g, forensic image. This special file is a hierarchical database and inside you have records that contains a series of attributes about a file, directory and indicates where it resides on the physical disk and if is active or inactive. The size of each MFT record is usually 1024-bytes. Each record contains a set of attributes. Some of the most important attributes in a MFT entry are the $STANDART_INFORMATION, $FILENAME and $DATA. The first two are rather important because among other things they contain the file time stamps. Each MFT entry for a given file or directory will contain 8 timestamps. 4 in the $STANDARD_INFORMATION and another 4 in the $FILENAME. These time stamps are known as MACE.
    • Tools: Parse and analyze it with SleuthKit originally written by Brian CarrierMFT2CSV from Joakim Schicht or PLASO/log2timeline originally created by Kristinn Gudjonsson
  • NTFS INDX Attribute
    • Description: The MFT records for directories contain a special attribute called $I30. This attribute contains information about file names and directories that are stored inside a directory. This special attribute is also known as $INDX and consists of three attributes, the $INDEX_ROOT, $INDEX_ALLOCATION and $BITMAP. So, What? Well, this attribute stores information in a B-tree data structure that keeps data sorted so the operating system can perform fast searches in order to determine if a file is present. In addition, this attribute grows to keep track of file names inside the directory. However, when you delete a file from a directory the B-tree re-balances itself but the tree node with metadata about the deleted file remains in a form of slack space until it gets reused. This means we can view the $I30 attribute contents and we might find evidence of files that once existed in a directory but are no longer there.
      Tools: o Parse it and analyze it with INDXParse from William Ballenthin or MFT2CSV from Joakim Schicht.
  • NTFS $LogFile
    • Description: NTFS has been developed over years with many features in mind, one being data recovery. One of the features used by NTFS to perform data recovery is the Journaling. The NTFS Journal is kept inside NTFS Metadata in a file called $LOGFILE. This file is stored in the MFT entry number 2 and every time there is a change in the NTFS Metadata, there is a transaction recorded in the $LOGFILE. These transactions are recorded to be possible to redo or undo file system operations. After the transaction has been logged then the file system can perform the change. When the change is done, another transaction is logged in the form of a commit. The $LOGFILE allows the file system to recover from metadata inconsistencies such as transactions that don’t have a commit. The size of the $LOGFILE can be consulted and changed using chkdsk /l and per default is 65536 KB. Why would $LOGFILE be important for our investigation? Because the $LOGFILE keeps record of all operations that occurred in the NTFS volume such as file creation, deletion, renaming, copy, etc. Therefore, we might find relevant evidence in there.
    • Tools: Parse it and analyze it with LogFileParser from Joakim Schicht
  • NTFS $UsnJrnl
    • Description: The change journal contains a wealth of information that shouldn’t be overlooked. Another interesting aspect of the change journal is that allocates space and deallocates as it grows and records are not overwritten unlike the $LogFile. This means we can find old journal records in unallocated space on a NTFS volume. How to obtain those? Luckily, the tool USN Record Carver written by PoorBillionaire can carve journal records from binary data and thus recover these records
    • Tools: Parse and analyze it with UsnJrnl2Csv from Joakim Schicht or from unallocated space with USN Record Carver from PoorBillionaire.

Scenario 2: Which account did the attacker used to log into the system when he placed “kas.exe” on the file system?

  • Windows Event Logs
    • Description: The Windows Event logs record activities about the operating system and its applications. What is logged depends on the audit features that are turned thus impacting the information that one can obtain. From a forensic perspective the Event Logs capture a wealth of information. The main three Windows Event Logs are Application, System, and Security and on Windows Vista and beyond they are saved on %System32%\winevt\Logs in a binary format. For example the Event id’s 4624, 4625 might give us answers.
    • Tools: Parse it and Analyze it with PLASO/Log2timeline, LibEvtx-utils from Joakim Schicht , python-evtx from William Ballenthin or Event Log Explorer. You likely get better results if in your environment if you have consistent and enhanced audit policy settings defined that track both success and failures. In case the attacker  deletes the Windows Event Logs, there is the possibility to recover Windows Event Log records from the pagefile.sys or from unallocated space, from Volume Shadow copies or even the system Memory. You could use EVTXtract from Willi Ballenthin to attempt to recover Event logs from raw data.

Scenario 3: Attacker executed the “kas.exe” binary. Which artifacts might record this evidence?

  • Windows Prefetch / Superfetch
    • Description: To improve customer experience, Microsoft introduced a memory management technology called Prefetch. This functionality was introduced into Windows XP and Win-dows 2003 Server. This mechanism analyses the applications that are most frequently used and preloads them in advance in order speed the operating system booting and application launching. On Windows Vista, Microsoft enhanced the algorithm and introduced SuperFetch which is an improved version of Prefetch. The Prefetch files are stored in %SYSTEMROOT%\Prefetch directory and have a .pf extension. The Superfetch files have a .db extension. Prefetch files keep track of programs that have been executed in the system even if the original file is no longer present. In addition Prefetch files can tell you when the program was executed, how many times and from which path.
    • Tools: PLASO/log2timeline, Windows-Prefetch-Parser from Adam Witt, Prefetch Parser from Eric Zimmerman. For Superfetch you could use SuperFetch tools.
  • ShimCache either from Registry or from Kernel Memory
    • Description: Microsoft introduced the ShimCache in Windows 95 and it remains today a mechanism to ensure backward compatibility of older binaries into new versions of Microsoft op-erating systems. When new Microsoft operating systems are released some old and legacy application might break. To fix this Microsoft has the ShimCache which acts as a proxy layer between the old application and the new operating system. A good overview about what is the ShimCache is available on the Microsoft Blog on an article written by Tim Newton “Demystifying Shims – or – Using the App Compat Toolkit to make your old stuff work with your new stuff“. The interesting part is that from a forensics perspective the ShimCache is valuable because the cache tracks metadata for binary that was executed and stores it in the ShimCache.
    • Tools: From Kernel memory, you can parse it and analyze it with Volatility ShimCache and ShimCacheMem plugin. From the Registry you can use ShimCacheParser You can also use RegRipper from Harlan Carvey or AppCompatCacheParser from Eric Zimmerman. In addition, to analyze ShimCache artifacts at scale you can use AppCompatProcessor from Mattias Bevilacqua,
  • AMCache
    • Description: On Windows 8, Amcache.hve replaced the RecentFileCache.bcf file, a registry file used in Windows 7 as part of the Application Experience and Compatibility feature to ensure compatibility of existing software between different versions of Windows. Similar to its predecessor, Amcache.hve is a small registry hive that stores a wealth of information about recently run applications and programs, including full path, file timestamps, and file SHA1 hash value. Amcache.hve is commonly found at the following location: C:\Windows\AppCompat\Programs\Amcache.hve. The Amcache.hve file is standard within the Windows 8 operating system, but has been found to exist on Windows 7 systems as well.
    • Tools: To read the amcache HIVE you could use RegRipper or Willi Ballenthin stand-alone script or Eric Zimmerman AmcacheParser. To analyze AMCache artifacts at scale you can use AppCompatProcessor from Mattias Bevilacqua,
  • Windows Event Logs. 
    • The Windows Event logs – for example id 4688 – could track binary execution if you have the proper audit settings or you use Sysmon.

Scenario 4: The execution of “kas.exe” dropped three files on disk that used DLL Search Order Hijacking to achieve persistence and install the malicious payload. Which artifacts might help identifying this technique?

Identifying evidence of DLL Search Order hijacking is not easy if no other leads are available. Likely you need a combination of artifacts. The following artifacts / tools might help.

  • NTFS MFT, INDX, $LogFile, $UsnJrnl.
  • Prefetch / SuperFetch.
  • ShimCache either from Registry or from Kernel Memory.
  • AMCache.
  • Windows Event Logs could track process execution and give you leads if you have the proper audit settings or you use Sysmon
  • Volatility to perform memory analysis.
  • RegRipper – One thing you could try, among many others that this powerful tool allows,is to identify different persistence mechanism that could have resulted as part of the DLL Search Order Hijacking technique.
  • AppCompatProcessor to analyze ShimCache and AMCache at scale combined with with PlugX signatures.

Scenario 5: The PlugX dropped files have the NTFS timestamps manipulated i.e., It copies the timestamps obtained from the operating system filename ntdll.dll to set the timestamps on the dropped files. What artifacts could be used to detect this?

The time modification will cause a discrepancy between the NTFS $STANDART_INFORMATION and $FILENAME timestamps. You could combine the NTFS artifacts with the execution artifacts to spot such anomalies.  Other technique you could use is with AppCompat Processor which has the Time Stomp functionality that will search for appcompat entries outside of the Windows,  System and SysWOW64 folders with last modification dates matching a list of known operating system files.

Scenario 6: Attacker used the PlugX controller to Invoke a command shell and execute Windows built-in commands. Are there any artifacts left behind that could help understand commands executed?

  • ShimCache either from Registry or from Kernel Memory.
  • Memory analysis with Volatility and look for Process creation, Console History, cmdscan or consoles plugin.
  • The Windows Event logs could track process execution if you have the proper audit settings or you use Sysmon.

Scenario 7: Attacker established a persistence mechanism either using a Service or Registry Key. 

  • Producing a timeline of the Registry would help identify the last modification dates of the registry keys. You could use RegRipper from Harlan Carvey or RECmd from Eric Zimmerman. The Windows Event logs would also help in case the there was a service created on the operating system. For example Event ID 7009, 7030, 7035, 7036, 7040, 7023 or 7045 could help. In addition, to list the services and its properties you could perform memory analysis with Volatility or use RegRipper.

Scenario 8: The attacker accessed the Active Directory database using the “ntdsutil.exe” command. What could be used to detect this activity?

  • As we saw previously, command execution could be identified using ShimCache either from Registry or from Kernel Memory. Because “ntdsutil.exe” would be executed on a Server system, Prefetch won’t help here because its not enabled on Server systems. One of the most usefull artifacts would be the Windows Event logs but you need to have the right settings so it could track binary execution and the interactions with the Active Directory. One thing that might help in case the memory image has been acquired not long after the attacker activity is to perform memory analysis and creating a timeline of the artifacts with Volatility might help identifying the process creation and its parent(s). In addition, you might get interesting leads just by running strings (little and  big endian) on the pagefile.sys. Other than that, the execution of “ntdsutil.exe” the way it was executed on the scenario, leaves behind artifacts on the NTFS metadata.

That’s it for today. With this article I presented a quick listing on some artifacts and tools that can help you perform forensic analysis on a system and help you answer your investigative questions. Many other tools and artifacts would be available depending on the attacker activities, for example if the attacker logged into a system interactively, but the ones listed might give you a starting point and might help you understand what happened and when. One thing that would greatly complement the findings of a system forensic analysis the network data such as the ones that comes from Firewall, Router, IDS or Proxy logs or any other kind of networking logs you might have. Specially if attacker is using a C2 and is clearing evidence such as the threat group that used a file named “a.bat” to clean several artifacts as illustrated on the “Paranoid PlugX” article written by Tom Lancaster and Esmid Idrizovic from Unit 42.

Happy hunting and If you have dealt with a security incident where PlugX was used, please leave your comments about the tools or techniques you used to detect it.

Tagged , , , , , , , , , ,

Malware Analysis – PlugX – Part 2

Following my previous article on PlugX, I would like to continue the analysis but now use the PlugX controller to mimic some of the steps that might be executed by an attacker. As you know the traditional steps of an attack lifecycle follow, normally, a predictable sequence of events i.e., Reconnaissance, initial compromise, establish foothold, escalate privileges, internal reconnaissance, move laterally, maintain persistence, complete mission. For sake of brevity I will skip most of the steps and will focus on the lateral movement.I will use the PlugX controller and C2 functionality to simulate an attacker that established a foothold inside an environment and obtained admin access to a workstation. Following that, the attacker moved laterally to a Windows Domain Controller. I will use the PlugX controller to accomplish this scenario and observe how an attacker would operate within a compromised environment.

As we saw previously, the PlugX controller interface allows an operator to build payloads, set campaigns and define the preferred method for the compromised hosts to check-in and communicate with the controller. In the PlugX controller, English version from Q3 2013, an operator can build the payload using two techniques. One is using the “DNS Online” technique which allows the operator to define the C2 address e.g, an URL or IP address, that will be used by the payload to speak with the C2. The other method, is the “Web Online”, which allows the operator to tell the payload from where it should fetch the C2 address. This method allows the operator to have more control over the campaign. The following diagram illustrates how the “Web Online” technique works.

Why do I say this technique would allow an attacker to have more control? Consider the case that an organization was compromised by a threat actor that use this PlugX technique. In case the C2 is discovered, the impacted organization could block the IP or URL on the existing boundary security controls as a normal reaction to the concerns of having an attacker inside the network. However, the attacker could just change the C2 string and point it to a different system. In case the organization was not able to scope the incident and understand the TTP’s (Tools, Tactics and Procedures) then the attacker would still maintain persistence in the environment. This is an example that when conducting incident response, among other things, you need to have visibility into the tools and techniques the attacker is using so you could properly scope the incident and perform effective and efficient containment and eradication steps. As an example of this technique, below is a print screen from a GitHub page that has been used by an unknown threat actor to leverage this method.

So, how to leverage this technique on the PlugX builder? The picture below shows how the operator could create a payload that uses the “Web Online” technique. The C2 address would be fetched from a specified site e.g. a Pastebin address, which on its turn would redirect the payload to the real C2 address. The string “DZKSAFAAHHHHHHOCCGFHJGMGEGFGCHOCDGPGNGDZJS” in this case is used to decode to the real C2 address which is “”. On the “PlugX: some uncovered points” article, Fabien Perigaud writes about how to decode this string. Palo Alto Unit42 gives another example of this technique on the “Paranoid PlugX” article. The article “Winnti Abuses GitHub for C&C Communications” from Cedric Pernet ilustrates an APT group leveraging this technique using GitHub.

For sake of simplicity, in this article, I’m going to use the DNS Online technique using “” as C2 address. Next, on the “First” tab I specify the campaing ID and the password used by the payload to connect to the C2.

Next, on the Install tab I specify the persistence settings, in this case I’m telling the payload to install a service and I can specify different settings including where to deploy the binaries, the service name and service description. In addition, I can specify that if the Service persistence mechanism fails due to some reason the payload should install the persistence mechanism using the Registry and I can specify which HIVE should be used.

Then, In the inject tab I specify which process will be injected with the malicious payload. In this case I choose “svchost.exe”. This will make PlugX start a new instance of “svchost.exe” and then inject the malicious code into svchost.exe process address space using process hollowing technique.

Other than that, the operator could define a schedule and determine which time of the week the payload should communicate with the C2. Also the operator could define the Screen Recording capability that will take screenshots at a specific frequency and will save them encrypted in a specific folder.

Last settings on the “option” tab allow the operator to enable the keylogger functionality and specify if the payload should hide it self and also delete itself after execution.

Finally, after all the settings defined, the operator can create/download the payload in different formats. An executable, binary form (Shellcode), or an array in C that can then be plugged in another delivery mechanism e.g, PowerShell or MsBuild. After deploying and installing the payload on a system, that system will check-in into the PlugX controller and an operator can call the “Manager” to perform the different actions. In this example I show how an attacker, after having compromised a system, uses the C2 interface to:

  • Browse the network

  • Access remote systems via UNC path

  • Upload and execute a file e.g., upload PlugX binary

  • Invoke a command shell and perform remote commands e.g., execute PlugX binary on a remote system

Previous pictures illustrate actions that the attacker could perform to move laterally and, for example, at some point in time, access a domain controller via UNC path, upload the PlugX payload to a directory of its choice and execute it. In this case the pictures show that the PlugX payload was dropped into c:\PerfLogs\Admin folder and then was executed using WMI. Below example shows the view from the attacker with two C2 sessions. One for one workstation and another for a domain controller.

Having access to a domain controller is likely one of the goals of the attacker so he can obtain all the information he needs about an organization from the Active Directory database.

To access the Active Directory database, the attacker could, for example, run the “ntdsutil.exe” command to create a copy of the “NTDS.dit” file using Volume Shadow Copy technique. Then, the attacker can access the database and download it to a system under his control using the PlugX controller interface. The picture below illustrates an attacker obtained the relevant data that was produced using the “ntdsutil.exe” command.

Finally, the attacker might delete the artifacts that were left behind on the file system as consequence of running “ntdsutil.exe”.

So, in summary, we briefly looked at the different techniques a PlugX payload could be configured to speak with a Command and Controller. We built, deploy and install a payload. Compromised a system and obtain a perspective from PlugX operator. We move laterally to a domain controller and installed the PlugX payload and then used a command shell to obtain the Active Directory database. Of course, as you noted, the scenario was accomplished with an old version of the PlugX controller. Newer versions likely have many new features and capabilities. For example, the print screen below is from a PlugX builder from 2014 (MD5: 534d28ad55831c04f4a7a8ace6dd76c3) which can create different payloads that perform DLL Search order hijacking using Lenovo’s RGB LCD Display Utility for ThinkPad (tplcdclr.exe) or Steve Gibson’s Domain Name System Benchmarking Utility (sep_NE.exe). The article from Kaspersky “PlugX malware: A good hacker is an apologetic hacker” outlines a summary about it.

That’s it! With this article we set the table for the next article focusing on artifacts that might helps us uncover the hidden traits that were left behind by the attacker actions performed during this scenario. Stay tuned and have fun!



Tagged , , , ,

Intro to American Fuzzy Lop – Fuzzing with ASAN and beyond

In the previous article I showed how easily you could run American Fuzy Lop (AFL) to fuzz an open source target. I used a 64-bit system and used AFL to fuzz the 64-bits version of tcpdump. Today, I’m going to fuzz the 32-bit version of tcpdump. Why the 32-bit version? Because I would like to combine AFL with address sanitizer (ASAN). As written by Michal Zalewski on the “notes_for_asan.txt” document (AFL documentation), ASAN requires a lot of virtual memory. In certain circumstances, due to some input, it could cause a lot of memory consumption and cause system instability. Therefore, Michal suggests to use ASAN to fuzz a 32-bit target and tell AFL to enforce a memory limit. Other options are possible to combine AFL with ASAN on a 64-bit target such as using the option “ASAN_OPTIONS=hard_rss_limit_mb=N”, but I will keep it simple and try that. And why use ASAN? Because ASAN, is a fast memory error detector, created by Google (Konstantin Serebryany, Derek Bruening, Alexander Potapenko and Dmitry Vyukov) and released in 2011 [2][7], that is part of the LLVM compiler infrastructure. It performs compile-time instrumentation and uses a runtime library that replaces memory management functions such as malloc and free with custom code. With these properties, while fuzzing with AFL, you increase the likelihood of finding memory corruption errors such as heap buffer overflow, memory leaks, user -after-free and others [1]. The ASAN custom code that replaces the memory management functions use a shadow memory technique that keeps track of all the malloc-ed and free-ed regions and marks the areas of memory around it as red zones, i.e. it poisons the memory regions surrounding the malloc-ed and free-ed regions [4]. Then while fuzzing, if the input causes some sort out-of-bounds read or write, ASAN would catch that.

So, to use AFL with ASAN, instead of using afl-gcc compiler wrapper, like I did in previous article, I need to use the afl-clang-fast compiler which is a wrapper for Clang compiler. Noteworthy, if you are not leveraging ASAN, you should use the AFL Clang compiler because it is much faster than the AFL GCC compiler.

So, let’s start. First, I start by installing all dependencies for the 32-bit architecture. This step works well on a freshly installed 64-bit system, on an existing system you might need to resolve some dependencies between the different architectures.

Then, like in the previous article, I’m going to download AFL and compile it, but this time I also need to compile the LLVM mode support and give it the correct flags and version of Clang.

Next, to compile the 32-bit version of tcpdump, I also need to compile the 32-bit version of libpcap. At the time of this writing the last version is 1.8.1.  When compiling libpcap I use the “-m32” flag so it produces 32-bits code.  When compiling tcpdump, I add the ASAN flags “-fsanitize=address” and “-fno-omit-frame-pointer” to add the AddressSanitizer run-time library and get better stack traces reports, respectively [3].

After that, I’m going to use the same corpus as the one outlined in the previous article and will put the corpus into a ramdisk. This is not a mandatory step but rather a good practice due to the fact that fuzzing is I/O intensive and could, eventually, reduce the lifetime of your hard drive specially with SSD’s.

Then, start fuzzing! When you start AFL for the first time, you might want to start it without screen, to make sure it executes properly because AFL might printout instructions about tuning that needs to be performed on the operating system such as checking core_pattern and the CPU scaling governor. After having the master node running, I start the slave nodes. In this case I’m using a system with 32 cores, so I should be able to run 32 instances of AFL.

AFL will start reading the different test cases from the input directory, and fuzz them using the different deterministic and non-deterministic stages, find new test cases and queue them for future stage rounds. To check the status of the fuzzing session across the different, I could use afl-whatsup. After a while, you should, hopefully, start to see crashes on some of the fuzzer instances. Then you might want to start looking at the crashes across the different fuzzing sessions. Picture below illustrates a quick way to run the crash reports and grep for the ERROR line.

To get more information about were the crash happened and the stack trace, you could tell ASAN to output debug symbols information by using the options “ASAN_OPTIONS=symbolize=1” and “ASAN_SYMBOLIZER_PATH=/usr/bin/llvm-symbolizer-3.5”. In the picture below I run the crash files with ASAN options so I could see symbols.

Before I continue, it is worth to mention that apart of running AFL with ASAN you could instead instrument the binary with MemorySanitizer (MSan). MSan is another address sanitizer that helps detect uninitialized memory reads in C/C++ programs which occur when a program tries to read allocated memory before it is written. For sake of brevity I’m only showing how to instrument tcpdump with MSAN and run a crash file report with symbolized stack trace for debugging purposes. MSAN needs a newer version of LLVM compiler infrastructure.

So, after you find a crash condition, what are the next steps? You report the bug to the vendor, maintainer or entity and provide them all the details. Then, they might work with you to find the root cause and finally they fix the code for the greater good of overall security. However, another path, instead of just reporting the fault, is finding the exploitability of a crash condition. This is another important aspect of the vulnerability analysis process, and nowadays the economical and professional incentives to perform this work are significant. However, in my opinion, this path is much harder. With the race between bugs being found and security measures being adopted by the different operating systems, the depth of knowledge that one needs in order to be able to triage the crash, likely produced by a fuzzer, debug the program, perform code analysis, understand its internals, assess its state when it crashes and determine if is there is any chance of exploitability under the operating system environment where crash occurred, is a remarkable and unique skill to have.

However, even if you are not the one to have those kind of skills, like myself, you could at least start to understand if by controlling the input allows you to have different outcomes. For example, are you able to, somehow, control the value on the memory address that causes a segmentation fault?  AFL can help determine this to some degree. With AFL you can use one crash file and use the “crash exploration” mode which is also known as the “peruvian were-rabbit” mode. Essentially you feed AFL with the test case that causes a crash condition and AFL will quickly determine the different code paths that can be reached while maintain the crash state. With this AFL will generate a number of crash files that could help you understand to which degree the input has control over the fault address.

In this case, for illustration purposes, I executed AFL in crash exploration mode using as input the file “stack-buffer-overflow-printf_common-hncp-cve2017-13044.pcap”. This is a packet capture that triggers CVE-2017-13044 : HNCP parser in tcpdump before 4.9.2 has a buffer over-read in print-hncp.c:dhcpv4_print(). After a few minutes, AFL was able to determine 42 unique crashes. This means 42 unique code paths that lead to the same crash state.

Quickly looking at the ASAN reports we can obtain two crashes that overrun the buffer boundary and allow us to READ different sizes of adjacent memory.

Picture below shows the ASAN stack trace report with Symbols when running the packet capture that triggers the CVE-2017-13044 on tcpdump version 4.9.0.

So, after that, in an attempt to understand more, you could identify the function and look at the tcpdump source code and also start to debug it with a debugger. Other than that I wanted to understand if by changing a particular byte in the input file I could control something. Therefore I mapped the packet capture to the protocol in question (HNCP) which is described in RFC7788. After some trial and error, it seems there are a 2-bytes in the “Prefix Length” field that cause the buffer over read condition. The picture below shows the hex listing and mapping for the two crash conditions related to CVE-2017-13044. On the packet from the left, setting the HNCP “Prefix Length” on any value between 0x1f and 0x98 will make the ndo_printf() print stack contents. Other values don’t produce the leak. On the packet from the right, the range is from 0x25 to 0x98. So, I could see that I could leak at least 2 memory addresses from the stack but I couldn’t not find a way to control what is printed. Nonetheless, perhaps this packet capture could be merged with another one that has another bug that allows you to control, somehow, the saved return pointer and use the leak to bypass ASLR.

The printscreen below shows the output of GDB (with PEDA plugin) when setting a breakpoint on the function responsible to print the fault address. The debugging session was done with tcpdump compiled without ASAN. Noteworthy if the program is compiled with ASAN and you want to load it into GDB and view the program state before the ASAN crash report you could set a break point on “__asan_report_error”.

That’s it for today. this was the second episode about American Fuzzy Lop. I presented a small step-by-step guide on how you could start using AFL with the LLVM compiler infrastructure. Plus, among other things, leverage the address sanitizers such as ASAN or MSAN to fuzz an open source target and catch a bigger variety of bugs related to memory corruption. In addition, we started to look at the crashes and with small steps understand if it was exploitable.

Using the various techniques described in this article, during the last weeks, I fuzzed couple of tools (tcpdump, affutils, libevt and sleuthkit) and found some faults that I reported to the respective maintainers. I will update the below list as soon as all the faults are disclosed and acknowledged.

  • CVE-2018-8050 : The af_get_page() function in lib/afflib_pages.cpp in AFFLIB (aka AFFLIBv3) through 3.7.16 allows remote attackers to cause a denial of service (segmentation fault) via a corrupt AFF image that triggers an unexpected pagesize value. I reported this bug to Phillip Hellewell who promptly analyzed the crash condition and quickly released a fix for it.
  • CVE-2018-8754 : The libevt_record_values_read_event() function in libevt_record_values.c in libevt before 2018-03-17 does not properly check for out-of-bounds values of user SID data size, strings size, or data size. I reported this bug to Joachim Metz who promptly analyzed the crash condition and quickly released a fix for it.

Credits: Thanks to Aleksey Cherepanov for his enthusiasm, valued support and willingness to help me with my never ending questions and Rui Reis for his ideas and support to make this article better.


[4] (page 29)
Vulnerability Discovery and Triage Automation training material from Richard Johnson

Tagged , , , , , , , , , ,

Intro to American Fuzzy Lop – Fuzzing in 5 steps

After attending the workshop on “Vulnerability Discovery and Triage Automation” given by the well-known security researcher Richard Johnson, I rolled up my sleeves and started practicing. Essentially, looking at the different tools and techniques that were shared and reading more about the different topics, specially related on how to perform fuzzing. Not being a coder or not having a software engineering background, some of the topics are harder to grasp, but never late to learn new things.

In this article I want to share a step-by-step guide on how to run American Fuzzy Lop (AFL) to fuzz an open source target. AFL was written by the renowned and respected Polish security researcher Michal Zalewski aka lcamtuf. AFL is a powerful, versatile and open source fuzzer written in C and assembly. A brilliant piece of software engineering that reduces significantly the entry-level and amount of knowledge needed for someone that doesn’t have a software engineering background to perform scalable and intelligent fuzzing against a variety of targets. Fuzzing is the process by which you feed random inputs to a target and observe its execution with the goal to find a condition that will lead to a bug. To perform fuzzing, AFL uses compile-time instrumentation and genetic algorithms combined with more traditional fuzzing techniques that allow it to fuzz targets without any knowledge whatsoever about what the target expects. This makes it an ideal tool to fuzz a broad number of targets. It is especially suited to fuzz targets that take as input a particular file format. It can detect changes to the program control flow and constantly adapt, calibrate and find new test cases that find a new path and grow the code coverage thus increasing the likelihood of finding a condition that trigger a bug.  Code coverage is an important metric in fuzzing and dynamic analysis because allows you to get visibility into the amount of code that was executed, you want to reach code paths that are deep, hard to reach and perhaps less tested. Luckily, AFL algorithm does a good job at it. Bottom line, as written by Michal, “The fuzzer is thoroughly tested to deliver out-of-the-box performance far superior to blind fuzzing or coverage-only tools.

So, anyone can start using AFL against open source code in 5 steps:

  1. Download, compile and install AFL.
  2. Download, instrument and install target.
  3. Get data that will be used to feed AFL, reduce it in quantity and increase the quality (code coverage).
  4. Create ramdisk and store AFL fuzzing session input and output directories in there.
  5. Start fuzzing and let AFL perform its magic.

At the time of this writing, the latest version of AFL is 2.52b and can be downloaded from Michal website and easily installed. The below guide was executed on a system running Ubuntu 16.04.

Then, you need a target. I decided to use tcpump as a target of my fuzzing session. Mainly because it is easy to find packet captures to be used as input for fuzzing and is easy to run AFL against it. But better is that by looking at tcpdump release notes I could see that version 4.9.0, released on January 18, 2017, is likely an interesting target to practice because many bugs have been found in this version. So, I downloaded tcpdump source code, extracted its contents, defined the environment variables for the C compiler command to use afl-gcc, which is a replacement for GCC that performs the instrumentation of the target and, finally, compiled it and installed it.

Ok, now that I have the target instrumented with AFL and ready to be fuzzed, I need to prepare my corpus of input files that are going to be mangled, changed and what not by AFL. After collecting around 8 gigabytes of packet captures from different sources  containing different protocols – In case you don’t have a easy way to collect packet captures you could download a variety of them from WireShark sample captures website.– , I used editcap to reduce the size of each capture file. Essentially, for each one of the packet captures I had, I exported it in chunks of 4 packets each. This allowed me to create a corpus of packet capture files with reduced size which is a key factor for AFL and its fuzzing process. Size does matter, Michal in the README file writes that files under 1024 bytes are ideal, not mandatory tough.

Following that, I could analyze each one of the pcaps with afl-cmin.  The afl-cmin tool is part of AFL toolkit and allows to analyze a large amount of corpus data and identify the different files that exercise the most different code paths. For example, I started a minimization corpus session against 1.5M files and afl-cmin concluded that only 273 files are needed in order to exercise the same quantity of code coverage.

After having the corpus minimized, I prepared the input and output directories to run the fuzzing session onto a Ramdisk.

Then I copied the reduced corpus to the Ramdisk and could start the fuzzing.  In my case I have a 64-bits system with 4 CPU cores and because AFL uses 1 CPU core per session, I can parallelize AFL jobs across the cores I have. First starting the master instance. I use  screen manager to start the session in a new terminal, then invoke afl-fuzz with “-i” suffix to indicate the input directory where the corpus is, then the “-o” suffix to specify the directories where AFL will save its work, next the “-M” flag is used to specify this is the master node followed by a unique name. Finally, the command line that I want to run which is separated by the “–” suffix. In this case I use tcpdump with verbose and headers output. The last parameter is “@@” which is used by AFL to replace the input file with the test case.

When you start AFL for the first time, you might want to start it without screen, to make sure it executes properly because AFL might printout instructions about tuning that needs to be performed on the operating system such as checking core_pattern and the CPU scaling governor. After having the master node running, I start the slave nodes.

AFL will start reading the different test cases from the input directory, and fuzz them using the different deterministic and non-deterministic stages, find new test cases and queue them for future stage rounds. To check the status of the fuzzing session across the different, I could use afl-whatsup.

For each one of the fuzzer nodes you start within your fuzzing session, AFL will create a very simple directory structure. Inside, for each fuzzer node, you can see the crashes, hangs and a queue directory. The name is explicit for its intent.

So, after you started the AFL fuzzing session, when should you stop? In the AFL documentation, Michal writes that you should consider three things. One is that the master node should at least show “1” in the cycles done metric in the status screen. The master node plays an important role in the fuzzing session and is the one that performs the deterministic stages, This means the fuzzer has completed the different deterministic stages across all the discovered favored paths. Second case is that the “pending” metric on the status screen shows “0”, Finally the third case is when no new paths have been discovered for quite some time e.g., a week, indicating that likely there isn’t much to find.

Ok, the fuzzers are running …  now we wait and if you get lucky you might start seeing an increase in the number of “Crashes found”. After couple hours of fuzzing I started to see the number of hangs slowly increasing.  After analyzing some of the hangs, I came across:

  • CVE-2017-13044: The HNCP parser in tcpdump before 4.9.2 has a buffer over-read in print-hncp.c:dhcpv4_print().  This bug was initially discovered by Bhargava Shastry and fixed by the tcpdump maintainers. Taking a quick look seems the bug would allow an attacker to leak data from the stack frame.
  • CVE-2017-12989: The RESP parser in tcpdump before 4.9.2 could enter an infinite loop due to a bug in print-resp.c:resp_get_length(). This bug was initially discovered by Otto Airamo & Antti Levomäki and fixed by the tcpdump maintainers. Taking a quick look seems the bug causes an infinite loop.

That’s it for today. In this article I presented a small step-by step guide on how to start using AFL to fuzz an open source target. In the next article (as soon as the time permits) let’s see how you could use other compilers to improve AFL speed and use other tools to catch a bigger variety of memory corruption bugs.

Stay tuned and have fun!


AFL Documentation that comes in the AFL tarball
Fuzzing workflows; a fuzz job from start to finish
Collection of papers on Fuzzing

Tagged , , , , ,

Malware Analysis – PlugX

[The PlugX malware family has always intrigued me. I was curious to look at one variant. Going over the Internet and the research articles and blogs about it I came across the research made by Fabien Perigaud. From here I got an old PlugX builder. Then I set a lab that allowed me to get insight about how an attacker would operate a PlugX campaign. In this post, l will cover a brief overview about the PlugX builder, analyze and debug the malware installation and do a quick look at the C2 traffic. ~LR]

PlugX is commonly used by different threat groups on targeted attacks. PlugX is also refered as KORPLUG, SOGU, DestroyRAT and is a modular backdoor that is designed to rely on the execution of signed and legitimated executables to load malicious code. PlugX, normally has three main components, a DLL, an encrypted binary file and a legitimate and signed executable that is used to load the malware using a technique known as DLL search order hijacking. But let’s start with a quick overview about the builder.

The patched builder, MD5 6aad032a084de893b0e8184c17f0376a, is an English version, from Q3 2013,  of the featured-rich and modular command & control interface for PlugX that allows an operator to:

  • Build payloads, set campaigns and define the preferred method for the compromised hosts to check-in and communicate with the controller.
  • Proxy connections and build a tiered C2 communication model.
  • Define persistence mechanisms and its attributes.
  • Set the process(s) to be injected with the payload.
  • Define a schedule for the C2 call backs.
  • Enable keylogging and screen capture.
  • Manage compromises systems per campaign.

Then for each compromised system, the operator has extensive capabilities to interact with the systems over the controller that includes the following modules:

  • Disk module allows the operator to write, read, upload, download and execute files.
  • Networking browser module allows the operator to browse network connections and connect to another system via SMB.
  • Process module to enumerate, kill and list loaded modules per process.
  • Services module allows the operator to enumerate, start, stop and changing booting properties
  • Registry module allows the operator to browse the registry and create, delete or modify keys.
  • Netstat module allows the operator to enumerate TCP and UDP network connections and the associated processes
  • Capture module allows the operator to perform screen captures
  • Control plugin allows the operator to view or remote control the compromised system in a similar way like VNC.
  • Shell module allows the operator to get a command line shell on the compromised system.
  • PortMap module allows the operator to establish port forwarding rules.
  • SQL module allows the operator to connect to SQL servers and execute SQL statements.
  • Option module allows the operator to shut down, reboot, lock, log-off or send message boxes.
  • Keylogger module captures keystrokes per process including window titles.

The picture below shows the Plug-X C2 interface.

So, with this we used the builder functionality to define the different settings specifying C2 comms password, campaign, mutex, IP addresses, installation properties, injected binaries, schedule for call-back, etc. Then we build our payload. The PlugX binary produced by this version of the builder (LZ 2013-8-18) is a self-extracting RAR archive that contains three files. This is sometimes referred in the literature as the PlugX trinity payload. Executing the self-extracting RAR archive will drop the three files to the directory chosen during the process. In this case “%AUTO%/RasTls”. The files are: A legitimate signed executable from Kaspersky AV solution named “avp.exe”, MD5 e26d04cecd6c7c71cfbb3f335875bc31, which is susceptible to DLL search order hijacking . The file “avp.exe” when executed will load the second file: “ushata.dll”, MD5 728fe666b673c781f5a018490a7a412a, which in this case is a DLL crafted by the PlugX builder which on is turn will load the third file. The third file: “ushata.DLL.818”, MD5 “21078990300b4cdb6149dbd95dff146f” contains obfuscated and packed shellcode.

So, let’s look at the mechanics of what happens when the self-extracting archive is executed. The three files are extracted to a temporary directory and “avp.exe” is executed. The “avp.exe” when executed will load “ushata.dll” from the running directory due to the DLL search order hijacking using Kernel32.LoadLibrary API.

Then “ushata.dll” DLL entry point is executed. The DLL entry point contains code that verifies if the system date is equal or higher than 20130808. If yes it will get a handle to “ushata.DLL.818”, reads its contents into memory and changes the memory address segment permissions to RWX using Kernel32.VirtualProtect API. Finally, returns to the first instruction of the loaded file (shellcode). The file “ushata.DLL.818” contains obfuscated shellcode. The picture below shows the beginning of the obfuscated shellcode.

The shellcode unpacks itself using a custom algorithm. This shellcode contains position independent code. Figure below shows the unpacked shellcode.

The shellcode starts by locating the kernel32.dll address by accessing the Thread Information Block (TIB) that contains a pointer to the Process Environment Block (PEB) structure. Figure below shows a snippet of the shellcode that contains the different sequence of assembly instructions for the code to find the Kernel32.dll.

It then reads kernel32.dll export table to locate the desired Windows API’s by comparing them with stacked strings. Then, the shellcode decompresses a DLL (offset 0x784) MD5 333e2767c8e575fbbb1c47147b9f9643, into memory using the LZNT1 algorithm by leveraging ntdll.dll.RtlDecompressBuffer API. The DLL contains the PE header replaced with the “XV” value. Restoring the PE header signature allows us to recover the malicious DLL.

Next, the payload will start performing different actions to achieve persistence. On Windows 7 and beyond, PlugX creates a folder “%ProgramData%\RasTl” where “RasTl” matches the installation settings defined in the builder. Then, it changes the folder attributes to “SYSTEM|HIDDEN” using the SetFileAttributesW API. Next, copies its three components into the folder and sets all files with the “SYSTEM|HIDDEN” attribute.

The payload also modifies the timestamps of the created directory and files with the timestamps obtained from ntdll.dll using the SetFileTime API.

Then it creates the service “RasTl” where the ImagePath points to “%ProgramData%\RasTl\avp.exe”

If the malware fails to start the just installed service, it will delete it and then it will create a persistence mechanism in the registry by setting the registry value “C:\ProgramData\RasTls\avp.exe” to the key “HKLM\SOFTWARE\Classes\SOFTWARE\Microsoft\Windows\CurrentVersion\Run\RasTls” using the RegSetValueExW API.

If the builder options had the Keylogger functionality enabled, then it may create a file with a random name such as “%ProgramData%\RasTl\rjowfhxnzmdknsixtx” that stores the key strokes. If the payload has been built with Screen capture functionality, it may create the folder “%ProgramData%\RasTl \RasTl\Screen” to store JPG images in the format <datetime>.jpg that are taken at the frequency specified during the build process. The payload may also create the file “%ProgramData%\DEBUG.LOG” that contains debugging information about its execution (also interesting that during execution the malware outputs debug messages about what is happening using the OutputDebugString API. This messages could be viewed with DebugView from SysInternals). The malicious code completes its mission by starting a new instance of “svchost.exe” and then injects the malicious code into svchost.exe process address space using process hollowing technique. The pictures below shows the first step of the process hollowing technique where the payload creates a new “svchost.exe” instance in SUSPENDED state.

and then uses WriteProcessMemory API to inject the malicious payload

Then the main thread, which is still in suspended state, is changed in order to point to the entry point of the new image base using the SetThreadContext API. Finally, the ResumeThread API is invoked and the malicious code starts executing. The malware also has the capabilities to bypass User Account Control (UAC) if needed. From this moment onward, the control is passed over “svchost.exe” and Plug-X starts doing its thing. In this case we have the builder so we know the settings which were defined during building process. However, we would like to understand how could we extract the configuration settings. During Black Hat 2014, Takahiro Haruyama and Hiroshi Suzuki gave a presentation titled “I know You Want Me – Unplugging PlugX” where the authors go to great length analyzing a variety of PlugX samples, its evolution and categorizing them into threat groups. But better is that the Takahiro released a set of PlugX parsers for the different types of PlugX samples i.e, Type I, Type II and Type III. How can we use this parser? The one we are dealing in this article is considered a PlugX type II. To dump the configuration, we need to use Immunity Debugger and use the Python API. We need to place the “” file into the “PyCommands” folder inside Immunity Debugger installation path. Then attached the debugger to the infected process e.g, “svchost.exe” and run the plugin. The plugin will dump the configuration settings and will also extract the decompressed DLL

We can see that this parser is able to find the injected shellcode, decode its configuration and all the settings an attacker would set on the builder and also dump the injected DLL which contains the core functionality of the malware.

In terms of networking, as observed in the PlugX controller, the malware can be configured to speak with a controller using several network protocols. In this case we configured it to speak using HTTP on port 80. The network traffic contains a 16-byte header followed by a payload. The header is encoded with a custom routine and the payload is encoded and compressed with LZNT1. Far from a comprehensive analysis we launched a Shell prompt from the controller, typed command “ipconfig” and observed the network traffic. In parallel, we attached a debugger to “svchost.exe” and set breakpoints: on Ws2_32.dll!WSASend and Ws2_32.dll!WSARecv to capture the packets ; on ntdll.dll!RtlCompressBuffer and ntdll.dll!RtlDecompressBuffer to view the data before and after compression. ; On custom encoding routine to view the data before and after. The figure below shows a disassemble listing of the custom encoding routine.

So, from a debugger view, with the right breakpoints we could start to observe what is happening. In the picture below, on the left-hand side it shows the packet before encoding and compression. It contains a 16-byte header, where the first 4-bytes are the key for the custom encoding routine. The next 4-bytes are the flags which contain the commands/plugins being used. Then the next 4-bytes is the size. After the header there is the payload which in this case contains is output of the ipconfig.exe command. On the right-hand side, we have the packet after encoding and compressing. It contains the 16-byte header encoded following by the payload encoded and compressed.

Then, the malware uses WSASend API to send the traffic.

Capturing the traffic, we can observe the same data.

On the controller side, when the packet arrives, the header will be decoded and then the payload will be decoded and decompressed. Finally, the output is showed to the operator.

Now that we started to understand how C2 traffic is handled, we can capture it and decode it.  Kyle Creyts has created a PlugX decoder that supports PCAP’s. The decoder supports decryption of PlugX Type I.But Fabien Perigaud reversed the Type II algorithm and implemented it in python. If we combine Kyle’s work with the work from Takahiro Haruyama and Fabien Perigaud we could create a PCAP parser to extract PlugX Type II and Type III. Below illustrates a proof-of-concept for this exercise against 1 packet. We captured the traffic and then used a small python script to decrypt a packet. No dependencies on Windows because it uses the herrcore’s standalone LZNT1 implementation that is based on the one from the ChopShop protocol analysis and decoder framework by MITRE.

That’s it for today! We build a lab with a PlugX controller, got a view on its capabilities. Then we looked at the malware installation and debugged it in order to find and interpret some of its mechanics such as DLL search order hijacking, obfuscated shellcode, persistence mechanism and process hollowing. Then, we used a readily available parser to dump its configuration from memory. Finally, we briefly looked the way the malware communicates with the C2 and created a small script to decode the traffic. Now, with such environment ready, in a controlled and isolated lab, we can further simulate different tools and techniques and observe how an attacker would operate compromised systems. Then we can learn, practice at our own pace and look behind the scenes to better understand attack methods and ideally find and implement countermeasures.

Analysis of a PlugX malware variant used for targeted attacks by
Operation Cloud Hopper by PWC
PlugX Payload Extraction by Kevin O’Reilly
Other than the authors and articles cited troughout the article, a fantastic compilation about PlugX articles and papers since 2011 is available here.

Credits: Thanks to Michael Bailey who showed me new techniques on how to deal with shellcode which I will likely cover on a post soon.

Tagged , , , , ,

Digital Forensics – Artifacts of interactive sessions

In this article I would like to go over some of the digital forensic artifacts that are likely to be useful on your quest to find answers to investigative questions. Specially, when conducting digital forensics and incident response on security incidents that you know the attacker performed its actions while logged in interactively into a Microsoft Windows systems. Normally, one of the first things I look is the Windows Event logs. When properly configured they are a treasure trove of information, but in this article, I want to focus on artifacts that can be useful even if an attacker attempts to cover his tracks by deleting the Event Logs.

Let’s start with ShellBags!

To improve the customer experience, Microsoft operating systems stores folder settings in the registry. If you open a folder, resize its dimensions, close it and open it again, did you notice that Windows restored the view you had? Yep, that’s ShellBags in action. This information is stored in the user profile hive “NTUSER.dat” within the directory “C:\Users\%Username%\” and in the hive “UsrClass.dat” which is stored at “%LocalAppData%\Microsoft\Windows”. When a profile is loaded into the registry, both hives are mounted into the HKEY_USERS and then then linked to the root key HKEY_CURRENT_USER and HKEY_CURRENT_USER\Software\Classes respectively. If you are curious, you can see where the different files are loaded by looking at the registry key “HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\hivelist”. On Windows XP and 2003 the ShellBags registry keys are stored at HKEY_USERS\{SID}\Software\​Microsoft\Windows\Shell\ and HKEY_USERS\{SID}\Software\​Microsoft\Windows\ShellNoRoam\.  On Windows 7 and beyond the ShellBags registry keys are stored at “HKEY_USERS\{SID}_Classes\​Local Settings\Software\​Microsoft\Windows\Shell\”.

Why are ShellBags relevant?

Well, this particular artifact allows us to get visibility about the intent or knowledge that a user or an attacker had when accessing or browsing directories and, when. Even if the directory does no longer exists. For example, an attacker that connects to a system using Remote Desktop and accesses a directory where his toolkit is stored. Or an unhappy employee that accesses a network share containing business data or intellectual property weeks before his last day and places this information on a USB drive. ShellBags artifacts can help us understand if such actions were performed. So, when you obtain the NTUSER.dat and UsrClass.dat hives you could parse it and then placed events into a timeline.  When corroborated with other artifacts, the incident response team can reconstruct user activities that were performed interactively and understand what happened and when.

Which tools can we use to parse ShellBags?

I like to use RegRipper from Harlan Carvey, ShellBags Explorer from Eric Zimmerman or Sbags from Willi Ballenthin. The below picture shows an example of using Willi’s tool to parse the ShellBags information from the NTUSER.dat and UsrClass.dat hives. As an example, this illustration shows that the attacker accessed several network folders within SYSVOL and also accessed “c:\Windows\Temp” folder.

To give you context, why I’m showing you this particular illustration of accessing the SYSVOL folder, is because they contain Active Directory Group Policy preference files that in some circumstances might contain valid domain credentials that can be easily decrypted. This is a known technique used by attackers to obtain credentials and likely to occur in the beginning of an incident. Searching for passwords in files such as these are simple ways for attackers to get credentials for service or administrative accounts without executing credential harvesting tools.

Next artifact on our list, JumpLists!

Once again, to increase the customer experience and accelerate the workflow, Microsoft introduced on Windows 7 the possibility to allow a user to access a list of recently used applications and files. This is done by enabling the feature to store and display recently opened programs and items in the Start Menu and the taskbar. There are two files that store JumpLists information. One is the {AppId}.automaticDestination-ms and the other is {AppId}.customDestination-ms where {AppId} corresponds to a 16 hex string that uniquely identifies the application and is calculated based on application path CRC64 with a few oddities. These files are stored in the folder “C:\Users\%USERNAME%\AppData\​Roaming\Microsoft\Windows\​Recent\AutomaticDestinations” and “C:\Users\%USERNAME%\AppData\​Roaming\Microsoft\Windows\​Recent\CustomDestinations”. The folder AutomaticDestinations contain files {16hexchars}.automaticDestination-ms and these files are generated by common operating system applications and stored in a in Shell Link Binary File Format known as [MS-SHLLINK] that are encapsulated Inside a Compound File Binary File Format known as MS-CFB or OLE. The folder CustomDestinations contain files {16hexchars}.customDestination-ms and these files are generated by applications installed by the user or scripts there were executed and stored in Shell Link Binary File Format known as [MS-SHLLINK].

Why are JumpLists relevant?

Just like like ShellBags, this artifact allows us to get visibility about the intent or knowledge an attacker had when opening a particular file, launching a particular application or browsing a specific directory during the course of an interactive session. For example, consider an attacker that is operating on a compromised system using Remote Desktop and launches a browser, the JumpList associated with it will contains the most visited or the recently closed website. If the attacker is pivoting between system using the Terminal Services client, the JumpList shows the system that was specified as an argument. If an attacker dumped credentials from memory and saved into a text file and opened it with Notepad, the JumpList will show evidence about it. Essentially, the metadata stored on these JumpList files can be parsed and will show you a chronological list of Most Recently Used (MRU) or Most Frequently Used (MFU) files opened by the user/application. Among other things, the information contains the Standard Information timestamps from the list entry and the time stamps from the file at the time of opening. Furthermore, it shows the original file path and sizes. This information, when placed into a timeline and corroborated with another artifact can give us a clear picture of the actions performed.

Which tools can we use to parse JumpLists?

JumpListsView from NIRSOFT, JumpLister from Mark Waon or JumpLists Explorer from Eric Zimmerman. Below an example of using Eric’s tool to parse the JumpLists files. More specifically the JumpList file that is associated with Notepad. As an example, this illustration shows that an attacker opened the file “C:\Windows\Temp\tmp.txt”with Notepad. It shows when the file was created and the MFT entry. Very useful.

Next artifact, LNK files!

Again, consider an attacker operating on a compromised system using a Remote Desktop session where he dumped the credentials to a text file and then double clicked on the file. This action will result in the creation of the corresponding Windows shortcut file (LNK file). LNK files are Windows Shortcuts. Everyone that has used Windows has created a shortcut of your most favorite folder or program. However, the Windows operating system behind the scenes also keeps track of recently opened files by creating LNK files within the directory “C:\Documents and Settings\%USERNAME%\Recent\”.   The LNK files, like JumpLists, are stored in Shell Link Binary File Format known as [MS-SHLLINK]. When parsed, the LNK file, contains metadata that, among other things, shows the target file Standard Information timestamps, path, size and MFT entry number. This information is maintained even if the target file does no longer exists on the file system. The MFT entry number can be valuable in case the file was recently deleted and you would like to attempt to recover by carving it from the file system.

Which tools can we use to parse .LNK files?

Joachim Metz has an utility that to parse the information from the Windows Shortcut files. The utility is installed by default on SIFT workstation. In the illustration below, while analyzing a disk image, we could see that there are several .LNK files created under a particular profile. Knowing that this profile has been used by an attacker you could parse the files. In this case parsing, when parsing the file “tmp.lnk” file we can see the target file “C:\Windows\Temp\tmp.txt”, its size and when was created.

Next artifact, UserAssist!

The UserAssist registry key keeps track of the applications that were executed by a particular user. The data is encoded using ROT-13 substitution cipher and maintained on the registry key HKEY_USERS\{SID}\Software\​Microsoft\Windows\CurrentVersion​\Explorer\UserAssist.

Why is UserAssist relevant?

Consider an attacker operating on a compromised system where he launched “cmd.exe” to launch other Windows built-in commands, or opened the Active Directory Domains and Trusts Snap-in “domain.msc” to gather information about a particular domain, or launched a credential dumper from an odd directory. This action will be tracked by the UserAssist registry key. The registry key will show information about which programs have been executed by a specific user and how frequently. Due to the nature of how timestamps are maintained on registry ie., only the last modified timestamp is kept, this artifact will show when was the last time that a particular application was launched.

Which tools can we use to parse the UserAssist registry keys?

Once again RegRipper from Harlan Carvey is a great choice. Another tool is UserAssist from Didier Stevens. Other method that I often use is to use log2timeline using Windows Registry plugin and then grepping for the UserAssist parser. In this example, we can see that an attacker while operating under a compromised account, executed “cmd,exe”, “notepad.exe”and “mmc.exe”. Now combining these artifacts with the Shellbags, JumpLists and .LNK files, I can start to interpret the results.

Next artifact, RDP Bitmap Cache!

With the release of RDP 5.0 on Windows 2000, Microsoft introduced a persistent bitmap caching mechanism that augmented the bitmap RAM cache. With this mechanism, when you do a Remote Desktop connection, the bitmaps can get stored on disk and are available for the RDP client, allowing it to load them from disk instead of waiting on the latency of the network connection. Of course this was developed with low bandwidth network connections in mind. On Windows 7 and beyond the cache folder is located on “%USERPROFILE%\AppData\Local\Microsoft\Terminal Server Client\Cache\ ” and there two types of cache files. One that contains a .bmc extension and a newer format that was introduced on Windows 7 that follows the naming convention of “cache{4-digits}.bin’. Both files have tiles of 64×64 pixels. The .bmc files support different bits per pixel ranging from 8-bits to 32-bits. The .bin files are always 32-bits and have more capacity and a file can store up to 100Mb of data.

Why are RDP Bitmap cache files relevant?

If an attacker is pivoting between systems in a particular environment and is leveraging Remote Desktop then, on the system where the connection is initiated you could find the bitmap cache that was stored during the attacker Remote Desktop session. After reconstructing the bitmaps, that translate what was being visualized by the attacker, it might be possible to reconstruct the bitmap puzzle and observe what was seen by the attacker while performing the Remote Desktop connections to the compromised systems. A great exercise for people who like puzzles!

Which tools can we use to parse RDP Bitmap Cache files?

Unfortunately, there aren’t many tools available. ANSSI-FR released a RDP Bitmap Cache parser that you could use to extract the bitmaps from the cache files. There was a tool called BmcViewer that was available on a now defunct website and is great tool to parse the .bmc files. The tool doesn’t support the .bin files. If you know how to code, an interesting project might be to develop a parser that allows you to puzzle the tiles.

Finally, combining these artifacts with our traditional file system metadata timeline and other artifacts such as ShimCache, would allows us to further uncover more details. Below an illustration of parsing ShimCache from a memory image using Volatility and the ShimCacheMem plugin written by Fred House. We could see that there are some interesting files. For example “m64.exe” and looking at the adjacent entries we can see that it shows the execution of “notepad.exe”, “p64.exe” and “reg.exe”. Searching for those binaries on our file system timeline uncovers that for example m64.exe is Mimikatz.

That’s it for today! As I wrote in the beginning, the Windows Even Logs are a treasure trove of information when properly configured but If an attacker attempts to cover his tracks by deleting the Event Logs, there are many other artifacts to look for. Combine the artifacts outlined in this article with File system metadata, ShimCache, AMCache, RecentDocs, Browser History, Prefetch, WorldWheelQuery, ComDlg32, RunMRU, and many others and you likely will end up having a good understanding of what happened and when. Happy hunting!

PS: Due to the extensive list of references I decided to attach a text file with links: references. Without them, this article won’t be possible.

Luttgens, J., Pepe, M., Mandia, K. (2014) Incident Response & Computer Forensics, 3rd Edition
Carvey, H. (2011) Windows Registry Forensics: Advanced Digital Forensic Analysis of the Windows Registry, Second Edition
SANS 508 – Advanced Computer Forensics and Incident Response

Tagged , , , , , , , ,

Analysis of a Master Boot Record – EternalPetya

NoPetya or EternalPetya has kept the security community pretty busy last week. A malware specimen that uses a combined arms approach and maximizes its capabilities by using different techniques to sabotage business operations. One aspect of the malware that raised my interest was the ability to overwrite the Master Boot Record (MBR) and launch a custom bootloader. This article shows my approach to extract the MBR using digital forensic techniques and then analyze the MBR using Bochs. Before we roll up our sleeves let’s do a quick review on how the MBR is used by today’s computers during the boot process.

The computers that rely on BIOS flash memory instead of the new EFI standard, when they boot, the BIOS code is executed and, among other things, the code performs a series of routines that perform hardware checks i.e., Power-On-Self-Tests (POST). Then, the BIOS attempts to find a bootable device. If the bootable device is a hard drive, the BIOS reads the sector 1, track 0, head 0 and if contains a valid MBR (valid means that the sector ends with bytes 0xAA55) it will load that sector into a fixed memory location. By convention the code will be loaded into the real-mode address 0000:7c00. Then, the instruction pointer register is transferred into that memory location and the CPU will start executing the MBR code. What happens next is dependable on the MBR implementation code i.e., different operating systems have different MBR code  Nonetheless, the code needs to fit in the 512-bytes available at disk sector. The MBR follows a standard and its structure contains executable code, the partition table (64-bytes) with the locations of the primary partitions and finally 2-bytes with 0xAA55 signature. This means only 446-bytes are available to implement a MBR. In the Microsoft world, when the MBR code is executed, its role is to find an active partition, read its first sector, which contains the VBR code, load it into memory and transfer execution into it. The VBR is a bootloader program that will find the Windows BootMgr program and executes it All this happens in 16-bits real-mode.

Now, that we have a brief overview about the boot process, how can we extract and analyze the MBR? More specifically the MBR that is used by EternalPetya? Well, we infect a victim machine on a controlled and isolated environment. We know that EternalPetya main component is a DLL and we can launch it and infect a Windows machine by running “rundll32.exe petya.dll, #1 10”. Our setup consisted of 2 Virtual Machines. One running with Windows 7 and another running REMnux. We created a snapshot of the victim machine before the infection. Then executed the malware. Following that, we waited 10 minutes for the infection to complete. The scheduled task created by the malware restarted the operating system and a ransom note appeared. Then, I shutdown the Windows 7 virtual machine and used vmware-vdiskmanager.exe utility to create a single VMDK file from the disk state before and after the infection. Next, I moved the VMDK files to a Linux machine where I used QEMU to convert the VMDK images to RAW format.

Following that I could start the analysis and look at the MBR differences. The picture below illustrates the difference between the original MBR and the EternalPetya MBR. On the right side you have the EternalPetya MBR, the first 147 bytes (0x00 through 0x92) contain executable code.  The last 66 bytes (0x1be through 0x1fd) contain the partition table and are equal to the original MBR.

So, we are interested in the code execution instructions. We can start by extracting the MBR into a binary file and convert it to assembly instructions. This can be done using radare, objdump or ndisasm. Now, this is the hard part of the analysis. Read the assembly instructions and understand what it does. We can look at the instructions and perform static analysis but we can also perform dynamic analysis by running the MBR code, combining both worlds we will have better understanding – or at least we can try.

To perform dynamic analysis of the MBR code we will use Bochs. Bochs is an open source, fully fledged x86 emulator. Originally written by Kevin Lawton in 1994 is still being actively maintained today and last April version 2.6.9 was released. Bochs brings a CLI and GUI debugger and is very useful to debug our MBR code.  In addition, Bochs can be integrated with IDA PRO and Radare.  You can download Bochs from here.  In our case, we want to use Bochs to dynamically debug our MBR code. For that we need a configuration file called Bochsrc which is used to specify different options such as the disk image and also the CHS parameters for the disk. This article from Hex-Rays contains a how-to on how to integrate Bochs with IDA PRO. At the end of the article there is the file that you can download. We will use these files to use Bochs standalone or combined with IDA PRO in case you have it. The Bochsrc file that comes with the ZIP file contains options that are deprecated on the newer Bochs version. The picture below shows the Bochsrc options I used. The Bochs user guide documents this file well.

Then you can try your configuration setup and launch Bochs. If you have IDA PRO then you can use this guide starting from step 6 in order to integrate it with IDA PRO. If all is set up, the debugging session will open and stop at the first instruction from its own BIOS code at memory address F000:FFF0.  Why this address? You can read this and many other low level things in the outstanding work from Daniel B. Sedory.

Uncomment the last line from the Bochsrc configuration file, to tell Bochs to use the Enhanced Debugger. For further references, you can read the “How to DEBUG System Code using The Bochs Emulator on a Windows™ PC” article. Start Bochs again and the GUI will show up. You can load the stack view by pressing F2. Then set a breakpoint where the MBR code will be loaded by issuing the command “lb 0x7c00” and then the “c” to continue the debugging session.

Now we can look at the code, step into the instructions, inspect the registers and stack. After some back and forth with the debugger I created the following listing with my interpretation of some of the instructions.

Bottom line, the MBR code will perform a loop that uses BIOS interrupt 0x13 function 0x42 to read data starting at sector 1 of the hard-drive. The loop copies 8880 (0x22af) bytes of data into memory location 0x8000. When the copy is done, the execution is transferred to the address 0x8000 by performing a far jump and the malicious bootloader is executed. The malicious bootloader code has been uploaded by Matthieu Suiche to Virus Total here. You can also extract it from the hard drive by extracting the sector 1 through 18 or better using the commands from the following picture. Then you can perform static and dynamic analysis.

The 16-bits bootloader code is harder to analyze than the MBR code but it is based on the Petya ransomware code from 2016. In this great article, from Hasherezade, she analyzes both Petya and EternalPetya bootloader using IDA PRO. When you use Bochs integrated with IDA PRO disassembler and debugger, the analysis is more accessible due to the powerful combination.

That’s it for today – Entering the world of real-mode execution on x86 is quite interesting. Analyzing code that relies on BIOS services such as software interrupts to perform different operations like reading from disk or writing to screen or, accessing the memory through segments is revealing. What we learned today might be a starting point to start looking at bootkits that are beneath the operating system and subvert the MBR boot sequence.

Have fun!


Windows Internals, Sixth Edition, Part 2 By: Mark E. Russinovich, David A. Solomon, and Alex Ionescu

Various articles written by Daniel B. Sedory :

Tagged , , ,