Detecting malware capabilities in superblocks using static binary analysis

Abstract

Malware, or malicious software, is a central piece to the success of most cyber attacks since often times it is the element that is programmed and used to carry out the objectives of the threat actors once they gain access to a system. This has made it vital for cybersecurity analysts to study malware and accurately categorize it, so that they remain a step ahead of the attackers, and are able to respond quickly in the case of time-sensitive cyber breach situations. The large volume of malware that has created a necessity for tools that automate the process of analyzing new malware, and especially tools aimed at extracting the behavior and capabilities of the encountered malware. One of the most widely used tools for this purpose is the capa tool by Mandiant’s FLARE team. This thesis aims to improve FLARE’s capa malware capability extraction tool by increasing the accuracy of its capability extractions and categorization. Currently, it extract features from instructions, basic blocks, and functions, and then matches them with the rules that apply to each individual scope. This extraction method presents an issue of false negatives if a rule is written to be applied on basic blocks, but the features to-be-matched end up being split into two or more basic blocks by the compiler (in the case of the malware author adding error-checking code between two related malicious API calls for instance); as well as potential false positives if the previous rule’s matching scope is set to function instead (to circumvent the stated limitation), which might lead to completely unrelated features being paired up together in the case of large functions. The identified limitation is relevant because usage of if-else statements for error checking are common in software in general and malware specifically, and those are a major reason why semantically-related code would end up being segmented into multiple basic blocks. The work adds a new "superblock" matching scope to capa that makes it possible to extract malware capabilities whose constituent features are spread across a number of basic blocks in direct sequence, therefore reducing the possibility of false positives and false negatives when matching some rules, as previously described. Using the newly-introduced scope, we were able to rewrite an existing capa rule intended for detecting RC4 encryption logic, and have it match the relevant logic on the superblock scope as opposed to the old function scope. By doing so, we were able to retain the tool’s true positive rate, while eliminating some observed false positive cases such as the wmemcmp() buffer comparison method from the C runtime being incorrectly identified as performing RC4 encryption.