NOTE: This blog is for a special topics course at Texas A&M (ML for Cyber Defenses). During each lecture a student presents information from the assigned paper. This blog summarizes and further discusses each topic.

During this seminar, Johnny Le presented Shallow Security: on the Creation of Adversarial Variants to Evade Machine Learning-Based Malware Detectors. After his presentation our class had an open discussion related to the paper and more. This blog post will cover a summary of the information presented as well as a summary of our class discussion.

Presentation Summary

Introduction

  • ML-based approaches are currently state-of-the-art techniques for malware detection
  • They are not completely perfect, stimulating the existing arms race with attackers generating malware to exploit problems in ML-based approaches, and defenders creating new models
  • Companies, such as Endgame, Inc, have formed challenges to develop better solutions to defend against adversarial attacks
    • Endgame, Inc launched a challenge to evaluate the performance of three static analysis-based ML models (with two being deep neural networks and one being a decision tree)
    • The authors of this paper participated in the challenge and bypassed all three models, while also being able to bypass real AVs as well
  • The authors found drawbacks in their evaluation of the models, which were discussed during this seminar

The Challenge

  • The competition involves completing 50 tasks by classifying distributed binaries with three models (two based on raw data and one on PE features) to earn points, with each classifier bypass earning 1 point, up to a total of 150 points
  • Participants aim to create adversarial malware that mimics the original binaries’ behavior in a sandboxed environment by modifying binary sections and adding data, ensuring they produce the same Indicators of Compromise (IoCs)
  • Despite multiple scoreboard resets since its August 2019 start and being top performers, the team prioritized sharing their insights from investigating all samples over winning the ongoing competition, focusing on the realistic assessment of third-party ML models for malware classification
  • The dataset contains 50 Portable Executable samples from 21 different malware families, aiming to represent a wide range of threats for Microsoft Windows, with an emphasis on diversity to challenge detection methods
  • Four prominent malware families are highlighted
    • Emotet (5 samples): a banking trojan that steals sensitive and private information by downloading or dropping malware which spreads to other devices on the network
    • Loki (4 samples): designed for data theft (data being passwords, login credentials, and cryptocurrency wallets) and exfiltrating the stolen data to a C&C (Command and Control) host viat HTTP
    • Ramnit (4 samples): a worm capable of stealing and exfiltrating cookines, login credentials, and files to a C&C
    • Xtrat (4 samples): allows the attacker to interact with the victim via C&C servers, and enables the attackers to manage the infected machines from any connected devices such as webcameras and microphones
  • The distribution and characteristics of these malware samples are analyzed using labels from VirusTotal, normalized by AVClass, demonstrating a variety of attack vectors and the need for diversified security approaches
  • ML models:
    • MalConv and Non-Negative MalConv Models: Utilize an end-to-end deep learning framework, with the latter having only non-negative weights, both trained on 1.1M binary files from the Ember 2018 dataset. They analyze raw bytes through a complex process involving 8-dimensional embeddings and gated 1D convolution layers to classify files’ maliciousness
    • LightGBM Model: Employs gradient boosting decision tree techniques on a feature matrix derived from binary files’ characteristics (e.g., PE headers, file size), trained using the same Ember 2018 dataset
    • Dataset Overview: The Ember 2018 dataset comprises 1.1 million binary files, split into 900K for training and 200K for testing, serving as the training foundation for all three models
    • It was observed that models which use raw data are biased towards the detection of malware samples, resulting in a high FPR

Model’s Weaknesses

  • Experiments were conducted to explore strategies for bypassing machine learning models by appending up to 5MB of random data to binary inputs
  • This strategy aimed to test the models’ ability to detect modified binaries, evaluating their effectiveness in overcoming the challenge’s restrictions
  • Models such as Malconv, are particularly vulnerable to false negatives when analyzing random data chunks, emphasizing the importance of developing and evaluating a variety of classifiers to enhance robustness, particularly for data chunks larger than 1MB
  • Appending goodware strings to malware binaries can effectively bypass classifiers that rely on feature frequency, with models based on the PE structure showing the least susceptibility to this strategy
    • Modifying binary headers by swapping malware PE features with those from harmless software can trick classifiers, exploiting Windows’ loader tolerance for ignored or corrupted PE fields
    • This method leverages Microsoft’s design choice in the Windows binary loader, which disregards certain PE fields like version numbers and checksums, allowing altered (even potentially corrupted) binaries to execute
    • The team automated the replacement of binaries using a Python script with the PEFile library
    • The script’s operation and modifications led to bypassing only six samples across all models, indicating that models based on PE features learn from aspects beyond just header values
  • Packing and unpacking samples with UPX, an open-source packer, could effectively bypass classifiers by altering the PE binary’s sections, a hypothesis supported by testing on binary samples
    • Classifiers were bypassed when strings were appended to UPX-extracted payloads, but not to directly UPX-packed payloads, indicating a bias in the model against files packed with UPX
    • An exploratory evaluation with 150 UPX-packed and 150 non-packed samples from the malshare database confirmed the model’s bias by classifying UPX-packed files as malicious
    • UPX-packed binaries are more detected by classifiers than their unpacked counterparts, suggesting a bias towards detecting UPX-packed content
    • Bypassing detection was initially attempted by extracting and modifying UPX-packed binaries, prompting further exploration of alternative approaches
    • Packing samples with TeLock, a different packer, was effective in evading all detection models by encrypting and compressing the binary in a way that classifiers could not identify, although it caused execution issues with some samples like Extreme RAT
  • Embedding samples in a non-encrypted, non-compressed dropper (Dr0p1t) successfully bypassed detection without impacting execution, but resulted in binaries larger than 5MB, violating challenge rules
  • These findings indicate the necessity of exploring diverse strategies to circumvent bias in malware detection classifiers while adhering to operational constraints

Automatic Exploitation

  • Automated exploitation of ML model weaknesses for malware detection involved generating variants that bypass detection by packing/embedding malware in files with standard headers and appending data
  • Implemented a custom dropper to circumvent file size limitations for malware distribution, avoiding existing tools that produce larger files, thus ensuring the malware remains undetected by real AV engines
  • The dropper embeds malware as a PE binary resource and operates by retrieving the binary, creating a new file, dropping the content into it, and launching a process from the dropped file
  • This technique successfully bypassed all challenges without hindering the execution of the malware samples
  • The process transforms original malware into adversarial malware by embedding it in a new file with legitimate PE headers and appending data from legitimate software (goodware) to bias classification towards goodware
  • Goodware samples from a clean Windows system are selected and their data is appended to the malware, effectively deceiving malware detection models into misclassifying the altered malware as goodware
  • The method significantly alters the classification confidence of various models, making them incorrectly identify the malware as goodware with high confidence, demonstrating the effectiveness of this adversarial technique
  • Experimentation with adversarial malware revealed that modified samples were less detectable by real antivirus (AV) engines, including those powered by machine learning (ML) models, indicating potential vulnerabilities in current malware detection strategies
  • Detection rates for these modified samples dropped significantly in some cases, highlighting the importance of developing more robust ML models for malware detection
  • Generating adversarial malware resulted in significantly larger file sizes for the modified samples, due to additional data appended to evade detection, posing a practical limitation to their deployment

Discussion Summary

  • Susceptibility to Appended Data: Simple appending of data to binaries can bypass raw data-based models, indicating their non-robustness against adversarial attacks.
  • Windows Loader Policy Exploitation: The lenient policy of the Windows loader regarding PE fields allows attackers to append content without affecting binary functionality, suggesting a need for stricter loading policies.
  • Adversarial Malware Size: Adversarial malware tend to be larger due to additional data needed for bypassing classifiers, posing distribution challenges for attackers.
  • Feature Presence vs. Frequency: To mitigate appended data impact, models should focus on the presence of features rather than their frequency, reducing the effectiveness of adversarial tactics.
  • Domain-Specific Model Biases: Models may exhibit biases, such as mistakenly flagging all UPX-packed binaries as malicious, highlighting the need for less biased, domain-specific models.
  • Adoption of Variants Robustness: Assessing ML-based detectors should include malware variants robustness to ensure effectiveness and address real-world challenges, aiming for a representation that remains invariant to adversarial manipulation
  • Reviewing file resources and embedded Portable Executable (PE) files is essential for machine learning (ML) feature extraction to detect embedded malicious payloads, addressing malware dropper deception
  • Converting malware samples into downloaders: evades static detectors by fetching the malicious payload online, highlighting the need for a comprehensive threat model beyond classifier accuracy
  • The concept of creating adversarial samples to trick malware detectors is a form of adversarial attack applicable across various domains, necessitating domain-specific bypass techniques
    • For malware detection, unlike image classifiers, adversarial malware must replicate the original malware’s actions, not its appearance, challenging the effectiveness of simple modifications like adding noise

My Thoughts

This paper was very interesting as it discussed the opposite side of cyber defenses that we have been learning: the attack. It is great to learn about the weaknesses in popular malware detection classifiers such as MalConv, Non-negative MalConv, and LibGBM. These experiments demonstrated how biases in models can be exposed. For example, the appendage of goodware on malware binaries bypass models that rely on feature frequency or employing packers like UPX and TeLock tricked models by altering the PE binary’s sections. These examples reveal a concerning bias within classifiers towards certain file characteristic. This bias, along with exploitation of the Windows loader’s leniency towards certain PE fields, underscores the necessity for a more nuanced approach in malware detection strategies. It’s evident that relying solely on feature frequency or the presence of specific binary structures can lead to substantial security oversights.

The automated generation of adversarial malware, which embeds malicious payloads within seemingly benign files, further complicates the landscape, pushing the boundaries of what constitutes effective evasion techniques. These findings call for a reevaluation of ML models used in malware detection, advocating for a shift towards more robust, less biased systems capable of withstanding the cunning adaptability of malware creators. The challenge lies not only in detecting malware but in anticipating and countering the ever-evolving methods of obfuscation and evasion employed by adversaries.

Discussion Summary

  • MalConv doesn’t extract features (it doesn’t learn features), it just utilizes the bytes and statistics to feed in as input to a neural network.
    • Each byte maps to a specific neuron, for example
    • If file size is larger than number of neurons, then there is a many to one relationship of bytes to neurons. Therefore, an attacker can just add benign bytes on top of malicious ones that all map to the same neuron
    • Attacker can just append random bytes to the end of a sample’s header, so that it remains executable but can bypass a raw data-based detector such as MalConv
  • Detectors look at how binaries look like in the end
    • Attackers can bypass this by making a malicious binary that looks like goodware in static analysis
    • The classifier just externally looks at the binary file, but with a dropper an attacker can internally add malware so that dynamically, the binary is still executable
    • Dropper is a kind of packer that embeds a file inside another (just like embedding some malware in a benign binary)
  • A solution to embedding packers is utilizing a pipeline that unpacks packed files before sending to the model
  • Packer attacks work, because operating systems don’t update the loader and tries to correct binaries which turns into binary execution
    • Therefore, sometimes the solution cannot be found in machine learning, but a different domain instead
  • Two types of ways to deliver a load to a system (two types of packers):
    • First way is to store data locally to yourself
    • Second way is storing data externally
  • How to detect if the downloader is malicious or not
    • Check if the URL is malicious

My Thoughts

Once again, the discussion assisted in reiterating and detailing important concepts from the seminar. The discussion of MalConv’s specifics helped me in understanding appending goodware from a different angle. With a large file and a set number of neurons in MalConv, there will be a many-to-one relationship where an attacker could append benign bytes which tricks the model into thinking the input is clean. Addtionally, classifiers typically scan files dynamically, so if an attacker adds malware using a dropper while ensuring the file is still executable, they are able to bypass malware detection systems. A riveting concept from this discussion, and previous discussions, is the use of a pipeline. There are so many steps involved in a malware detection system, making it infeasible to perform well with only one component. Thus, decoupling components in the pipeline such as a file unpacker can significantly boost model performance. The model will not need to consider packed or dropped files, as the file unpacker already accomplishes that.

That is all for my summary, thanks for reading!

<
Previous Post
Transcending TRANSCEND: Revisiting Malware Classification in the Presence of Concept Drift (Seminar 5.2)
>
Next Post
No Need to Teach New Tricks to Old Malware: Winning an Evasion Challenge with XOR-based Adversarial Samples (Seminar 6.2)