AI Agent for Malware Analysis: A QEMU Sandbox and LLM Reverse Engineering
Sh0ny
10 июля 20262 min read
In short
The author of the experiment on Habr tested whether an LLM agent, combined with an isolated virtual machine and traditional software analysis tools, could replace a reverse engineer during the initial investigation phase. The result is a working pipeline from sample execution to reporting via Ghidra.
An analysis of an experiment has been published on Habr, in which a combination of an isolated virtual machine, traditional malware analysis tools, and an LLM agent (Codex) was used to conduct an in-depth analysis of a specific sample. The goal is not stream analysis, but rather a quick initial response for the researcher, based on which the reverse engineer decides whether to dive deeper.
Hypothesis
The author hypothesized that, with controlled access to analysis tools, Codex would be able to conduct an initial investigation: identify compromised sections in the binary, correlate dynamic and static data, and generate conclusions and reports. The result—spoiler: he succeeded. However, the researcher retained control over key decisions: VM isolation, the selection of a reference snapshot, and the timing of the dynamic analysis.
Architecture
The system is divided into two zones—the Linux host and the guest Windows 11 VM. The main rule: as much data as possible is collected from the host side. Once the VPO is launched, the guest system is considered potentially compromised, so data from it is marked as untrusted.
Inside the VM is a minimal set of components: a launch agent, PowerShell scripts, Sysmon configuration, a sample collection directory, and a mechanism for recording basic artifacts. Network channels, shared folders, and drag-and-drop are disabled.
The main orchestration takes place on the host:
- creating a disk from a reference snapshot;
- attaching a virtual removable media device to the VM;
- launching the VM via libvirt;
- capturing PCAP;
- taking a memory dump;
- analysis using Volatility 3;
- static analysis using Ghidra with Ghidra MCP;
- generating reports.
Why QEMU/libvirt
We chose QEMU/libvirt because this combination provides low-level control over the virtual machine’s lifecycle. Memory dumps, PCAPs, and Volatility 3 analysis on the host are not a luxury but a necessity: the guest is considered fully compromised, and the task is to analyze the compromised target from the outside.
What the agent was tasked with
Codex took on the tedious but substantive work: preparing scripts, verifying the environment, running the pipeline to transfer the sample from quarantine to the VM, processing artifacts, analyzing the memory dump, importing into Ghidra, and generating reports. The author emphasizes that AI is no substitute for a specialist—especially in information security, where the AI’s work must be supervised.