ModuleMap: support packed/obfuscated Windows binaries

[BMorgan1296]

Hi,

I am having an issue where no coverage info is generated (i.e. tbcoverage-0.json is empty) due to Vmi: Could not find host path for s2e.sys, and the same error but for the binary itself.

I assume the ‘host’ here is the machine I have S2E installed on, therefore the guest tools and binary should be in the project folder? I don’t see why they wouldn’t be known to S2E during execution.

I have tried executing binaries using the following Windows images:

Windows 10 Pro 1909-x86_64 using provided s2e.sys
Windows 10 Pro 2004-x86_64 with a custom built s2e.sys for the differing kernel
Windows 7 SP1 x86_64 using provided s2e.sys

I’ve followed all the build steps to the best of my ability, everything seems to be alright. If I enable GUI when executing the binary, I can see things happening in the terminal, but I don’t see the “hello world” output due to s2e.sys not being loaded. Execution takes significantly longer as well.

Code coverage on linux binaries works fine in contrast.

Here is my warnings.txt for a small binary where every byte runs.

WebServiceInterface: SeedSearcher not present, seed statistics will not be available
WebServiceInterface: Recipe plugin not present, recipe statistics will not be available
GuestCodeHooking: You should specify modules to hook in moduleNames
25 [State 0] WindowsMonitor: Size mismatch for e1g6032e.sys
25 [State 0] WindowsMonitor: Size mismatch for swenum.sys
Vmi: Could not find host path for s2e.sys
25 [State 0] BaseInstructions: ERROR: invokePlugin could not find plugin StaticStateMerger
Vmi: Could not find host path for dpe64small.exe
32 [State 0] WindowsMonitor: Size mismatch for dpe64small.exe
Vmi: Could not find host path for dpe64small.exe
Vmi: Could not find host path for dpe64small.exe
32 [State 0] WindowsMonitor: Size mismatch for dpe64small.exe
Vmi: Could not find host path for dpe64small.exe
Vmi: Could not find host path for dpe64small.exe
Vmi: Could not find host path for dpe64small.exe
All states were terminated
Terminating node id 0 (instance slot 0)

And then for a hello world program:

WebServiceInterface: SeedSearcher not present, seed statistics will not be available
WebServiceInterface: Recipe plugin not present, recipe statistics will not be available
GuestCodeHooking: You should specify modules to hook in moduleNames
26 [State 0] WindowsMonitor: Size mismatch for e1g6032e.sys
26 [State 0] WindowsMonitor: Size mismatch for swenum.sys
Vmi: Could not find host path for s2e.sys
26 [State 0] BaseInstructions: ERROR: invokePlugin could not find plugin StaticStateMerger
All states were terminated
Terminating node id 0 (instance slot 0)

Any ideas?

[vitalych]

Vmi: Could not find host path for s2e.sys should not affect code coverage for your driver. Could you please export the project (s2e export_project -r) so I can have a look?

[BMorgan1296]

hh2.golden.tar.xz.txt
DPE64small.tar.xz.txt

Here you go, just need to remove the trailing '.txt', GitHub didn't like the tar.xz upload.

[vitalych]

For hh2, the binary has overlapping sections, the ModuleMap plugin can't keep track of them, so ModuleExecutionDetector can't figure out which section is executing, resulting in loss of coverage in TranslationBlockCoverage.

I think that the fix would be to support overlapping sections in ModuleMap.

[vitalych]

This is a quick fix: S2E/s2e#39

[vitalych]

@BMorgan1296 does the fix work for you?

[BMorgan1296]

@BMorgan1296 does the fix work for you?

The small test programs did fully complete execution, however I received no coverage for DPE64small. So better than without the fix!

I'll be trialling the plug-in changes over a larger sample size of Windows binaries soon.

[vitalych]

I see, I didn't get a chance to run on DPE64small, only hh2.

[vitalych]

I tried to run DPE64small. This one is complicated, I can't even disassemble it in IDA. It doesn't seem to have any sections. If you explain to me how Windows loads and runs such binaries, I may be able to make a fix.

[vitalych]

I'll be trialling the plug-in changes over a larger sample size of Windows binaries soon.

Let me know how this is going, on my side, the testsuite passes so I guess it's mergeable. I was thinking of adding hh2 to the testsuite, but I'd rather not include a binary there, especially when virustotal finds it suspicious. Is this some malware sample? If not, let me know if you have source code for it.

[vitalych]

By the way, you need to call your binary dpe64small.exe, lower case. See this issue: #452

[vitalych]

S2E parses the location of executable sections in the binary. If parsing fails for some reason, it will not be able to locate the binary in the address space and coverage recording will fail. S2E relies on the libvmi library to parse PE files. It has a tool you can use to dump the binaries' sections:

Here is how sections would look like on a normal binary:

(venv) vitaly@pc-thr:~/s2e/env$ ./build/libvmi-release/dump -s images/windows-xpsp3pro-i386/guestfs/windows/system32/shell32.dll

Sections (Name, Type, VirtualAddr, PhysicalAddr, FileSize, VirtSize)
====================================================================
.text                LR-X 0x007c9c1000 0x0000000000 0x00001fe400 0x00001fe3a0
.data                LRW- 0x007cbc0000 0x0000000000 0x000001a200 0x000001c210
.rsrc                LR-- 0x007cbdd000 0x0000000000 0x00005df600 0x00005df4a0
.reloc               LR-- 0x007d1bd000 0x0000000000 0x000001a600 0x000001a508

Here is how they look like on hh2.golden.exe. You can see that some values are strange. Virtual addresses and sizes seem incorrect, but S2E still tolerates it. You'd still get coverage because the start/size accidentally contain the actual binary.

(venv) vitaly@pc-thr:~/s2e/env$ ./build/libvmi-release/dump -s projects/hh2.golden/hh2.golden.exe
hh2.golden.exe: Could not read export directory
hh2.golden.exe: Could not load directory 1
hh2.golden.exe: Could not load directory 3

Sections (Name, Type, VirtualAddr, PhysicalAddr, FileSize, VirtSize)
====================================================================
                     L--- 0x00e0700020 0x0000000000 0x0015fff56a 0x0000000000
_1@                  L--- 0x006cac6548 0x0000000000 0x0057202c6f 0x0000403157
                     LR-X 0x0000400031 0x0000000000 0x0045000000 0x0072000030

For dpe64small.exe, it cannot parse anything. IDA could not open it either.

(venv) vitaly@pc-thr:~/s2e/env$ ./build/libvmi-release/dump -s projects/DPE64small/dpe64small.exe
dpe64small.exe: Could not load directory 0
dpe64small.exe: Could not read export directory
dpe64small.exe: Could not load directory 1
dpe64small.exe: Could not load directory 5
dpe64small.exe: Could not load directory 3

Sections (Name, Type, VirtualAddr, PhysicalAddr, FileSize, VirtSize)
====================================================================
No sections present

I don't know how Windows loads these binaries, maybe it uses other things than segments. You will have to explain that to me, I don't have time to investigate it myself.

[BMorgan1296]

Hi,

Found the issue I was having with code coverage! Rookie mistake. The windows binaries I was testing did not have file extensions. As soon as I appended ".exe", code coverage magically started working. Using the dump tool above showed them to look normal so I knew I could assume it wasn't an issue with the binaries themselves.

I believe this issue can be closed now. At least it was a quick fix!