lundi 24 mars 2008

New QEmu detection method found in a malware

I decided to translate a post from my colleague Fabien Perigaud, from french to english. The original post in french is here.

I wanted to share this great post with foreign readers, because this topic is always of high interest for me, as I am a heavy virtual machine user... Please apologize for the bad translation, I'm dead tired and will not re-read it I guess :-p

There it goes :

"In the world of malware, the race between malware author and anti-virus companies is famous, the first ones trying to make their creation the most invisible against the second one's solutions.
Before the so-called malware can be detected as malicious by an anti-virus product, he undergoes a strong analysis, so that his behaviour is understood. That way, a signature can be extracted of it, to allow detection, and to have a way of cleaning the infected machines.
Malware developers have got to counter the analysis tools, the first one being the virtual machine, which allows to launch the malware without any fear, because the system can be restored in its previous state in few seconds.
A new caught malware, spreading via a malicious Flash banner found on many web sites, has shown us a new method of virtual machine detection, impacting QEmu (and probably Virtual PC, and VMware without hardware acceleration).
The packer used by the malware, not identified at the time of writing, executes the "asin" function after having decrypted a part of itself. This function is part of the msvcrt library. This function will return in EAX register a different value according to the value of the Control Word register from the FPU (Typically, a 0 value will be given to EAX if FCW is not equal to 0x27F).
The FCW register undergoes some operations equivalent to a logical AND with a 0x23F mask, at the loading of the msvcrt.dll library (FPU register init). According to Intel's specs, the 6th bit of the FCW register is marked as reserved, and it has been discovered it was fixed to 1 on physical machines.

On a physical machine, FCW will therefore have the value 0x27F after the loading of the library. Unfortunately, QEmu's full emulation mode (without using kqemu) doesn't set the 6th bit to 1 ; the register takes the value of 0x23F, provoking a different result to EAX after it gets out of the asin function.
According to this value of EAX, the malware will initialise or not the variable of its decrypting loop, provoking its own crash if not set properly (writing on an unauthorized memory zone), preventing any analysis.

We can expect more and more malware use this kind of detection method, preventing this way automated analysis based on virtual machines. So the race goes on..."